Understanding Django Deployment

This is not going to be a tutorial about how to set up a Django environment on a production server. This is just a post to make you understand what you commonly do while you set up a Django production environment.

At first, you might want to check out a couple of other articles about how to set up a Django production server. There is that SimpleIsBetterThanComplex article, DigitalOcean also has its own one. Linode provides a documentation as well. You can also find your own path. Some of the documentations on the internet about deploying Django are similar to the level of identical. They have a common pattern. My business in this post is to reveal this pattern for those who did not set up a server before.

However, firstly, let's talk about why it is not easy as it seems.

Why is runserver not good enough?

Many documentations and StackOverflow answers simply states this:

Because it is only meant to be used in development environment.

And nothing more than that. I want to expand this a little more.

Processing Problem

Let's assume you have an endpoint in your URLs as below:

# the pattern
/contract/<int:pk>/download

# the implementations
/contract/1/download
/contract/2/download
# and so on...

While the client can see the details of a so-called contract on /contract/1 page in an HTML template with a nice template, they also have the option to download the contract as a file with /contract/1/download, the view of which is supposed to generate the contract as a -for instance- PDF and serve it as application/force-download (see what it means and discussions here).

That generation on the fly -assuming we do not cache- is a problem. runserver is a single process. So, if you deploy your application with runserver, this happens:

  1. Client A requests /contract/1/download.
  2. Python is busy generating the PDF file.
  3. Client B requests home page.
  4. However, Client B needs to wait until (2) is complete and god knows when it is going to be done.

So, since runserver is a single process, one is blocked while the other requests something from the application. That is why we do not deploy with runserver.

Asset Problem

In Django, at the end of the process, what we do is to simply generate a page (to put it correctly, a response) with the data given by the client for them to consume. That's all a web framework is all about in essence, it is to make this process easier for developers.

However, some things on our application (i) do not require data to be given by the client, (ii) are thing that do not change their own content, is not generated or in other words, are static and (iii) these contents are directly served from disk.

As you might have guessed, these are the scripts, the stylesheets, fonts, images and maybe much heavier media types such as photos, audios or videos.

When you run your server with runserver, all these things are served from within Django and Python in order to keep the development process as simple as possible. However, it is not a good idea to do so in the production because what you usually want to do is to send them from your server to client as fast as possible and do not want to lay the burden to Python to handle these things because Python already deals with Django and Django deals with rendering templates, responding to another clients, doing async stuff etc..

How do We Overcome These Problems?

So, overall, what we need are;

  • A software that, upon receiving a request, (i) respond with static file if the client requests an asset, (ii) redirect the request to Django if the request is not about a static file and respond with what Django responds
  • A software that fires up multiple Django processes so that one client is not blocked while the other requests something

This is what we need and below is usually what we set up according to step-by-step guides on the internet;

Django Deployment Architecture

However, that are the units in this schema?

  • A Server: What a server software does is to respond to a client correctly. That is what it does. In the schema, I've specified Nginx, a popular server software. Another popular alternative is Apache .
  • Process Manager: A process manager software handles how many Django applications to start and wait them for a request. What it does is, if a request is received, find a free Django process, give the request info to the process, receive the result. That is what it does. supervisord is a popular process manager, but the developers have not migrated to Python 3 for years. A process manager's Python version does not supposed to match the Python version of your codebase because it is just another program. However, if you want to stick to Python 3, Circus is another alternative. It even has a web based process monitor to check your processes' health.
  • Application: It is your Django application. It could be anything else like Flask, pyramid etc.
  • wsgi Implementation/Server: Python simply cannot connect with server -Nginx- so easily. So, what wsgi implementation does is to create a bridge between server and application and understand each other. This is Gunicorn's job.

You can change any unit in this schema with alternatives but the skeleton will remain the same.

We now know what units mean in this schema and now it is time to understand it. We can interpret this schema as below;

  1. The client (outside world) requests something from the server.
  2. The request hits server, in our case, it is Ngnix .
  3. If a static content is requested...
    1. Nginx reads static content from the disk.
    2. Nginx responds to the client.
  4. If the requested resource is not a static content...
    1. Nginx requests the resource from wsgi server, in our case, it is Gunicorn .
    2. Gunicorn requests the resource from process manager, in our case, it is either supervisord or circus.
      1. Process manager finds a free Django process.
      2. Process manager requests the resource from the free Django process.
        1. Django grabs the request and generates a response.
        2. Django responds to process manager.
      3. Process manager responds to Gunicorn.
    3. Gunicorn responds to Nginx.
  5. Nginx responds to the client.

That is how request-response cycle works on a deployed Django application and that is why it is complex.

In this process, as you can see, requests regarding to static content do not reach Django, even Gunicorn. It is simply served by Nginx and it is good at serving assets.

Final Words

As I have said in the beginning, my intention is to make you understand what you do in those step-by-step guides. I would recommend you to check out the deployment articles with this knowledge again.

Self Promotion

I have a Python library project called tglogger. It is simply a logging handler that sends log records on your Python application to a Telegram chat. It has some features such as;

  • Telegram markdown formatting
  • hashtags so that you can search your chat history
  • Django integration
  • timestamp (with timezone if it is a Django project)

and many more.