Part 1: Introduction to Django on AWS Elastic Beanstalk with Docker

Sept. 23, 2015
Django Python AWS docker

Docker Logo

Please Note:
This post appears to be over 6 months old.

In this tutorial you will learn the fundamentals of Docker by deploying a Django application to AWS Elastic Beanstalk. As we progress you will build on what you've learned while incorporating best practices to speed up your deployment. 

Why Beanstalk and Docker anyway?

Deploying any application can sometimes feel like you're playing roulette. In the past Beanstalk has been criticised for it's lack of flexibility and customisation, but in April 2014 Beanstalk introduced support for Docker and those limitations somewhat lifted.

Running Docker containers on Elastic Beanstalk allows you to define your own environment. This means you get to choose your own platform, programming language and still take advantage of Beanstalks capacity provisioning, load balancing, scaling, and application monitoring. 

Sounds good right? Well, kinda. Although Elastic Beanstalk can make deployments less painful it takes time to develop and to create your own deployment strategy. It can also be overwhelming if your new to Docker and/or AWS in general. This practical tutorial will help explain some of the fundamentals of Docker by using a boilerplate template to deploy a Django application on AWS. 

Prerequisites

This tutorial assumes you have a basic understanding of supervisor, Django/Python, running on a Linux/UNIX type system and have an AWS account. If you don't have an AWS account you can signup for their free tier.

CLI for AWS Elastic Beanstalk

Before you start you will need to pip install the Elastic Beanstalk command line interface (CLI). This allow you access to Beanstalk functionality without directly logging into the AWS Management Console:

$ pip install awsebcli

The current version as of writing is CLI EB 3.5.3 running Python 2.7.1. You can check your own version using the "eb --version" command. The Python version of CLI has no bearing on the version of Python we will be running inside your container (more on containers later).

Boilerplate Django Docker template

To begin, create a new Django project using the boilerplate from GitHub:

$ django-admin.py startproject --template=https://github.com/glynjackson/django-docker-template/zipball/master mysite

If preferred you can download the boilerplate directly here.

Important: The rest of the tutorial is based on using this boilerplate template.

Once downloaded you will find a directory containing a default Django project, Dockerfile and a Dockerrun.aws.json file. We will look at each one of these files in more detail now. 

Dockerfile

In the root of the project directory is a Dockerfile. A Dockerfile is a simple list of instructions that gets executed to build an image see Dockerfile reference. Each line in the Dockerfile starts with an 'instruction' of which gets executed in-sequence. Each instruction should be in uppercase, for example, FROMRUN or EXPOSE. Each step is described as a layer. Docker uses layers to build a single image. One advantage of this layered approach is to prevent the whole image from being rebuilt when ever a change is made. Instead, only the layer modified will be updated. This concept of layers is an important one and helps you understand how Docker differs from other solutions.

Open the Dockerfile in your favourite editor. There are some special instructions you should become familiar with. The first of which is 'FROM' which pulls your base layer. We can see an example of this in the Dockerfile:

FROM python:3.4
MAINTAINER Glyn Jackson (me@glynjackson.org)

It builds the official Python Image form Dockers public repositories which itself is built from a Debian image. The ':3.4' number at the end of the argument is called a tag and refers to the build we want to install. 

Another instruction you can see in the template is RUN. This allows you to perform shell commands. Each command is executed in a new layer, for example:

RUN apt-get update \
 && apt-get upgrade -y \
 && apt-get install -y
RUN apt-get install -y apt-utils
RUN apt-get -y install binutils libproj-dev gdal-bin

The ADD instruction copies files/directories and adds them to the filesystem of the container. Their are a couple of times you can see this in the template Dockerfie, for example below will add the local file contents to the filesystem: 

ADD . /var/projects/mysite

EXPOSE opens up ports to be publicly accessible on the container when it's run. You can expose multiple ports on a container. The example below exposes 8002 because this is the port Django will run on inside the container. It is worth noting however that on Beanstalk's single container environments only the first port you expose (8002) is mapped to port 80 using a passthrough server on the actual EC2 instance running Docker.

EXPOSE 8002

The template Dockerfile shows one CMD instruction. In fact, a Dockerfile should only ever have one CMD instruction in it. In your Dockerfile we are using CMD to run a shell script for default parameters to ENTRYPOINT. The CMD instruction in the template is executed whenever a new container is run. 

CMD ["sh", "./deploy/container_start.sh"]

This shell script does just a few things on start-up. It runs Django's migrate command and starts supervisor:

#!/bin/sh
cd /var/projects/mysite && python manage.py migrate --noinput
supervisord -n -c /etc/supervisor/supervisord.conf

This script get executed when a container is run so the filesystem is that of container. At this point you also have access to any environment variables you may have set. Hopefully you can already start to see how this could be used in a real environment. 

Quick recap

  • Instructions reference at: https://docs.docker.com/engine/reference/builder/
  • Instructions get executed in the order they appear in. 
  • The first instruction in a Dockerfile MUST be FROM (excluding any comments).
  • Beanstalk always proxies the first exposed port on a single container environment.
  • There should only be one CMD instruction in a Dockerfile.

Example Dockerfile:

# Base python 3.4 build, inspired by https://github.com/Pakebo/eb-docker-django-simple
# Python 3.4 | Django
FROM python:3.4
MAINTAINER Glyn Jackson (me@glynjackson.org)
####################################################
# Environment variables
####################################################
# Get noninteractive frontend for Debian to avoid some problems:
# debconf: unable to initialize frontend: Dialog
ENV DEBIAN_FRONTEND noninteractive
####################################################
# OS Updates and Python packages
####################################################
RUN apt-get update \
 && apt-get upgrade -y \
 && apt-get install -y
RUN apt-get install -y apt-utils
# Libs required for geospatial libraries on Debian...
RUN apt-get -y install binutils libproj-dev gdal-bin
####################################################
# A Few pip installs not commonly in requirements.txt
####################################################
RUN apt-get install -y nano wget
# build dependencies for postgres and image bindings
RUN apt-get install -y python-imaging python-psycopg2
####################################################
# setup startup script for gunicord WSGI service
####################################################
RUN groupadd webapps
RUN useradd webapp -G webapps
RUN mkdir -p /var/log/webapp/ && chown -R webapp /var/log/webapp/ && chmod -R u+rX /var/log/webapp/
RUN mkdir -p /var/run/webapp/ && chown -R webapp /var/run/webapp/ && chmod -R u+rX /var/run/webapp/
####################################################
# Install and configure supervisord
####################################################
RUN apt-get install -y supervisor
RUN mkdir -p /var/log/supervisor
ADD ./deploy/supervisor_conf.d/webapp.conf /etc/supervisor/conf.d/webapp.conf
####################################################
# Install dependencies and run scripts.
####################################################
ADD . /var/projects/mysite
WORKDIR /var/projects/mysite
RUN pip install -r requirements.txt
####################################################
# Run start.sh script when the container starts.
# Note: If you run migrations etc outside CMD, envs won't be available!
####################################################
CMD ["sh", "./deploy/container_start.sh"]
# Expose listen ports
EXPOSE 8002

Dockerfile.aws.json

This file describes how to deploy a container in AWS EB however, it's not part of the Docker specification. Later in part two of the tutorial you will learn how to use this file to pull images directly. However, we will be using it to configure AWS to built the image each time you push to Beanstalk.

{
 "AWSEBDockerrunVersion": "1",
 "Logging": "/var/eb_log"
}

Once the image has been built Beanstalk runs a separate container on each EC2 instance you have running. 

Initialise your environment

But before we can get Beanstalk to build our image we need to initialise the Elastic Beanstalk environment your application versions will sit inside. In terminal from the project root directory enter the following to initialise. 

$ eb init

If you've setup CLI correctly you will be prompted to select the region your instance will be deployed in.

Next, you will be prompted to create or select an name for your application. For the propose of this tutorial enter something like "mydemo-app ". 

Last, Elastic Beanstalk should detect the Dockerfile already included in your project directory from the boilerplate. If you're not given this option then make sure you're running from the root of the project directory. The message should read "It appears you are using Docker. Is this correct?", answer yes. You may also have to choose which platform version you are using, select "Docker 1.6.2" from the options given.

When "eb init" has finished doing its stuff, your project directory will contain a new hidden folder called ".elasticbeanstalk" with a file "config.yml". The content of config.yml looks something like this:

branch-defaults:
 master:
 environment: none
global:
 application_name: mysite
 default_ec2_keyname: name_of_your_key
 default_platform: Docker 1.6.2
 default_region: eu-west-1
 profile: glynjackson
 sc: git

Your own region, key and profile will differ from above course.

You should now have an empty application, to see it in the AWS dashboard type:

$ eb console

Looking back at the "config.yml" file take note of the "branch-defaults". You will see "master" has no environment setup yet. The environment relates to your local git branch and is also the version it deploys. At the moment you don't have anything configured for master. So lets change that...

Create an environment & first deployment

In your project directory enter the following in terminal:

$ eb create

You will be prompted with a series of questions about the environment. Follow a naming convention like "my-app-staging" or "my-app-production" which reflects the branch you will be deploying. This will make more sense when you have multiple environment deployed. Running the above command means Elastic Beanstalk will start to create your new environment for you, the first time this can take several minutes so go make a cup of tea!

If you return from that lovely cup of Earl Grey and see your environment is red (in the console), or there were errors running "eb create", you can run "eb logs" or "eb events" to view any problems that occurred during the creation of the environment. 

Common problems usually relate to bad permissions or incorrect keys, however if you find a problem with the boilerplate then please report it on GitHub.

If your environment is green you can open your newly deployed application by entering:

$ eb open 

You can also check your environment status by typing "eb status" in the command line.

Django start page screenshot

That's it! You have deployed a Django application using Docker. 

What about Nginx?

To help facilitate the traffic with a load balanced environment, Elastic Beanstalk uses a passthrough webserver on single container docker environments. Since Elastic Beanstalk already includes a webserver, it's not necessary to install Nginx on the docker container. 

To change the configuration of the Nginx server on the instance in Elastic Beanstalk, you need to include a ".config" file(s) in a folder called ".ebextensions" inside the root of mysite.

An example config that changes max client upload size would look something like this: 

01_nginx_proxy.config:

files:
 "/etc/nginx/conf.d/proxy.conf" :
 mode: "000755"
 owner: root
 group: root
 content: |
 client_max_body_size 25m;

Conclusion

You can already see that Docker can be a powerful tool, however, you may have also figured-out that Beanstalk is building the Docker image every time you deploy. One of the main advantages of using Docker is that you can build and ship anywhere. Building an image each time you deploy takes up valuable time and is counter intuitive! Thankfully, Beanstalk provide another way of running Docker by pulling a pre-built image from Docker Hub. 

In part 2 you will learn how to deploy and manage your application using a pre-built image from Docker Hub.

That's all Folks!

Thanks for reading. Let's keep in Touch:
Follow me on GitHub or Twitter @glynjackson


Glyn Jackson is a Python Nerd, consultant and developer.


Find out more