使用docker-compose部署nginx+gunicorn+mariadb的django应用

我们来使用 docker-compose 并基于 gunicorn + mariadb + nginx 部署一个简单的 django 博客应用；

1. docker-compose 项目的组织目录

.                                              
├── django-blog                                
│   ├── blogproject                            
│   │   ├── __init__.py                        
│   │   ├── settings.py                        
│   │   ├── urls.py                            
│   │   └── wsgi.py                            
│   ├── Dockerfile                             
│   ├── manage.py                              
│   ├── requirements.txt                       
│   └── static                                 
├── docker-compose.yaml                        
├── mysql                                      
│   ├── conf                                   
│   │   └── django-blog.cnf                    
│   └── data                                   
└── nginx                                      
    ├── conf                                   
    │   └── mysite.template                    
    ├── log                                    
    └── ssl     

2. 构建 mysql 容器

mysql/
├── conf
│   └── django-blog.cnf
└── data

• 配置数据库使用utf8mb4编码：

  # mysql/conf/django-blog.cnf
  
  [client]
  default-character-set = utf8mb4
  
  [mysql]
  default-character-set = utf8mb4
  
  [mysqld]
  character-set-client-handshake = FALSE
  character-set-server = utf8mb4
  collation-server = utf8mb4_unicode_ci
  init_connect='SET NAMES utf8mb4'

• docker-compose.yaml 中 mysql service 的配置：

  db:
      image: mariadb:10.4
      container_name: django-blog-db
      restart: always
      environment:
          MYSQL_ROOT_PASSWORD: 
          MYSQL_DATABASE: <容器启动时，自动创建的数据库名>
          MYSQL_USER: <容器启动时，自动创建的数据库用户，其对上述数据库拥有全部的权限>
          MYSQL_PASSWORD: <上述数据库用户的密码>       
      volumes:
          - ./mysql/conf:/etc/mysql/conf.d  # 挂载自定义配置目录，即上述的 mysql/conf/django-blog.cnf 中的配置
          - ./mysql/data:/var/lib/mysql  # 挂载数据目录
          - ./mysql/log:/var/log  # 挂载日志目录

• 为什么不用 mysql，而是使用 mariadb ？

  mysql:latest 默认的认证方式变为caching_sha2_password，直接使用会报错：Authentication plugin 'caching_sha2_password' cannot be loaded；

  使用 mariadb:latest 或者使用 mysql:5.7 是没有问题的；

  关于这个问题的讨论可以参考如下链接，里面提到一些规避的方法：

  • caching_sha2_password介绍：

    • https://mysqlserverteam.com/mysql-8-0-4-new-default-authentication-plugin-caching_sha2_password/

  • github 两个相关 issue 的讨论：

    • https://github.com/passbolt/passbolt_docker/issues/103
    • https://github.com/docker-library/mysql/issues/419

3. 构建 django-blog 容器

django-blog/
├── blogproject
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── Dockerfile
├── manage.py
├── requirements.txt  # 包依赖
└── static  # python manage.py collectstatic 命令收集到的静态文件的目录

• 编写 Dockerfile：

  FROM python:3.7-alpine  # 使用 alpine 版本，精简 image 体积；
  RUN echo "https://mirrors.ustc.edu.cn/alpine/latest-stable/main" > /etc/apk/repositories \
      && apk update \
      # alpine 中 mariadb 代替了 mysql: 
      && apk add mariadb-dev build-base
  RUN mkdir /code
  WORKDIR /code
  COPY requirements.txt ./
  RUN pip install --no-cache-dir -r requirements.txt

  其中， requirements.txt 内容为：

  -i https://mirrors.aliyun.com/pypi/simple
  Django==2.2.5
  django-haystack==2.8.1
  gunicorn==19.9.0
  jieba==0.39
  Markdown==3.1.1
  mysqlclient==1.4.4
  Whoosh==2.7.4

• docker-compose.yaml 中 web service 的配置：

  web:
      build: django-blog/
      image: django-blog:2.2.5  # 构建后，image 的名字，2.2.5是目前使用的 django 版本
      container_name: django-blog-web
      restart: always
      depends_on:
          - db
      volumes:
           - ./django-blog:/code  # 挂载项目代码
           # - /code/static   # 可以选择不挂载项目代码中的 static 文件夹；https://stackoverflow.com/questions/29181032/add-a-volume-to-docker-but-exclude-a-sub-folder#
      command: /bin/bash -c "python manage.py makemigrations && python manage.py migrate && gunicorn blogproject.wsgi -w 3 -k gthread -b 0.0.0.0:8000"
      ports:
          - "8000:8000"
      environment:  # settings.py 中可以使用到的环境变量
          DB_NAME: 
          DB_USER: 
          DB_PASS: 
          DB_PORT: 3306
          DJANGO_SECRET_KEY: ""

• 修改 django-blog/blogproject/settings.py 文件相关项：

  import os
  
  _env = os.environ
  
  # 使用上述定义的环境变量代替具体值
  DATABASES = {
      # 'default': {
      #     'ENGINE': 'django.db.backends.sqlite3',
      #     'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
      # }
      'default': {
          'ENGINE': 'django.db.backends.mysql',
          'NAME': _env['DB_NAME'],
          'USER': _env['DB_USER'],
          'PASSWORD': _env['DB_PASS'],
          'HOST': 'db',  # docker-compose.yaml 中数据库的服务名
          'PORT': _env['DB_PORT'],  # 默认的服务端口号
          'OPTIONS': {
              # 存储引擎启用严格模式，非法数据值被拒绝
              'init_command': "SET sql_mode='STRICT_TRANS_TABLES'",
              'charset': 'utf8mb4',
          },
      }
  }
  
  # 安全考虑，不要直接显示密钥，定义在容器的环境变量中
  SECRET_KEY = os.environ['DJANGO_SECRET_KEY']
  
  # 生产环境不要将其置为 True
  DEBUG = False

4. 构建 nginx 容器

nginx/
├── conf
│   └── mysite.template  # 用户配置
├── log
└── ssl  # 存放证书

• docker-compose.yaml 中 nginx service 的配置：

  nginx:
      image: nginx:stable  # 使用稳定版
      container_name: django-blog-nginx
      restart: always
      depends_on:
          - web
      ports:
          - 80:80
          - 443:443
      environment:  # 定义环境变量
          NGINX_HOST: luizyao.com
          NGINX_PORT: 80
          NGINX_SSL_PORT: 443
          WEB_PORT: 8000
      volumes:
          # 用户配置目录
          - ./nginx/conf/mysite.template:/etc/nginx/conf.d/mysite.template
          # ssl 证书目录
          - ./nginx/ssl:/etc/nginx/ssl
          # 静态文件，即 python manage.py collectstatic 命令收集到的静态文件的目录
          - ./django-blog/static:/data/apps/django-blog/static
          # 日志
          - ./nginx/log/:/var/log/nginx/
      command: /bin/bash -c "envsubst '$$NGINX_HOST $$NGINX_PORT $$NGINX_SSL_PORT $$WEB_PORT' < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"  # 只有这几个变量需要转义：'$$NGINX_HOST $$NGINX_PORT $$NGINX_SSL_PORT $$WEB_PORT'，否则会报错

• 编写 mysite.template 文件：

  upstream django-blog {
          server web:${WEB_PORT};
  }
  
  server {
          charset utf-8;
          listen ${NGINX_PORT};
          server_name ${NGINX_HOST};
  
          rewrite ^(.*)$ https://${server_name}$1 permanent;
  }
  
  server {
          listen ${NGINX_SSL_PORT} ssl http2 default_server;
          server_name ${NGINX_HOST};
  
          ssl_certificate /etc/nginx/ssl/1_www.luizyao.com_bundle.crt;
          ssl_certificate_key /etc/nginx/ssl/2_www.luizyao.com.key;
          ssl_session_timeout 5m;
          ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
          ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;
          ssl_prefer_server_ciphers on;
  
          location /static {
                  alias /data/apps/django-blog/static;
          }
  
          location / {
                  proxy_pass http://django-blog;
                  proxy_redirect    off;
                  proxy_set_header  Host $http_host;
                  proxy_set_header  X-Real-IP $remote_addr;
          }
  
          error_page 404 /404.html;
          location = /40x.html {}
  
          error_page 500 502 503 504 /50x.html;
          location = /50x.html {}
  }

5. docker-compose.yaml 文件

version: '3.7'  # docker 版本：18.06.0+

services:
    db:
        image: mariadb:10.4 
        container_name: django-blog-db
        restart: always
        environment:
            MYSQL_ROOT_PASSWORD: 
            MYSQL_DATABASE: <容器启动时，自动创建的数据库名>
            MYSQL_USER: <容器启动时，自动创建的数据库用户，其对上述数据库拥有全部的权限>
            MYSQL_PASSWORD: <上述数据库用户的密码>       
        volumes:
            - ./mysql/conf:/etc/mysql/conf.d  # 挂载自定义配置
            - ./mysql/data:/var/lib/mysql  # 挂载数据存储目录
            - ./mysql/log:/var/log  # 挂载日志存储目录
            
    web:
        build: django-blog/
        image: django-blog:2.2.5  # 构建后，image 的名字，2.2.5是目前使用的 django 版本
        container_name: django-blog-web
        restart: always
        depends_on:
            - db
        volumes:
             - ./django-blog:/code  # 挂载项目代码
             # - /code/static   # 可以选择不挂载项目代码中的 static 文件夹；https://stackoverflow.com/questions/29181032/add-a-volume-to-docker-but-exclude-a-sub-folder#
        command: /bin/bash -c "python manage.py makemigrations && python manage.py migrate && gunicorn blogproject.wsgi -w 3 -k gthread -b 0.0.0.0:8000"
        ports:
            - "8000:8000"
        environment:  # settings.py 中可以使用到的环境变量
            DB_NAME: 
            DB_USER: 
            DB_PASS: 
            DB_PORT: 3306
            DJANGO_SECRET_KEY: ""

    nginx:
        image: nginx:stable  # 使用稳定版
        container_name: django-blog-nginx
        restart: always
        depends_on:
            - web
        ports:
            - 80:80
            - 443:443
        environment:  # 定义环境变量
            NGINX_HOST: luizyao.com
            NGINX_PORT: 80
            NGINX_SSL_PORT: 443
            WEB_PORT: 8000
        volumes:
            # 用户配置目录
            - ./nginx/conf/mysite.template:/etc/nginx/conf.d/mysite.template
            # ssl 证书目录
            - ./nginx/ssl:/etc/nginx/ssl
            # 静态文件，即 python manage.py collectstatic 命令收集到的静态文件的目录
            - ./django-blog/static:/data/apps/django-blog/static
            # 日志
            - ./nginx/log/:/var/log/nginx/
        command: /bin/bash -c "envsubst '$$NGINX_HOST $$NGINX_PORT $$NGINX_SSL_PORT $$WEB_PORT' < /etc/nginx/conf.d/mysite.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"  # 只有这几个变量需要转义：'$$NGINX_HOST $$NGINX_PORT $$NGINX_SSL_PORT $$WEB_PORT'，否则会报错

6. docker-compose up 启动服务组

[luizyao@centos_7_6_1810 blog]$ docker-compose up
Creating network "blog_default" with the default driver
Creating django-blog-db ... done
Creating django-blog-web ... done
Creating django-blog-nginx ... done
Attaching to django-blog-db, django-blog-web, django-blog-nginx
django-blog-db | 2019-09-24  8:45:41 0 [Note] mysqld (mysqld 10.4.8-MariaDB-1:10.4.8+maria~bionic) starting as process 1 ...
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Using Linux native AIO
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Uses event mutexes
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Number of pools: 1
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Using SSE2 crc32 instructions
django-blog-db | 2019-09-24  8:45:41 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Initializing buffer pool, total size = 256M, instances = 1, chunk size = 128M
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: Completed initialization of buffer pool
django-blog-db | 2019-09-24  8:45:41 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
django-blog-db | 2019-09-24  8:45:42 0 [Note] InnoDB: 128 out of 128 rollback segments are active. django-blog-db | 2019-09-24  8:45:42 0 [Note] InnoDB: Creating shared tablespace for temporary tables
django-blog-db | 2019-09-24  8:45:42 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
django-blog-db | 2019-09-24  8:45:42 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
django-blog-db | 2019-09-24  8:45:42 0 [Note] InnoDB: Waiting for purge to start
django-blog-db | 2019-09-24  8:45:42 0 [Note] InnoDB: 10.4.8 started; log sequence number 21042225; transaction id 15378
django-blog-db | 2019-09-24  8:45:42 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
django-blog-db | 2019-09-24  8:45:42 0 [Note] Plugin 'FEEDBACK' is disabled.
django-blog-db | 2019-09-24  8:45:42 0 [Note] Server socket created on IP: '::'.
django-blog-db | 2019-09-24  8:45:42 0 [Note] Reading of all Master_info entries succeeded
django-blog-db | 2019-09-24  8:45:42 0 [Note] Added new Master_info '' to hash table
django-blog-db | 2019-09-24  8:45:42 0 [Note] mysqld: ready for connections.
django-blog-db | Version: '10.4.8-MariaDB-1:10.4.8+maria~bionic'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
django-blog-web | No changes detected
django-blog-db | 2019-09-24  8:45:43 0 [Note] InnoDB: Buffer pool(s) load completed at 190924  8:45:43
django-blog-web | Operations to perform:
django-blog-web |   Apply all migrations: admin, auth, blog, comment, contenttypes, sessions
django-blog-web | Running migrations:
django-blog-web |   No migrations to apply.
django-blog-web | [2019-09-24 08:45:44 +0000] [1] [INFO] Starting gunicorn 19.9.0
django-blog-web | [2019-09-24 08:45:44 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
django-blog-web | [2019-09-24 08:45:44 +0000] [1] [INFO] Using worker: gthread
django-blog-web | [2019-09-24 08:45:44 +0000] [10] [INFO] Booting worker with pid: 10
django-blog-web | [2019-09-24 08:45:44 +0000] [11] [INFO] Booting worker with pid: 11
django-blog-web | [2019-09-24 08:45:44 +0000] [12] [INFO] Booting worker with pid: 12