项目部署 nginx + uwsgi + flask/django 踩坑

记录一下在腾讯云上部署项目nginx + uwsgi + python-flask/django

软件版本

• python 3.6
• flask lastest
• django 1.11.8
• uwsgi 2.0.18
• nginx lastest
• anaconda 3.5.2.0

登录云之后先更新软件

$ sudo apt update
$ sudo apt upgrade

配置conda环境

# 安装conda
$ curl -O https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh

# 运行安装程序
$ bash Anaconda3-5.2.0-Linux-x86_64.sh  # 一路yes完事
# 使用sudo conda会安装在/root目录下
# 不使用sudo conda就会安装在/home目录下

# 更新环境变量
$ source ~/.bashrc
# source .zshrc

# 新建conda环境
$ conda create --name django1.11.8 python=3.6

# 激活虚拟环境
$ source activate django1.11.8

安装软件

1. 配置nginx

$ sudo apt install nginx

1. 存放在/etc/nginx下, 配置文件放在/etc/nginx/sites-available(sites-enabled)/default中
2. 日志放在了/var/log/nginx中
3. 程序文件在/usr/sbin/nginx
4. 在/etc/init.d/下创建了启动脚本nginx
5. nginx默认端口号为80

# 开启nginx服务
$ sudo /etc/init.d/nginx start

# 停止nginx服务
$ sudo /etc/init.d/nginx stop

# 重启nginx服务
# 每次修改配置文件都需要重启nginx
$ sudo /etc/init.d/nginx restart

2. 配置uwsgi

• 安装

# 安装
# 在conda环境使用pip安装uwsgi会出现莫名的ModuleImportError
$ conda install uwsgi

# 安装完毕, 测试安装是否成功
$ uwsgi --version
# uwsgi: error while loading shared libraries: libicui18n.so.58: cannot open shared object file: No such file or directory

在安装uwsgi过程中发生了一些错误, 最后发现是缺少一些so文件
以下解决方法参考于 (https://blog.csdn.net/qq_26105397/article/details/79928222)

# 首先查看uwsgi的安装位置
$ which uwsgi
# /usr/local/bin/uwsgi


# 随后使用ldd查看可执行模块的dependency
$ ldd /home/ubuntu/anaconda3/bin/uwsgi 
    linux-vdso.so.1 (0x00007ffd043b4000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f03c3470000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f03c30d2000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f03c2ece000)
    libz.so.1 => /home/ubuntu/anaconda3/bin/../lib/libz.so.1 (0x00007f03c2cb7000)
    libpcre.so.1 => /home/ubuntu/anaconda3/bin/../lib/libpcre.so.1 (0x00007f03c2a72000)
    libyaml-0.so.2 => /home/ubuntu/anaconda3/bin/../lib/libyaml-0.so.2 (0x00007f03c2854000)
    libjansson.so.4 => /home/ubuntu/anaconda3/bin/../lib/libjansson.so.4 (0x00007f03c3da3000)
    libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f03c25c7000)
    libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f03c20fc000)
    libxml2.so.2 => /home/ubuntu/anaconda3/bin/../lib/libxml2.so.2 (0x00007f03c1d95000)
    liblzma.so.5 => /home/ubuntu/anaconda3/bin/../lib/liblzma.so.5 (0x00007f03c1b6f000)
    libiconv.so.2 => /home/ubuntu/anaconda3/bin/../lib/libiconv.so.2 (0x00007f03c3cbd000)
    libicui18n.so.58 => not found 
    libicuuc.so.58 => not found
    libicudata.so.58 => not found
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f03c1937000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f03c1734000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f03c152c000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f03c113b000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f03c3b95000)
    libstdc++.so.6 => /home/ubuntu/anaconda3/bin/../lib/././libstdc++.so.6 (0x00007f03bef53000)
    libgcc_s.so.1 => /home/ubuntu/anaconda3/bin/../lib/././libgcc_s.so.1 (0x00007f03bed41000)

缺少libicui18n.so.58, libicuuc.so.58, libicudata.so.58这三个so库, 将anaconda下的so库链接到相应的系统/lib目录下(32位系统/lib, 64位系统则是相应的/lib64)

$ sudo ldd -s /home/ubuntu/anaconda3/lib/libicui18n.so.58 /lib/libicui18n.so.58
$ sudo ldd -s /home/ubuntu/anaconda3/lib/libicuuc.so.58 /lib/libicuuc.so.58
$ sudo ldd -s /home/ubuntu/anaconda3/lib/libicudata.so.58 /lib/libicudata.so.58

# 测试
$ uwsgi --version
# 2.0.18

3. flask项目测试

• 先写一个简单的flask的hello_world程序

## tmp.py
from flask import Flask
app = Flask(__name__)

app.route('/')
def hello_world():
    return "
Hello World
"

if __name__ == "__main__":
    app.run(host="0.0.0.0", debug=True)

• 首先不使用nginx, 单独使用uwsgi启动flask

# 无需配置ini, 直接使用shell 参数启动uwsgi服务
$ uwsgi --socket 0.0.0.0:5000 --protocol http -w tmp:app

可以看到使用外部浏览器访问也可以成功显示hello world， 并且再flask中会自动识别html标签

• 接下来配置nginx，首先修改nginx的配置文件

# 修改nginx的配置文件 /etc/nginx/sites-available (sites-enabled)/default
# 再server节点下添加新的location项, 指向wsgi的IP和端口

server {
    ...
        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                # try_files $uri $uri/ =404;
                uwsgi_pass 0.0.0.0:5000;
                include /etc/nginx/uwsgi_params;
        }

}

重启nginx服务

# 更改配置重启
$ sudo /etc/init.d/nginx restart
[ ok ] Restarting nginx (via systemctl): nginx.service.

使用配置文件ini代替传参

# file: mysitetest/uwsgi.ini
[uwsgi]
# 使用uwsgi接收请求
socket = 0.0.0.0:5000
# web应用的入口模块名称
module = run:app
# 程序内启用的application变量名
callable = app
vacuum = true
# 项目启动文件的目录
# django中对应wsgi.py的路径
# flask中是app文件对应的路径
wsgi-file = /home/ubuntu/run.py
# 是否开启主进程监控子进程
master = true
# 项目当前的工作目录
chdir = /home/ubuntu
# log文件的位置
logto = tmp.log
# uwsgi对应启动时的进程pid号, 结束服务时需要使用
pidfile = uwsgi.pid
# 设置内存使用上限, 参照自己服务器内存使用情况
buffer-size = 32768
# uwsgi开启的进程数, 同样参照自己服务器情况
# 如果开启master, 会开启三个进程, 其中一个进程只负责调度子进程
processes = 2
# 每个进程的线程数
threads = 2

开启服务

$ uwsgi --ini uwsgi.ini
[uWSGI] getting INI configuration from uwsgi.ini

这里nginx默认监听端口是80, 接到请求后会重定向到5000端口

4. django项目测试

• 创建新的django项目

$ django-admin startproject mysitetest

# django项目结构
$ cd mysitetest
$ tree .

django项目结构

• 新建uwsgi配置文件uwsgi.ini

$ nano uwsgi.ini
[uwsgi]
socket = 0.0.0.0:8000
processes = 3 # 进程数, 自定
threads = 2 # 线程数, 自定
pidfile = uwsgi.pid
daemonize = uwsgi.log
chdir = /home/ubuntu/mysite_test/mysitetest # 当前工作目录
wsgi-file = mysitetest/wsgi.py # django的wsgi.py的相对路径
master = true # 开启父进程

# 查看目录结构
$ tree .

• 配置nginx

$ sudo nano /etc/nginx/sites-enabled/default
# 添加下面两行
server {
        ...
        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                # try_files $uri $uri/ =404;
                uwsgi_pass 0.0.0.0:8000;
                include /etc/nginx/uwsgi_params;
        }
}

• 开启服务

$ uwsgi --ini uwsgi.ini
[uWSGI] getting INI configuration from uwsgi.ini

成功运行

# 查看当前后台运行的uwsgi进程
$ ps aux | grep "uwsgi"

uwsgi后台进程

我们可以看到有四个进程在后台运行, 因为我们在配置文件中开启了父进程master=true, 父进程负责调度子进程

# 查看目录结构
$ tree .

项目运行后的目录结构

首先项目目录中新增了pycache文件夹, 这是py文件运行后, python解释器保存的字节码, 而另外的uwsgi.pid和uwsgi.log就是我们在配置文件中设置的pidfile和daemonize, 分别保存uwsgi后台运行的进程pid号以及uwsgi运行的log日志

$ cat uwsgi.pid
11435

这个pid文件中保存着父进程的pid号, 当我们想要停止uwsgi服务时, 只需要使用uwsgi --stop uwsgi.pid 这样系统就会杀死父进程, 由于父进程负责调度另外的进程, 其他进程也会结束

$ cat uwsgi.log
*** Starting uWSGI 2.0.18 (64bit) on [Thu Jan  7 00:07:36 2021] ***
compiled with version: 7.3.0 on 09 February 2019 23:40:56
os: Linux-4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020
nodename: VM-0-17-ubuntu
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /home/ubuntu/mysite_test/mysitetest
writing pidfile to uwsgi.pid
detected binary path: /home/ubuntu/anaconda3/envs/django1.11.8/bin/uwsgi
chdir() to ~/mysite_test/mysitetest
chdir(): No such file or directory [core/uwsgi.c line 2623]
*** Starting uWSGI 2.0.18 (64bit) on [Thu Jan  7 00:11:26 2021] ***
compiled with version: 7.3.0 on 09 February 2019 23:40:56
os: Linux-4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020
nodename: VM-0-17-ubuntu
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /home/ubuntu/mysite_test/mysitetest
writing pidfile to uwsgi.pid
detected binary path: /home/ubuntu/anaconda3/envs/django1.11.8/bin/uwsgi
chdir() to /home/ubuntu/mysite_test/mysitetest
your processes number limit is 7075
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:8000 fd 3
Python version: 3.6.7 | packaged by conda-forge | (default, Nov 21 2018, 03:09:43)  [GCC 7.3.0]
Python main interpreter initialized at 0x555a259b36d0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 166752 bytes (162 KB) for 2 cores
*** Operational MODE: threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x555a259b36d0 pid: 10878 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 10878)
spawned uWSGI worker 1 (pid: 10891, cores: 2)
[pid: 10891|app: 0|req: 1/1] 111.56.143.81 () {42 vars in 753 bytes} [Wed Jan  6 16:11:32 2021] GET / => generated 1716 bytes in 8 msecs (HTTP/1.1 200) 3 headers in 95 bytes (1 switches on core 0)
SIGINT/SIGQUIT received...killing workers...
worker 1 buried after 1 seconds
goodbye to uWSGI.
*** Starting uWSGI 2.0.18 (64bit) on [Thu Jan  7 00:14:21 2021] ***
compiled with version: 7.3.0 on 09 February 2019 23:40:56
os: Linux-4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020
nodename: VM-0-17-ubuntu
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /home/ubuntu/mysite_test/mysitetest
writing pidfile to uwsgi.pid
detected binary path: /home/ubuntu/anaconda3/envs/django1.11.8/bin/uwsgi
chdir() to /home/ubuntu/mysite_test/mysitetest
your processes number limit is 7075
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address 0.0.0.0:8000 fd 3
Python version: 3.6.7 | packaged by conda-forge | (default, Nov 21 2018, 03:09:43)  [GCC 7.3.0]
Python main interpreter initialized at 0x5635857876d0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 333504 bytes (325 KB) for 6 cores
*** Operational MODE: preforking+threaded ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x5635857876d0 pid: 11435 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 11435)
spawned uWSGI worker 1 (pid: 11448, cores: 2)
spawned uWSGI worker 2 (pid: 11449, cores: 2)
spawned uWSGI worker 3 (pid: 11450, cores: 2)

这里保留着uwsgi运行输出的日志, 我们可以看到当前系统的版本, python版本以及uwsig版本等等

6. 配置nginx静态文件路径

• 配置nginx

# file: /etc/nginx/sites-available(sites-enabled)/default
# 新添加location /static 路由配置, 重定向到指定的绝对路径

server{
    location /static {
        # root static 文件夹所在的绝对路径
        root /home/path/to/mysite;
    }
}

• 配置django

# file: /mysitetest/settings.py
# 在django配置文件中新增静态文件的地址
STATICFILES_DIRS = (os.path.join(BASE_DIR, "static"), )

但是在测试时发现静态文件访问报403, 去查了一下发现需要将nginx的user改为root

# file: /etc/nginx/nginx.conf
# 更改user为root
# user www-data;
user root;

# 重启nginx服务
$ sudo /etc/init.d/nginx restart