使用Docker搭建分布式LNMP架构

最近在学习docker的时候发现网上很少有此类的资料，导致我碰到问题之后排错卡了许久，现在将问题解决了特来记录一下，也给后面学的同学做一个参考。

为了使文章看起来简单明了，我会将部分正常执行过程中的输出不贴上来（比如使用yum安装时的各种输出）。

基础环境搭建

环境：

[root@docker ~]# cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

[root@docker ~]# cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core) 

在开始装包之前建议先更换一下yum源，具体使用哪个看个人，我这里用的是OPENTUNA源

[root@docker ~]# yum repolist 
extras                                                                                        | 2.9 kB  00:00:00     
os                                                                                            | 3.6 kB  00:00:00     
updates                                                                                       | 2.9 kB  00:00:00     
(1/4): os/7/x86_64/group_gz                                                                   | 153 kB  00:00:00     
(2/4): extras/7/x86_64/primary_db                                                             | 225 kB  00:00:00     
(3/4): updates/7/x86_64/primary_db                                                            | 5.7 MB  00:00:10     
(4/4): os/7/x86_64/primary_db                                                                 | 6.1 MB  00:00:12     
repo id                                        repo name                                                       status
extras/7/x86_64                                OpenTUNA centos extras - x86_64                                    453
os/7/x86_64                                    OpenTUNA centos os - x86_64                                     10,072
updates/7/x86_64                               OpenTUNA centos updates - x86_64                                 1,729
repolist: 12,254

开始安装基础包

[root@docker ~]# yum install epel-release -y

这边建议修改一下epel的源，官方的fastestmirror这个插件我个人感觉挺鸡肋的

[root@docker ~]# sed -e 's!^metalink=!#metalink=!g' \
>     -e 's!^#baseurl=!baseurl=!g' \
>     -e 's!//download\.fedoraproject\.org/pub!//opentuna.cn!g' \
>     -e 's!http://mirrors\.tuna!https://mirrors.tuna!g' \
>     -i /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel-testing.repo
[root@docker ~]# 
[root@docker ~]# yum clean all
[root@docker ~]# yum repolist 
Loaded plugins: fastestmirror
Determining fastest mirrors
epel                                                                                          | 4.7 kB  00:00:00     
extras                                                                                        | 2.9 kB  00:00:00     
os                                                                                            | 3.6 kB  00:00:00     
updates                                                                                       | 2.9 kB  00:00:00     
(1/7): epel/x86_64/group_gz                                                                   |  96 kB  00:00:00     
(2/7): epel/x86_64/updateinfo                                                                 | 1.0 MB  00:00:00     
(3/7): os/7/x86_64/group_gz                                                                   | 153 kB  00:00:00     
(4/7): extras/7/x86_64/primary_db                                                             | 225 kB  00:00:00     
(5/7): updates/7/x86_64/primary_db                                                            | 5.7 MB  00:00:01     
(6/7): epel/x86_64/primary_db                                                                 | 6.9 MB  00:00:02     
(7/7): os/7/x86_64/primary_db                                                                 | 6.1 MB  00:00:02     
repo id                                 repo name                                                              status
epel/x86_64                             Extra Packages for Enterprise Linux 7 - x86_64                         13,558
extras/7/x86_64                         OpenTUNA centos extras - x86_64                                           453
os/7/x86_64                             OpenTUNA centos os - x86_64                                            10,072
updates/7/x86_64                        OpenTUNA centos updates - x86_64                                        1,729
repolist: 25,812
[root@docker ~]# yum install bash-completion vim lrzsz wget tree openssl-devel -y

部署Docker

安装docker的方式有很多种，大家查看官方文档就好，已经写的很清楚，我这里就使用最方便的yum安装

[root@docker ~]# yum remove docker \
>                   docker-client \
>                   docker-client-latest \
>                   docker-common \
>                   docker-latest \
>                   docker-latest-logrotate \
>                   docker-logrotate \
>                   docker-engine -y
[root@docker ~]# yum install -y yum-utils
[root@docker ~]# yum-config-manager \
>     --add-repo \
>     https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
[root@docker ~]# sed -i 's+download.docker.com+opentuna.cn/docker-ce+' /etc/yum.repos.d/docker-ce.repo    # 这一步是更换docker的国内源，可选择使用
[root@docker ~]# yum repolist 
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
docker-ce-stable                                                                              | 3.5 kB  00:00:00     
(1/2): docker-ce-stable/7/x86_64/updateinfo                                                   |   55 B  00:00:00     
(2/2): docker-ce-stable/7/x86_64/primary_db                                                   |  58 kB  00:00:00     
repo id                                      repo name                                                         status
docker-ce-stable/7/x86_64                    Docker CE Stable - x86_64                                            108
epel/x86_64                                  Extra Packages for Enterprise Linux 7 - x86_64                    13,558
extras/7/x86_64                              OpenTUNA centos extras - x86_64                                      453
os/7/x86_64                                  OpenTUNA centos os - x86_64                                       10,072
updates/7/x86_64                             OpenTUNA centos updates - x86_64                                   1,729
repolist: 25,920
[root@docker ~]# yum install docker-ce docker-ce-cli containerd.io -y   # 不建议生产使用最新版本的docker，指定版本安装请看官方wiki

启动并检查docker

[root@docker ~]# systemctl start docker
[root@docker ~]# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
b8dfde127a29: Pull complete 
Digest: sha256:308866a43596e83578c7dfa15e27a73011bdd402185a84c5cd7f32a88b501a24
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

[root@docker ~]# systemctl enable docker

出现这个界面就说明docker没有问题

安装docker-compose

[root@docker ~]# wget https://github.com/docker/compose/releases/download/1.28.5/docker-compose-$(uname -s)-$(uname -m)
[root@docker ~]# wget https://github.com/docker/compose/releases/download/1.28.5/docker-compose-$(uname -s)-$(uname -m).sha256
[root@docker ~]# ls
anaconda-ks.cfg  docker-compose-Linux-x86_64  docker-compose-Linux-x86_64.sha256
[root@docker ~]# sha256sum -c docker-compose-Linux-x86_64.sha256 
docker-compose-Linux-x86_64: OK
[root@docker ~]# chmod +x /usr/local/bin/docker-compose 
[root@docker ~]# docker-compose -v
docker-compose version 1.28.5, build c4eb3a1f

部署LNMP

以下部署步骤将使用Dockerfile和docker-compose完成，如果是刚入门的同学建议将docker-compose章节学完再来看，不然其中涉及到的部分代码可能会看不懂。

先将需要用到的镜像下载下来节约后续时间

[root@docker ~]# docker pull nginx:1.18.0
[root@docker ~]# docker pull docker pull php:7.4.16-fpm-buster
[root@docker ~]# docker pull mysql:5.7.32
[root@docker ~]# docker images
REPOSITORY    TAG                 IMAGE ID       CREATED        SIZE
php           7.4.16-fpm-buster   b566f1e25baf   7 days ago     405MB
nginx         1.18.0              b8a3d5ba01ad   7 days ago     133MB
hello-world   latest              d1165f221234   2 weeks ago    13.3kB
mysql         5.7.32              cc8775c0fe94   2 months ago   449MB

因为这个php镜像是基于Debian制作的，制作时并没有nginx这个用户，所以我们得先制作一个有nginx用户的php镜像，不然在配置时php容器将会起不来

[root@docker ~]# mkdir /opt/docker/{compose,build} -p
[root@docker ~]# mkdir /opt/docker/{compose,build} -p
[root@docker ~]# mkdir /opt/docker/build/vphp
[root@docker ~]# cat > /opt/docker/build/vphp/Dockerfile < b566f1e25baf
Step 2/4 : EXPOSE 9000
 ---> Running in a5e74c37cfd4
Removing intermediate container a5e74c37cfd4
 ---> 2420835d6dfd
Step 3/4 : RUN useradd nginx -U -M -s /sbin/nologin
 ---> Running in 6ab564183988
Removing intermediate container 6ab564183988
 ---> e7eff8b773e9
Step 4/4 : CMD ["php-fpm"]
 ---> Running in 277dba4aee12
Removing intermediate container 277dba4aee12
 ---> d1623c113b71
Successfully built d1623c113b71
Successfully tagged vphp:7.4.16-fpm

检查制作的镜像是否可用

[root@docker vphp]# docker images 
REPOSITORY    TAG                 IMAGE ID       CREATED          SIZE
vphp          7.4.16-fpm          d1623c113b71   31 seconds ago   406MB
php           7.4.16-fpm-buster   b566f1e25baf   7 days ago       405MB
nginx         1.18.0              b8a3d5ba01ad   7 days ago       133MB
hello-world   latest              d1165f221234   2 weeks ago      13.3kB
mysql         5.7.32              cc8775c0fe94   2 months ago     449MB
[root@docker vphp]# docker run -d -it vphp:7.4.16-fpm 
efc10f31798e963c8c663fe55485696dccdd7eb29a178639e9320475253bf3ce
[root@docker vphp]# docker ps
CONTAINER ID   IMAGE             COMMAND                  CREATED         STATUS         PORTS      NAMES
efc10f31798e   vphp:7.4.16-fpm   "docker-php-entrypoi…"   2 seconds ago   Up 2 seconds   9000/tcp   jovial_kare
[root@docker vphp]# docker exec -it jovial_kare bash
root@efc10f31798e:/var/www/html# 
root@efc10f31798e:/var/www/html# 
root@efc10f31798e:/var/www/html# id nginx
uid=1000(nginx) gid=1000(nginx) groups=1000(nginx)
root@efc10f31798e:/var/www/html# 
root@efc10f31798e:/var/www/html# exit

开始制作docker-compose

[root@docker vphp]# cd /opt/docker/compose/lnmp
[root@docker lnmp]# cat docker-compose.yml
version: "3.0"

services:
  web01:
    container_name: nginx01
    depends_on:
      - "db01"
      - "php01"
    image: nginx:1.18.0
    ports:
      - "8080:80"
    volumes:
      - /opt/conf/nginx/:/etc/nginx  # nginx的配置文件目录
      - /opt/html/:/usr/share/nginx/html  # nginx的网页
    #restart: always
    networks:
      - lnmp

  db01:
    container_name: mysql01
    image: mysql:5.7.32
    ports:
      - "3306:3306"
    volumes:
      - db_data:/var/lib/mysql
    #environment:
    #  - MYSQL_ROOT_PASSWORD=123.com  # 这里是指定mysql的root用户密码，两种方式都可以，env_file更安全
    env_file:
      - env.mysql
    #restart: always
    networks:
      - lnmp

  php01:
    container_name: php-fpm01
    image: vphp:7.4.16-fpm
    ports:
      - "9000:9000"
    volumes:
      - /opt/conf/php-fpm/php.ini:/usr/local/php/php.ini
      - /opt/conf/php-fpm/php-fpm.conf:/usr/local/etc/php-fpm.conf
      - /opt/conf/php-fpm/www.conf:/usr/local/etc/php-fpm.d/www.conf
      - /opt/html/:/usr/share/nginx/html
    networks:
      - lnmp
    privileged: true

  dashboard:
    container_name: "portainer"  # portainer是一个docker的web面板，可用可不用，方便查看和调试
    image: "portainer/portainer-ce:2.1.1"
    restart: "always"
    ports:
      - "8000:8000"
      - "9001:9000"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "portainer_data:/data"
    networks:
      - "lnmp"

volumes:
  db_data:
#    external:
#      false
  portainer_data:

networks:
  lnmp:
#    external:
#     false

各个配置文件的准备

其实配置文件都可以从镜像启动一个容器然后从容器里面拷贝出来再进行修改，涉及到多个配置文件的（比如nginx）建议直接复制整个目录，然后在使用时直接挂载整个目录

MySQL：数据库涉及的配置过于复杂，且针对不同的业务有不同的配置方式，这里仅使用默认配置

[root@docker lnmp]# mkdir /opt/conf/{nginx,php-fpm} -p
[root@docker lnmp]# mkdir /opt/html
[root@docker lnmp]# cat >env.mysql <

nginx

[root@docker conf]# docker images 
REPOSITORY    TAG                 IMAGE ID       CREATED             SIZE
vphp          7.4.16-fpm          d1623c113b71   About an hour ago   406MB
php           7.4.16-fpm-buster   b566f1e25baf   7 days ago          405MB
nginx         1.18.0              b8a3d5ba01ad   7 days ago          133MB
hello-world   latest              d1165f221234   2 weeks ago         13.3kB
mysql         5.7.32              cc8775c0fe94   2 months ago        449MB
[root@docker conf]# docker run -d --name nginx-test nginx:1.18.0 
b6343451051259d5af497d4d6b8e7b24ac805c2fc70a191e6cd8cda92f592dfa
[root@docker conf]# 
[root@docker conf]# docker ps -a
CONTAINER ID   IMAGE             COMMAND                  CREATED             STATUS             PORTS      NAMES
b63434510512   nginx:1.18.0      "/docker-entrypoint.…"   3 seconds ago       Up 1 second        80/tcp     nginx-test
efc10f31798e   vphp:7.4.16-fpm   "docker-php-entrypoi…"   About an hour ago   Up About an hour   9000/tcp   jovial_kare
[root@docker conf]# docker exec -it nginx-test bash
root@b63434510512:/# cd /etc/nginx/  # nginx配置文件目录，涉及到多个配置文件，这里全部复制出来
root@b63434510512:/etc/nginx# ls
conf.d	fastcgi_params	koi-utf  koi-win  mime.types  modules  nginx.conf  scgi_params	uwsgi_params  win-utf
root@b63434510512:/etc/nginx# exit
[root@docker conf]# cd nginx/
[root@docker nginx]# docker cp nginx-test:/etc/nginx .
[root@docker nginx]# mv nginx/* .
[root@docker nginx]# mv nginx /tmp/
[root@docker nginx]# ls
conf.d  fastcgi_params  koi-utf  koi-win  mime.types  modules  nginx.conf  scgi_params  uwsgi_params  win-utf
[root@docker nginx]# pwd
/opt/conf/nginx

[root@docker nginx]# egrep -v "^$|#" nginx.conf 
user  nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;
    include /etc/nginx/conf.d/*.conf;
}

这里无需修改主配置文件，需要修改一下子配置文件

[root@docker nginx]# cp conf.d/default.conf{,.default}
将default.conf文件改为以下:
[root@docker nginx]# egrep -v "^$|#" conf.d/default.conf
server {
    listen       80;
    listen  [::]:80;
    server_name  localhost;
    location / {
        root   /usr/share/nginx/html;
        index  index.php index.html index.htm;  # 此处添加index.php
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
    location ~ \.php$ {  # 修改此处后端php配置
        root           /usr/share/nginx/html;
        fastcgi_pass   php-fpm01:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
	    fastcgi_param SCRIPT_NAME  $fastcgi_script_name;
        include        fastcgi_params;
    }
}

[root@docker nginx]# tree .
.
├── conf.d
│   ├── default.conf
│   └── default.conf.default
├── fastcgi_params
├── koi-utf
├── koi-win
├── mime.types
├── modules -> /usr/lib/nginx/modules
├── nginx.conf
├── scgi_params
├── uwsgi_params
└── win-utf

php：

因php官方镜像中未包含php.ini文件，所以建议从另一台机器手动安装php-fpm，并将配置文件复制过来，其余配置文件可从docker镜像启动的容器获取

[root@docker nginx]# cd /opt/conf/php-fpm/
[root@docker php-fpm]# docker ps -a
CONTAINER ID   IMAGE             COMMAND                  CREATED        STATUS         PORTS      NAMES
b63434510512   nginx:1.18.0      "/docker-entrypoint.…"   16 hours ago   Up 2 minutes   80/tcp     nginx-test
efc10f31798e   vphp:7.4.16-fpm   "docker-php-entrypoi…"   17 hours ago   Up 2 minutes   9000/tcp   jovial_kare
[root@docker php-fpm]# docker cp jovial_kare:/usr/local/etc/php-fpm.conf .
[root@docker php-fpm]# ls
php-fpm.conf
[root@docker php-fpm]# docker cp jovial_kare:/usr/local/etc/php-fpm.d/www.conf .
[root@docker php-fpm]# ls
php-fpm.conf  www.conf
[root@docker php-fpm]# scp 10.0.0.15:/root/php.ini .
The authenticity of host '10.0.0.15 (10.0.0.15)' can't be established.
ECDSA key fingerprint is SHA256:kvqksARrKjP+IqduNZYfpmsTd7N87p64e6ooQpfifko.
ECDSA key fingerprint is MD5:5f:76:16:60:62:2f:7f:44:bf:24:e5:7c:b9:84:b1:74.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.0.15' (ECDSA) to the list of known hosts.
[email protected]'s password: 
php.ini                                                                                                                                   100%   71KB  22.9MB/s   00:00    
[root@docker php-fpm]# ls
php-fpm.conf  php.ini  www.conf
[root@docker php-fpm]# sed -e "s|www-data|nginx|g" -e "s|127.0.0.1:9000|9000|g" -i www.conf

准备网页测试文件

[root@docker php-fpm]# cd /opt/html/
[root@docker html]# cat > index.php< 
> EOF
[root@docker html]# ls
index.php

启动

启动docker-compose

[root@docker html]# cd /opt/docker/compose/lnmp/
[root@docker lnmp]# ls
docker-compose.yml  env.mysql
[root@docker lnmp]# docker-compose up -d 
Creating network "lnmp_lnmp" with the default driver
Creating volume "lnmp_db_data" with default driver
Creating volume "lnmp_portainer_data" with default driver
Pulling dashboard (portainer/portainer-ce:2.1.1)...
2.1.1: Pulling from portainer/portainer-ce
94cfa856b2b1: Pull complete
49d59ee0881a: Pull complete
527b866940d5: Pull complete
Digest: sha256:5064d8414091c175c55ef6f8744da1210819388c2136273b4607a629b7d93358
Status: Downloaded newer image for portainer/portainer-ce:2.1.1
Creating mysql01   ... done
Creating portainer ... done
Creating php-fpm01 ... done
Creating nginx01   ... done

使用浏览器无痕模式访问docker机器的外网地址

[root@docker lnmp]# ip a s ens34
3: ens34:  mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:cf:e6:33 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.198/24 brd 192.168.0.255 scope global noprefixroute dynamic ens34
       valid_lft 84354sec preferred_lft 84354sec
    inet6 fe80::6e1f:8dbb:8375:1c39/64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::1267:5ed8:8ad8:29f6/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

因为之前在配置docker-compose时配置的端口映射为容器的80端口映射到宿主机的8080端口，所以在访问时要加上端口号

显示以下phpinfo则表示服务部署成功