k8s实战案例之部署Nginx+Tomcat+NFS实现动静分离

1、基于镜像分层构建及自定义镜像运行Nginx及Java服务并基于NFS实现动静分离

1.1、业务镜像设计规划

根据业务的不同，我们可以导入官方基础镜像，在官方基础镜像的基础上自定义需要用的工具和环境，然后构建成自定义出自定义基础镜像，后续再基于自定义基础镜像，来构建不同服务的基础镜像，最后基于服务的自定义基础镜像构建出对应业务镜像；最后将这些镜像上传至本地harbor仓库，然后通过k8s配置清单，将对应业务运行至k8s集群之上；

1.2、Nginx+Tomcat+NFS实现动静分离架构图

客户端通过负载均衡器的反向代理来访问k8s上的服务， nginx pod和tomcat pod 由k8s svc 资源进行关联；所有数据（静态资源和动态资源）通过存储挂载至对应pod中；nginx作为服务入口，它负责接收客户端的请求，同时响应静态资源（到存储上读取，比如js文件，css文件，图片等）；后端动态资源，由nginx将请求转发至后端tomcat server 完成（tomcat负责数据写入，比如用户的上传的图片等等）；

2、自定义centos基础镜像构建

root@k8s-master01:~/k8s-data/dockerfile/system/centos# ls
CentOS7-aliyun-Base.repo  CentOS7-aliyun-epel.repo  Dockerfile  build-command.sh  filebeat-7.12.1-x86_64.rpm
root@k8s-master01:~/k8s-data/dockerfile/system/centos# cat Dockerfile 
#自定义Centos 基础镜像
FROM centos:7.9.2009 
ADD filebeat-7.12.1-x86_64.rpm /tmp
# 添加阿里源
ADD CentOS7-aliyun-Base.repo  CentOS7-aliyun-epel.repo /etc/yum.repos.d/  
# 自定义安装工具和环境
RUN yum makecache &&yum install -y /tmp/filebeat-7.12.1-x86_64.rpm vim wget tree  lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop &&  rm -rf /etc/localtime /tmp/filebeat-7.12.1-x86_64.rpm && ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime  && useradd nginx -u 2088
root@k8s-master01:~/k8s-data/dockerfile/system/centos# cat build-command.sh 
#!/bin/bash
#docker build -t  harbor.ik8s.cc/baseimages/magedu-centos-base:7.9.2009 .
#docker push harbor.ik8s.cc/baseimages/magedu-centos-base:7.9.2009

/usr/local/bin/nerdctl build -t harbor.ik8s.cc/baseimages/magedu-centos-base:7.9.2009 .
/usr/local/bin/nerdctl push harbor.ik8s.cc/baseimages/magedu-centos-base:7.9.2009
root@k8s-master01:~/k8s-data/dockerfile/system/centos# 

2.1、构建自定义centos基础镜像

2.2、验证自定义centos基础镜像

在harbor上验证镜像是否正常上传？
 

运行镜像为容器，验证对应镜像是否有我们添加的工具和环境？

3、基于自定义centos基础镜像构建nginx镜像

root@k8s-master01:~/k8s-data/dockerfile/web/pub-images/nginx-base# ls
Dockerfile  build-command.sh  nginx-1.22.0.tar.gz
root@k8s-master01:~/k8s-data/dockerfile/web/pub-images/nginx-base# cat Dockerfile 
#Nginx Base Image
# 导入自定义centos基础镜像
FROM harbor.ik8s.cc/baseimages/magedu-centos-base:7.9.2009 
# 安装编译环境
RUN yum install -y vim wget tree  lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop
# 添加nginx源码至/usr/local/src/
ADD nginx-1.22.0.tar.gz /usr/local/src/
# 编译nginx
RUN cd /usr/local/src/nginx-1.22.0 && ./configure  && make && make install && ln -sv  /usr/local/nginx/sbin/nginx /usr/sbin/nginx  &&rm -rf /usr/local/src/nginx-1.22.0.tar.gz 
root@k8s-master01:~/k8s-data/dockerfile/web/pub-images/nginx-base# cat build-command.sh 
#!/bin/bash
#docker build -t harbor.ik8s.cc/pub-images/nginx-base:v1.18.0  .
#docker push  harbor.ik8s.cc/pub-images/nginx-base:v1.18.0

nerdctl build -t  harbor.ik8s.cc/pub-images/nginx-base:v1.22.0  .
nerdctl push harbor.ik8s.cc/pub-images/nginx-base:v1.22.0
root@k8s-master01:~/k8s-data/dockerfile/web/pub-images/nginx-base# 

3.1、构建自定义nginx基础镜像

3.2、验证自定义nginx基础镜像

验证nginx基础镜像是否上传至harbor？
 

把nginx基础镜像运行为容器，看看nginx是否正常安装？

能够将nginx基础镜像运行为容器，并在容器内部启动nginx，表示nginx基础镜像就构建好了；

3.3、构建自定义nginx业务镜像

root@k8s-master01:~/k8s-data/dockerfile/web/magedu/nginx# ls
Dockerfile  app1.tar.gz  build-command.sh  index.html  nginx.conf  webapp
root@k8s-master01:~/k8s-data/dockerfile/web/magedu/nginx# cat Dockerfile 
#Nginx 1.22.0
# 导入nginx基础镜像
FROM harbor.ik8s.cc/pub-images/nginx-base:v1.22.0 
# 添加nginx配置文件
ADD nginx.conf /usr/local/nginx/conf/nginx.conf
# 添加业务代码
ADD app1.tar.gz  /usr/local/nginx/html/webapp/
ADD index.html  /usr/local/nginx/html/index.html
# 创建静态资源挂载路径
RUN mkdir -p /usr/local/nginx/html/webapp/static /usr/local/nginx/html/webapp/images 
# 暴露端口
EXPOSE 80 443
# 运行nginx
CMD ["nginx"] 
root@k8s-master01:~/k8s-data/dockerfile/web/magedu/nginx# cat nginx.conf 
user  nginx nginx;
worker_processes  auto;
daemon off;
events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';
    #access_log  logs/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    #keepalive_timeout  0;
    keepalive_timeout  65;
    upstream  tomcat_webserver {
        server  magedu-tomcat-app1-service.magedu:80; 
    }

    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        location /webapp {
            root   html;
            index  index.html index.htm;
        }
        location /app1 {
             proxy_pass  http://tomcat_webserver;
             proxy_set_header   Host    $host;
             proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_set_header X-Real-IP $remote_addr;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}
root@k8s-master01:~/k8s-data/dockerfile/web/magedu/nginx# cat build-command.sh 
#!/bin/bash
TAG=$1
#docker build -t harbor.ik8s.cc/magedu/nginx-web1:${TAG} .
#echo "镜像构建完成，即将上传到harbor"
#sleep 1
#docker push harbor.ik8s.cc/magedu/nginx-web1:${TAG}
#echo "镜像上传到harbor完成"

nerdctl build -t harbor.ik8s.cc/magedu/nginx-web1:${TAG} .
nerdctl push harbor.ik8s.cc/magedu/nginx-web1:${TAG}
root@k8s-master01:~/k8s-data/dockerfile/web/magedu/nginx# 

上述Dockerfile中主要基于nginx基础镜像添加业务代码，添加配置，以及定义运行nginx和暴露服务端口；

3.4、验证自定义nginx业务镜像

验证nginx业务镜像是否上传至harbor？
 

运行nginx业务镜像为容器，看看对应业务是否能够正常访问？