pv pvc kubernetes中部署LNMP

PV PVC

PV是让kubernetes知道宿主机可以访问NFS等资源

PVC是从PV里取得一块资源让容器使用，同一个目录其实可以定义多个PV，生成多个PV后，再新建PVC，类似是随机绑定一个，实验是绑定最后生成那一个

使用PVC如果用于数据库储存数据，同样的目录第二次使用时，需要把数据清空，很容易产生冲突；

另外一个PVC目录不能用于启动多个mysql容器，其实只能生成一个，目录里生成的文件会互相冲突，其他的容器只能不断重启；

不同容器添加密码变量的方式不一样，mysql 5.6 和5.7应该是有区别的；

• nfs配置

yum install nfs-utils
vim /etc/exports  #修改配置文件
/data/k8s 172.18.13.0/24(rw,sync,no_root_squash,anonuid=1000,anongid=1000)
systemctl start nfs
systemctl enable nfs
mkdir -p  /data/k8s/discuz/{db,web} #新建目录
chmod 777 /data/k8s   #可以修改一下权限；
showmount -e 172.18.13.164  #检难nfs;

• pv和pvc

[root@master ~]# cat mypv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv001   #名字；
spec:
  capacity:
    storage: 10Gi   #容量；
  accessModes:
    - ReadWriteMany   #读写权限；
  nfs:   #nfs配置；
    path: /data/k8s
    server: 172.18.13.164  
    
[root@master ~]# cat mypvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim   #使用这个名字引用；
spec:
  accessModes:
    - ReadWriteMany   #读写权限；
  resources:
    requests: 
      storage: 8Gi   #容量；
#
#
#
#
#

• mysql的yaml文件，php跟nginx独立开来
  依次把pv, pvc, dp, svc这些yaml，kubectl create就可以；

[root@master ~]# cat mysql-pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  nfs:
    path: /data/k8s/discuz/db
    server: 172.18.13.164

[root@master ~]# cat mysql-pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mysql-myclaim
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi

• rc和svc

[root@master ~]# cat mysql-dp.yaml 
apiVersion: apps/v1 
kind: Deployment 
metadata:
  name: dz-mysql  #deploy的名字；
  labels:
    app: discuz 
spec:
  replicas: 1   #副本数；
  selector:     #标注为；
    matchLabels: 
      app: discuz 
      tier: mysql
  strategy:
    type: Recreate
  template: 
    metadata:
      labels:
        app: discuz 
        tier: mysql
    spec:
      containers:
      - image: mysql:5.6
        name: dz-mysql
        env:   #密码变量；
        - name: MYSQL_ROOT_PASSWORD
          value: "123456"
        ports:   #容器监听的端口；
        - containerPort: 3306 
          name: dz-mysql
        volumeMounts:   #pvc挂载点；
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql 
      volumes:    #pvc引用；
      - name: mysql-persistent-storage 
        persistentVolumeClaim:
          claimName: mysql-myclaim
          
[root@master ~]# cat mysql-svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: dz-mysql
  labels:
    app: discuz
spec:
  ports:
    - port: 3306   #集群端口，也映射到节点的3306端口上；
  selector:    #选择哪些容器组合成服务；
    app: discuz
    tier: mysql

#
#
#
#
#

• Dockfile修改镜像，上传到harbor，再使用新的镜像生成pod

[root@master ~]# cat Dockerfile 
FROM richarvey/nginx-php-fpm
COPY localtime /etc/localtime   #localtime文件没办法在这里提供了，不使用不知道时间会否有问题；
RUN echo 'Asia/Shanghai' >/etc/timezone
RUN rm -f /etc/nginx/nginx.conf
COPY nginx.conf /etc/nginx/nginx.conf
RUN rm -f /usr/local/etc/php-fpm.d/www.conf
COPY php-fpm-www.conf /usr/local/etc/php-fpm.d/www.conf
ENTRYPOINT /usr/sbin/nginx -c /etc/nginx/nginx.conf && /usr/local/sbin/php-fpm -c /usr/local/etc/php-fpm.conf

[root@master ~]# cat  nginx.conf php-fpm-www.conf   #这两个文件跟Dockfile放在同一个目录；build的时候会复制进镜像；
user nobody;
worker_processes 1;
error_log /var/log/nginx_error.log crit;
worker_rlimit_nofile 51200;

events
{
    use epoll;
    worker_connections 6000;
}

http
{
    include mime.types;
    default_type application/octet-stream;
    log_format main '$remote_addr $http_x_forwarded_for [$time_local]'
    '$host "$request_uri" $status'
    '"$http_referer" "$http_user_agent"';
    sendfile on;
    tcp_nopush on;
    keepalive_timeout 30;
    client_header_timeout 3m;
    client_body_timeout 3m;
    send_timeout 3m;
    connection_pool_size 256;
    client_header_buffer_size 1k;
    large_client_header_buffers 8 4k;
    request_pool_size 4k;
    output_buffers 4 32k;
    postpone_output 1460;
    client_max_body_size 10m;
    client_body_buffer_size 256k;
    fastcgi_intercept_errors on;
    tcp_nodelay on;
    gzip on;
    gzip_min_length 1k;
    gzip_buffers 4 8k;
    gzip_comp_level 5;
    gzip_http_version 1.1;
    gzip_types text/plain application/x-javascript text/css text/htm application/xml;

server
{
    listen 80;
    server_name localhost;
    index index.html index.htm index.php;
    root /var/www/html;

    location ~ \.php$ {
        include fastcgi_params;
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    }
    rewrite ^([^\.]*)/topic-(.+)\.html$ $1/portal.php?mod=topic&topic=$2 last;
    rewrite ^([^\.]*)/forum-(\w+)-([0-9]+)\.html$ $1/forum.php?mod=forumdisplay&fid=$2&page=$3 last;
    rewrite ^([^\.]*)/thread-([0-9]+)-([0-9]+)-([0-9]+)\.html$ $1/forum.php?mod=viewthread&tid=$2&extra=page%3D$4&page=$3 last;
    rewrite ^([^\.]*)/group-([0-9]+)-([0-9]+)\.html$ $1/forum.php?mod=group&fid=$2&page=$3 last;
    rewrite ^([^\.]*)/space-(username|uid)-(.+)\.html$ $1/home.php?mod=space&$2=$3 last;
    rewrite ^([^\.]*)/(fid|tid)-([0-9]+)\.html$ $1/index.php?action=$2&value=$3 last;

}

}
[www]
listen = 127.0.0.1:9000
user = nginx
group = nginx
pm = dynamic
pm.max_children = 50
pm.start_servers = 20
pm.min_spare_servers = 20
pm.max_spare_servers = 30
pm.max_requests = 500
rlimit_files = 1024

[root@master ~]# docker build -t nginx_php:1 -f Dockerfile .    #新建镜像；
Successfully built 454da47caf09
Successfully tagged nginx_php:1
[root@master ~]# docker images   
[root@master ~]# docker tag nginx_php:1  harbor.yumeshoes.cn/library/nginx_php:1   #tag;
[root@master ~]# docker push harbor.yumeshoes.cn/library/nginx_php:1   #push;
The push refers to repository [harbor.yumeshoes.cn/library/nginx_php]

• nginx_php svc k8s yaml文件

[root@master ~]# cat web-pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: web-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  nfs:
    path: /data/k8s/discuz/web
    server: 172.18.13.164
    
[root@master ~]# cat web-pvc.yaml 
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: web-claim
  labels:
    app: discuz
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi
      
[root@master ~]# cat web-dp.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dz-web
  labels:
    app: discuz
spec:
  replicas: 2
  selector:
    matchLabels:
      app: discuz
      tier: nginx-php
  template:
    metadata:
      labels:
        app: discuz
        tier: nginx-php
    spec:
      imagePullSecrets:
      - name: my-secret
      containers:
      - image: harbor.yumeshoes.cn/library/nginx_php:1
        name: dz-web
        ports:
        - containerPort: 9000
        - containerPort: 80
          name: dz-web
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/www/html/
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: web-claim
          
[root@master ~]# cat web-svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: dz-web
  labels:
    app: discuz
spec:
  type: NodePort   #会生成NodePort
  ports:
    - port: 80     
      nodePort: 30080  #映射到集群的30080端口；
  selector:
    app: discuz
    tier: nginx-php

#
#
#
#
#

• 把网页数据放到nfs里

[root@master ~]# git clone https://gitee.com/ComsenzDiscuz/DiscuzX.git  #下载discuz;
正克隆到 'DiscuzX'...
remote: Enumerating objects: 9932, done.
remote: Counting objects: 100% (9932/9932), done.
remote: Compressing objects: 100% (5604/5604), done.
remote: Total 9932 (delta 4570), reused 9367 (delta 4274)
接收对象中: 100% (9932/9932), 13.40 MiB | 7.45 MiB/s, done.
处理 delta 中: 100% (4570/4570), done.
[root@master ~]# mv DiscuzX/upload/* /data/k8s/discuz/web/  #复制；
[root@master ~]# cd /data/k8s/discuz/web/
[root@master web]# ls
admin.php  archiver     crossdomain.xml  forum.php  index.php  member.php  portal.php  source    uc_client
api        config       data             group.php  install    misc.php    robots.txt  static    uc_server
api.php    connect.php  favicon.ico      home.php   m          plugin.php  search.php  template
[root@master web]# chown -R 100 data uc_server/data/ uc_client/data/ config/  #目录属主为nginx
bash-5.0# grep nginx /etc/passwd   #容器内部查看；
nginx:x:100:101:Linux User,,,:/var/cache/nginx:/sbin/nologin

[root@master ~]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
dz-web       ClusterIP   10.68.132.206   <none>        80/TCP     107m
kubernetes   ClusterIP   10.68.0.1       <none>        443/TCP    23h
[root@master ~]# curl 10.68.132.206 -I
HTTP/1.1 302 Found
Server: nginx/1.16.1
Date: Thu, 20 Feb 2020 13:03:01 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/7.4.2
location: install/

[root@master ~]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
dz-mysql     ClusterIP   10.68.178.192   <none>        3306/TCP   5s
dz-web       ClusterIP   10.68.132.206   <none>        80/TCP     134m
kubernetes   ClusterIP   10.68.0.1       <none>        443/TCP    24h
[root@master ~]# mysql -h10.68.178.192 -uroot -p123456
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.47 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

• kubernetes服务接口

ClusterIP是pod的服务接口，集群的节点master,node都可以访问，局域网内部其他机器访问不到；

NodePort是将poo的服务接口映射到集群的某一个端口上，通过这个映射，就可以以多个IP来访问到pod的服务；
集群上任何一台机器的IP+映射端口都可以访问到pod，包括127.0.0.1, 网卡IP，还有集群机器的公网IP都可以使用；

使用上web服务时一定需要映射出来，因为需要从外部使用服务；如果是集群产生的mysql服务，不一定需要映射出去，因为使用其服务的一般是集群产生的web服务，使用ClusterIP可以访问到；

试验上如果需要把mysql映射出NodePort，登陆时记得登陆里加上-h -P -u -p这四个参数；端口号大于30000；