k8s的咖啡伴侣 -- 自动化部署工具Drone

刚开始打算用Jenkins+shell 部署镜像到K8S，无意间看到网上推荐的drone，用了之后觉得drone和docker、K8S非常般配，Jenkins更像上一代产品。在这里分享和总结一下drone的使用过程。

通过docker-compose安装drone，和官网版本差不多，分享一个我的

#  cat drone/docker-compose.yaml
version: '2'

services:
  drone-server:
    image: drone/drone:0.8
    ports:
      - 8078:8000
      - 9000
    volumes:
      - /var/lib/drone:/var/lib/drone/
    restart: always
    environment:
      - DRONE_OPEN=true
      - DRONE_ADMIN=[git user name]
      - DRONE_HOST=[http url]
      - DRONE_GITEA=true
      - DRONE_GITEA_URL=[git url]
      - DRONE_SECRET=aHVubGlqaWRyb25lMTAw
        #   - DRONE_DATABASE_DRIVER=mysql
        #   - DRONE_DATABASE_DATASOURCE=root:123456@tcp(ip:3306)/drone?parseTime=true

  drone-agent:
    image: drone/agent:0.8
    command: agent
    restart: always
    depends_on:
      - drone-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - DRONE_DEBUG=true
      - DRONE_SERVER=drone-server:9000
      - DRONE_SECRET=aHVubGlqaWRyb25lMTAw

历史原因

• MVC中V被拆出来，作一个单独的git库。我们的项目框架是ThinkPHP3，经典的MVC模式，前端是.tpl文件，里面会用到controller传递的变量，所以部署上无法分离。
• 接着前端加入了webpack/react/vue等等技术栈，生出了10多个不同功能的git库。drone对合并git仓库并不友好，这对打包镜像带来难度。
• 我的解决方法是
  1. 把必要的2个代码库(php和tpl)通过volume cache插件，互相挂载。当php有更新时，把最新的tpl挂载过来（是最近一次tpl仓库更新完成才挂载出去的）。反之tpl更新时亦然。
  2. 对于其他git库，通过nginx反向代理到原服务器上, 后期再剥离部署。

Dockerfile

FROM registry.cn-hangzhou.aliyuncs.com/ns/alpine-php-fpm-nginx

COPY souce_code /server_root/.

php git库里的.drone.yml

#设置工作目录
workspace:
  base: /www
  path: php
pipeline:
  build: #集成前后端代码，打包镜像
      image: registry.cn-hangzhou.aliyuncs.com/ns/alpine-php-fpm-nginx
      secrets: [ docker_username, docker_password ]
      volumes:
          - /tmp/cache/tpl:/www/tpl
      commands:
          - cp -rf * /www/tpl/
  cache: #把最新的php文件挂载到宿主机
      image: drillster/drone-volume-cache
      rebuild: true
      mount:
        - /www/php
      volumes:
        - /tmp/cache:/cache
  publish-image: #调用目录下Dockerfile打包镜像，并push到远程仓库
      image: plugins/docker
      registry: registry.cn-hangzhou.aliyuncs.com
      repo: registry.cn-hangzhou.aliyuncs.com/namespace/project #远程仓库地址
      secrets: [ docker_username, docker_password ]
      tags: latest
  k8s-deploy: #调用k8s部署镜像
      image: registry.cn-hangzhou.aliyuncs.com/namespace/deploy #自己做的一个部署镜像，执行sh命令
      secrets: [ docker_username, docker_password ]
      commands:
          - deploy.sh ${DRONE_REPO_NAME}  ${DRONE_BUILD_NUMBER} ingress_name project:latest #把项目名、构建序号传递过去，便于管理版本
  notice: #企业微信通知一下
      image: clem109/drone-wechat
      secrets: [plugin_corpid, plugin_corp_secret, plugin_agent_id]
      to_user: "user token"
      title: "${DRONE_REPO_NAME} build No.${DRONE_BUILD_NUMBER} ${DRONE_PREV_BUILD_STATUS} !"
      description: "Author: ${DRONE_COMMIT_AUTHOR} \nBranch: ${DRONE_COMMIT_BRANCH} \n\n点击查看drone自动化部署报告->"
      msg_url: ${DRONE_BUILD_LINK}

branches: [ master ] #只对master代码触发部署

接着，介绍下k8s master上的相关脚本

目录结构

project
│   deployment.yaml #部署模板
│   upgrade.sh #执行部署任务，蓝绿发布
│   keepone.sh #php服务占用内存太高，只希望保持最新版本的服务，不考虑版本回退
│
└───deploys #deploys yaml logs
│   │   project.drone_num.yaml
│   │   ...
│   
└───ingress #ingress yaml 

deployment.yaml 定义了部署容器组和服务

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name:  #定义个变量，通过文本替换的方式赋值
  labels:
    app: 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: 
  template:
    metadata:
      labels:
        app: 
    spec:
      containers:
      - name: 
        image: registry.cn-hangzhou.aliyuncs.com/namespace/
        imagePullPolicy: Always #总是拉取最新的
        livenessProbe:#存活检查
          failureThreshold: 3
          httpGet:
            path: path_to_api
            port: 
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 3
        ports:
        - containerPort: 
      imagePullSecrets:
        - name:  #替换为自己的保密字典
---
apiVersion: v1
kind: Service #开个服务
metadata:
  labels:
     run: 
  name: -svc
spec:
  ports:
  - port: 
    protocol: TCP
    targetPort: 
  selector:
    app: 
  sessionAffinity: None
  type: NodePort

upgrade.sh

# 调用方式deploy.sh ${DRONE_REPO_NAME}  ${DRONE_BUILD_NUMBER} ingress_name docker_image:tag
#!/bin/bash
#只保证最新的服务存活
keep_one_live() {
        sleep 300 #给老版本的容器时间处理正在执行的进程
        wait
        sh keepone.sh $1
}

if [ ! $1 ]; then
  echo -e "\033[41;37m param1 enter a deploy name \033[0m \n"
  exit
fi
if [ ! $2 ]; then
  echo -e "\033[41;37m param2 enter a deploy version \033[0m \n"
  exit
fi
if [ ! $3 ]; then
  echo -e "\033[41;37m param3 enter ingress name or string 'none' \033[0m \n"
  exit
fi

if [ ! $4 ]; then
  echo -e "\033[41;37m param4 enter docker image name:tag \033[0m \n"
  exit
fi
LABEL=$1
VERSION=$2
INGRESS=$3
DOCKERIMAGE=$4
if [ ! $5 ]; then
  IMAGEPORT=80
else
  IMAGEPORT=$5
fi
DEPLOYMENTNAME=$1-$2
SERVICE=`echo $1-$2-svc`
DEPLOYMENTFILE="deploys/$1.$2.yaml"
INGRESSFILE="ingress/$3.yaml"
DEMO="deployment.yaml"

echo -e "\033[47;32m 1. Start to deployment:$DEPLOYMENTNAME \033[0m \n" #加个骚气的颜色 参考 https://blog.csdn.net/David_Dai_1108/article/details/70478826
cp -f $DEMO $DEPLOYMENTFILE #拷贝一份到deploys目录
sed -i "s//$DEPLOYMENTNAME/g" $DEPLOYMENTFILE #替换变量
sed -i "s//$DOCKERIMAGE/g" $DEPLOYMENTFILE
sed -i "s//$IMAGEPORT/g" $DEPLOYMENTFILE

echo -e "\033[47;32m 2. Create new service:$SERVICE \033[0m \n"
cat $DEPLOYMENTFILE
kubectl apply -f $DEPLOYMENTFILE #按模板创建容器组和服务

#此时会运行V1 V2 两个版本的容器，等V2启动成功，再切换路由，就是蓝绿发布
echo -e "\033[47;32m 3. Wait until the deployment:$DEPLOYMENTNAME is ready \033[0m \n";
READY=$(kubectl get deploy $DEPLOYMENTNAME -o json | jq '.status.conditions[] | select(.reason == "MinimumReplicasAvailable") | .status' | tr -d '"')
while [[ "$READY" != "True" ]]; do
READY=$(kubectl get deploy $DEPLOYMENTNAME -o json | jq '.status.conditions[] | select(.reason == "MinimumReplicasAvailable") | .status' | tr -d '"')
sleep 5
done

if [ $INGRESS !== "none" ]; then
  echo -e "\033[47;32m 4. Update ingress with service:$SERVICE \033[0m \n"
  LASTSVC=$(kubectl get ing $INGRESS -o json | jq '.spec|.rules|.[0]|.http|.paths|.      [0]|.backend|.serviceName' | grep  -oP "\w+(\-\w+)+")
  kubectl get ing $INGRESS -o yaml > $INGRESSFILE #把当前路由配置做个备份
  sed -i "s/$LASTSVC/$SERVICE/g" $INGRESSFILE #把v1服务替换v2服务
  kubectl replace -f $INGRESSFILE #应用当前路由配置
fi

echo -e "\033[47;32m 5. Delete old versions \033[0m \n"
keep_one_live $LABEL & #利用shell子进程，异步删除V1的容器组和服务

echo -e "\033[47;32m 6. Deployment:$DEPLOYMENTNAME job done. \033[0m \n"

keepone.sh

path="deploys"
files=$(ls $path -t) #按时间排序部署过的yaml文件
i=0
j=0
for filename in $files
do
   dp=`echo $filename  | cut -d "." -f1`
   if [ $dp == $1 ];then #只匹配相同项目名的文件
    ((i++))
    if [ $i -gt 1 ];then #排除最新的一个，保留2个版本就改为 `gt 2`
      ((j++))
      if [ $j -lt 2 ];then #这里有点绕，是为了避免删除已删除的容器组和服务
        echo "$filename deleted"
        kubectl delete -f "$path/$filename" #找到次新的yaml，删除其容器组和服务
      fi
    fi
   fi
done

---

通过以上脚本，就实现了自动化部署，drone的功能很简洁，没有一丝多余，最后放一张部署成功的截图

drone.png