cicd 04--构建自动化发布流程

1 简介

在实际项目中，为了提高开发人员的服务发布效率，避免用户手动build镜像、通过脚本(或kubectl)更新服务，通常需要为重要的服务构建一套自动化发布流程。

简单来说，开发人员提交代码，MR 到 master 分支，触发 build 镜像操作，并自动推送到镜像仓库，然后更新到测试或者预发布环境，用户测试ok后通知sre团队，sre团队触发更新到线上或生产环境。

本文基于该思路构建一个企业可用的自动化发布流程，分享在此处以便于有需要的i小伙伴或自己查阅学习，后续针对该类型的自动化流程的优化事宜也会补充在此处。

2 构建过程

2.1 功能说明

主要功能模块如下：

1. 清理空间(Clean workspace)
2. 下载代码(Clone Repo)
3. 同步代码(Scp to sre-cicd)
4. build镜像(Build docker images)
5. 打tag(Set tag)
6. 检查deploy(check deploy exists)
7. 发布到k8s(deploy to k8s)
8. 告警通知(Declarative: Post Actions)

2.2 jenkins & gitlab 配置

jenkins基础配置：

GitLab Repository Name: 填写gitlab 的url
Build Trigger-> Accepted Merge Request Events: True
Filter branches by regex: Target Branch Regex	填写 .*master
Secret token: 自动生成即可

gitlab 配置：

在 settings->integrations 中新加Webhooks，
URL： 填写jenkins 中build Triggers的url
Secret Token：填写jenkins中生成的 Secret token
Trigger： Merge request events（可按需增加其它方式）

涉及的参数说明：

cluster = 'yourNamespace'
namespace = 'sre-test'
deployment = 'xg-nginx-prod'
container_name = 'xg-nginx'
docker_repo = 'yourDockerRepoNamespace/yourDockerImageName'
branch = 'master'
repo_url = "git@yourGitlabUrl/test-flask.git"
work_dir = 'gitlab_test_flask'

涉及的节点:

SRE_BUILD_NODE： 用于build镜像，并将其推送到阿里云镜像仓库
slave： 该节点用于clone git代码，并将其rsync到 docker build机器上
SRE_NODE： 用于执行kubectl或helm操作

jenkinsfile 配置：

cluster = 'yourNamespace'
namespace = 'sre-test'
deployment = 'xg-nginx-prod'
container_name = 'xg-nginx'
docker_repo = 'yourDockerRepoNamespace/yourDockerImageName'
branch = 'master'
repo_url = "git@yourGitlabUrl/test-flask.git"
work_dir = 'gitlab_test_flask'

def createVersion() {
     
    return new Date().format('yyyyMMdd-HHmmss')
}
image_version = createVersion()

default_description = "${namespace}/${deployment}:${image_version}"
currentBuild.description = "${default_description}"

pipeline {
     
    agent {
      
        node {
      
            label 'SRE_BUILD_NODE' 
        } 
    }
    
    environment {
     
        para = "para_just_for_test"
    }
    
    stages {
     
        stage('Clean workspace') {
     
        agent {
      
            node {
      
                label 'SRE-Build01-Server' 
            } 
        }
         steps {
     
            sh """
            echo 'clean workspace'
            rm -fr /data/nas-sre-prod/jenkins/apps/${work_dir}/*
            """
        }
      }

      stage("Clone Repo"){
     
          agent {
      
             node {
      
                 label 'slave' 
             } 
         }
         steps {
     
            echo 'Clone repo, ${branch}'
            sh """
                pwd
                ls
            """
            deleteDir()
            dir("${work_dir}"){
     
                    git(
                        url: "${repo_url}",
                        credentialsId: '73*yourGitCredentialsId*74',
                        branch: "${branch}"
                    )
            }
         }
      }

      stage('Scp to sre-cicd') {
     
         agent {
      
             node {
      
                 label 'slave' 
             } 
         }
         steps {
     
            sh """
                pwd
                rsync -Lra ${work_dir} --exclude ${work_dir}/.git 106.yourSre-cicdMachine.148:/data/nas-sre-prod/jenkins/apps/
                if [ \$? -ne 0 ]
                then
                    echo "Found some error when copy the repo"
                fi
            """
         }
      }

      stage('Build docker images') {
     
         agent {
      
             node {
      
                 label 'SRE_BUILD_NODE' 
             } 
         }
         steps {
     
             sh """
                echo 'build docker image'
                pwd 
                ls
                cd /data/nas-sre-prod/jenkins/apps/${work_dir}/
                if [ ! -f Dockerfile ]
                then
                    echo "No available dockerfile in workspace"
                fi
                if [ ! -f build_docker.sh ]
                then
                    echo "No available build_docker.sh in workspace"
                fi
                bash build_docker.sh ${docker_repo} ${image_version}
                pwd 
                ls
            """
        }
      }

      stage('Set tag') {
     
         agent {
      
             node {
      
                 label 'slave' 
             } 
         }
         steps {
     
             dir("${work_dir}"){
     
                echo 'set tag'
                sh """
                    git tag -m "Build docker image ${image_version} for ${branch}/${image_version}" ${branch}/${image_version}`
                    git describe
                    git push origin ${branch}/${image_version}`
                """
             }
         }
      }

      stage('check deploy exists') {
     
         agent {
      
             node {
      
                 label 'SRE_NODE'
             } 
         }
         steps {
     
            echo 'Hello, check deployment ${cluster} ${namespace}/${deployment} exists'
            sh "/usr/local/bin/kubectl --kubeconfig /home/yourHome/.kube/config-${cluster} -n ${namespace} get deploy ${deployment}"
         }
      }
      
      stage('deploy to k8s') {
     
         agent {
      
             node {
      
                 label 'SRE_NODE'
             } 
         }
         steps {
     
            echo 'deploy to k8s'
            sh """
            /usr/local/bin/kubectl --kubeconfig /home/yourHome/.kube/config-${cluster} -n ${namespace} set image deployment/${deployment} ${container_name}=registry-vpc.cn-shanghai.aliyuncs.com/${docker_repo}:${image_version}
            """
         }
      }
    }
    
    post {
     
        always {
     
            echo 'I have finished'
        }
        success {
     
            echo "all in one, succeed!"
            sh """
            curl -X POST -H "Content-Type: application/json" -d '{"msg_type":"text","content":{"text":"sre_test_flask_debug notify, ${namespace}/${deployment}:${image_version}, succeed!"}}' https://open.feishu.cn/open-apis/bot/v2/hook/46**yourFeiShuGroupWebhook**05
            """
        }
        failure {
     
            echo "all in one, failed!"
            sh """
            curl -X POST -H "Content-Type: application/json" -d '{"msg_type":"text","content":{"text":"sre_test_flask_debug notify, ${namespace}/${deployment}:${image_version}, failed!"}}' https://open.feishu.cn/open-apis/bot/v2/hook/46**yourFeiShuGroupWebhook**05
            """
        }
    }

}

2.3 测试结果

jenkins 执行结果：

k8s 镜像效果：

3 注意事项

1. jenkins 中后缀匹配方式在不同版本中可能有轻微的区别，因此使用最新版本jenkins的时候需要根据实际情况对比测试。

4 说明

软件环境：
Docker: 20.10.*
K8s: 1.12
jenkins 版本:2.299
参考文档：
jenkins 官方文档