【云原生 | Kubernetes 系列】--Gitops持续交付 实现从代码克隆到应用部署
1. When
when类似if的功能,为task添加执行条件,满足条件执行task
- input
- 被评估的内容,支持使用静态值或变量(Parameters或Results变量)
- 默认值为空
- operator
- 比较操作符
- 仅支持in或notin两个
- values
- 由字符串组成的列表
- 必须 定义,且不能使用空值,但允许使用静态值或变量
2. Finally
Finally task
- 用于在tasks中的各任务执行结束后运行最后的任务
- 其定义格式与tasks字段相似
- 支持嵌套定义多个Task
- 这些Task上支持使用Parameter和Results
- 支持使用When表达式
常用场景:
- 发送通知
- 将pipeline的执行结果通知相关用户
- 清理资源
- 清理和释放此前占用的资源
- 终止任务执行
3. Source to Image
Task:
- Git-clone 取得源码克隆到workspace中,workspace用pvc实现可以在不同的pipelinerun中实现共享
- genterate-build-id 生成build id
- build-to-package 基于maven将workspace克隆的代码进行构建和打包成jar,依赖于手动创建的maven cache.这样不用每次都去下载相关扩展
- image-build-and-push 根据workspace中的Dockerfile去构建镜像,将jar复制到特定目录,将jar作为默认运行环境,并将镜像推送到image repo中
- deploy-to-cluster 基于预设的配置,将构建出的image部署到k8s环境
3.1 git clone
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-clone # task的名字
spec:
description: Clone the code repository to the workspace.
params:
- name: url # git仓库的url
type: string
description: git url to clone
default: ""
- name: branch # 分支默认main
type: string
description: git branch to checkout
default: "main"
workspaces:
- name: source # workspace名字
description: The git repo will be cloned onto the volume backing this workspace
steps:
- name: git-clone
image: alpine/git:v2.36.1
script: git clone -b $(params.branch) -v $(params.url) $(workspaces.source.path)/source
3.2 build package
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: build-to-package # task 名字
spec:
description: build application and package the files to image
workspaces:
- name: source
description: The git repo that cloned onto the volume backing this workspace
steps:
- name: build
image: maven:3.8-openjdk-11-slim
workingDir: $(workspaces.source.path)/source
volumeMounts:
- name: m2
mountPath: /root/.m2
script: mvn clean install
volumes:
- name: m2
persistentVolumeClaim:
claimName: maven-cache
3.3 image build
获取kaniko镜像
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/kaniko:debug
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/kaniko:debug harbor.intra.com/kaniko/executor:debug
docker push harbor.intra.com/kaniko/executor:debug
kaniko用于实现docker in docker
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: image-build
spec:
description: package the application files to image
params:
- name: dockerfile
description: The path to the dockerfile to build (relative to the context)
default: Dockerfile
- name: image-url
description: Url of image repository
- name: image-tag
description: Tag to apply to the built image
default: latest
workspaces:
- name: source
steps:
- name: build-and-push-image
image: harbor.intra.com/kaniko/executor:debug
securityContext:
runAsUser: 0 # 以管理员权限运行
command:
- /kaniko/executor
args:
- --dockerfile=$(params.dockerfile)
- --context=$(workspaces.source.path)/source
- --no-push # 只打镜像,不推送
3.4 Pipeline
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: source-to-image
spec:
params:
- name: git-url
- name: pathToContext
description: The path to the build context, used by Kaniko - within the workspace
default: .
- name: image-url
description: Url of image repository
- name: image-tag
description: Tag to apply to the built image
workspaces:
- name: codebase
tasks:
- name: git-clone
taskRef:
name: git-clone
params:
- name: url
value: "$(params.git-url)"
workspaces:
- name: source
workspace: codebase
- name: build-to-package
taskRef:
name: build-to-package
workspaces:
- name: source
workspace: codebase
runAfter:
- git-clone
- name: image-build
taskRef:
name: image-build
params:
- name: image-url
value: "$(params.image-url)"
- name: image-tag
value: "$(params.image-tag)"
workspaces:
- name: source
workspace: codebase
runAfter:
- build-to-package
3.5 PipelineRun
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: s2i-no-push-run-00001
spec:
pipelineRef:
name: source-to-image
params:
- name: git-url
value: https://gitee.com/mageedu/spring-boot-helloWorld.git
- name: image-url
value: kurtqiu1979/spring
- name: image-tag
value: latest
workspaces:
- name: codebase
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: nfs-csi
3.6 创建pipeline
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/04-tekton-pipeline-in-practise/01-s2i-no-push# kubectl apply -f .
task.tekton.dev/git-clone created
task.tekton.dev/build-to-package created
task.tekton.dev/image-build created
pipeline.tekton.dev/source-to-image created
pipelinerun.tekton.dev/s2i-no-push-run-00001 created
4. 构建镜像并实现自动推送到仓库
创建secret,这里secret的名字一定要和pipelinerun中定义的secretName相同
kubectl create secret generic docker-config --from-file=/root/.docker/config.json
创建后可以看到这个secret
# kubectl get secrets |grep docker
docker-config Opaque 1 31s
4.1 Task git-clone
# cat 01-task-git-clone.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-clone
spec:
description: Clone the code repository to the workspace.
params:
- name: url
type: string
description: git url to clone
default: ""
- name: branch
type: string
description: git branch to checkout
default: "main"
workspaces:
- name: source
description: The git repo will be cloned onto the volume backing this workspace
steps:
- name: git-clone
image: alpine/git:v2.36.1
script: git clone -b $(params.branch) -v $(params.url) $(workspaces.source.path)/source
4.2 Task source build
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: build-to-package
spec:
description: build application and package the files to image
workspaces:
- name: source
description: The git repo that cloned onto the volume backing this workspace
steps:
- name: build
image: maven:3.8-openjdk-11-slim
workingDir: $(workspaces.source.path)/source
volumeMounts:
- name: m2
mountPath: /root/.m2
script: mvn clean install
volumes:
- name: m2
persistentVolumeClaim:
claimName: maven-cache
4.3 TaskRun
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: image-build-and-push
spec:
description: package the application files to image
params:
- name: dockerfile
description: The path to the dockerfile to build (relative to the context)
default: Dockerfile
- name: image-url
description: Url of image repository
- name: image-tag
description: Tag to apply to the built image
default: latest
workspaces:
- name: source
- name: dockerconfig
# Secret resource which contains identity to image registry
mountPath: /kaniko/.docker
steps:
- name: image-build-and-push
image: harbor.intra.com/kaniko/executor:debug
#image: gcr.io/kaniko-project/executor:debug # 这个镜像是用来做镜像打包的,已经将它下载到本地harbor上
securityContext:
runAsUser: 0
env:
- name: DOCKER_CONFIG
value: /kaniko/.docker
script: |
#!/bin/sh
echo $(params.image-url):$(params.image-tag)
/kaniko/executor --dockerfile=$(params.dockerfile) --context=$(workspaces.source.path)/source --destination=$(params.image-url):$(params.image-tag)
4.4 Pipeline
分辨运行git-clone,build-to-package,image-build-and-push三个Task
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: source-to-image
spec:
params:
- name: git-url
- name: pathToContext
description: The path to the build context, used by Kaniko - within the workspace
default: .
- name: image-url
description: Url of image repository
- name: image-tag
description: Tag to apply to the built image
workspaces:
- name: codebase
- name: docker-config
tasks:
- name: git-clone
taskRef:
name: git-clone
params:
- name: url
value: "$(params.git-url)"
workspaces:
- name: source
workspace: codebase
- name: build-to-package
taskRef:
name: build-to-package
workspaces:
- name: source
workspace: codebase
runAfter:
- git-clone
- name: image-build-and-push
taskRef:
name: image-build-and-push
params:
- name: image-url
value: "$(params.image-url)"
- name: image-tag
value: "$(params.image-tag)"
workspaces:
- name: source
workspace: codebase
- name: dockerconfig
workspace: docker-config
runAfter:
- build-to-package
4.5 PipelineRun
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: s2i-image-push-run-00016
spec:
pipelineRef:
name: source-to-image
params:
- name: git-url
value: http://192.168.31.199/deploy/deploy.git # 这里需要做公开git
- name: image-url
value: kurtqiu1979/spring
#value: harbor.intra.com/kaniko/spring # 尝试了几次使用harbor都是失败.一直没搞定,用hub.docker就问题
# harbor机器人也试过了,始终是授权失败
- name: image-tag
value: v0.9.4
workspaces:
- name: codebase
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: nfs-csi
- name: docker-config
secret:
secretName: docker-config
4.6 部署并自动构建镜像
构建之后依次运行TaskRun实现代码克隆,代码编译及镜像的打包上传
kubectl apply -f .
5. 以当前时间为Tag
在上一个示例的基础上将原来需要每次手工输入的版本号,设定为以日期-时间的格式.
5.1 Task Build Tag
这个Task定义了build tag 的id
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: generate-build-id
spec:
params:
- name: version
description: The version of the application
type: string
results:
- name: datetime
description: The current date and time
- name: buildId
description: The build ID
steps:
- name: generate-datetime
image: harbor.intra.com/admin-box:v1.2
script: |
#!/usr/bin/env bash
apk add -q tzdata
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
datetime=`date +%Y%m%d-%H%M%S`
echo -n ${datetime} | tee $(results.datetime.path)
- name: generate-buildid
image: harbor.intra.com/admin-box:v1.2
script: |
#!/usr/bin/env bash
buildDatetime=`cat $(results.datetime.path)`
buildId=$(params.version)-${buildDatetime}
echo -n ${buildId} | tee $(results.buildId.path)
5.2 Task build
Build-di的Task负责将当前时间取出,并作为buildId存储在results中供其他TaskRun调用
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: generate-build-id
spec:
params:
- name: version
description: The version of the application
type: string
results:
- name: datetime
description: The current date and time
- name: buildId
description: The build ID
steps:
- name: generate-datetime
image: harbor.intra.com/admin-box:v1.2
script: |
#!/usr/bin/env bash
apk add -q tzdata
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
datetime=`date +%Y%m%d-%H%M%S`
echo -n ${datetime} | tee $(results.datetime.path)
- name: generate-buildid
image: harbor.intra.com/admin-box:v1.2
script: |
#!/usr/bin/env bash
buildDatetime=`cat $(results.datetime.path)`
buildId=$(params.version)${buildDatetime}
echo -n ${buildId} | tee $(results.buildId.path)
5.3 TaskRun
- generate-build-id和build-to-package都在git-clone结束后运行,这就意味着这2个Task可以并行执行
- 给generate-build-id定义了runAfter,只有当generate-build-id和build-to-package都结束后才会进行镜像构建
- 使用params传递image-url
- 使用results传递buildId
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: source-to-image
spec:
params:
- name: git-url
- name: pathToContext
description: The path to the build context, used by Kaniko - within the workspace
default: .
- name: image-url
description: Url of image repository
- name: version
description: The version of the application
type: string
default: "v0.9"
workspaces:
- name: codebase
- name: docker-config
tasks:
- name: git-clone
taskRef:
name: git-clone
params:
- name: url
value: "$(params.git-url)"
workspaces:
- name: source
workspace: codebase
- name: build-to-package
taskRef:
name: build-to-package
workspaces:
- name: source
workspace: codebase
runAfter:
- git-clone
- name: generate-build-id
taskRef:
name: generate-build-id
params:
- name: version
value: "$(params.version)"
runAfter:
- git-clone
- name: image-build-and-push
taskRef:
name: image-build-and-push
params:
- name: image-url
value: "$(params.image-url)"
- name: image-tag
value: "$(tasks.generate-build-id.results.buildId)"
workspaces:
- name: source
workspace: codebase
- name: dockerconfig
workspace: docker-config
runAfter:
- generate-build-id
- build-to-package
5.4 PipelineRun
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: s2i-buildid-run-000015
spec:
pipelineRef:
name: source-to-image
params:
- name: git-url
value: http://192.168.31.199/deploy/deploy.git
- name: image-url
value: kurtqiu1979/spring
- name: version
value: v
workspaces:
- name: codebase
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: nfs-csi
- name: docker-config
secret:
secretName: docker-config
5.5 部署并更新镜像
这里可以看到,generate-build-id和build-to-package是并行的
当运行build的时候,版本号通过params的参数传递和pipelinerun进行拼接,最终将拼接的结果作为images的tag号
构建完后自动上传.版本号也是我们期望的.这样就不需要每次都手动传递版本号,且镜像是什么时候打包的也比较明晰.
6. 自动构建自动发布
其他和上一个示例一样,有区别的就是deploy部分和ServiceAccount部分
6.1 Task deploy
这里其实就是利用sed将deploy.yaml中的版本号进行替换,实现自动发布最新打包的版本
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: deploy-using-kubectl
spec:
workspaces:
- name: source
description: The git repo
params:
- name: deploy-config-file
description: The path to the yaml file to deploy within the git source
- name: image-url
description: Image name including repository
- name: image-tag
description: Image tag
steps:
- name: update-yaml
image: alpine:3.16
command: ["sed"]
args:
- "-i"
- "-e"
- "s@__IMAGE__@$(params.image-url):$(params.image-tag)@g"
- "$(workspaces.source.path)/source/deploy/$(params.deploy-config-file)"
- name: run-kubectl
image: lachlanevenson/k8s-kubectl
command: ["kubectl"]
args:
- "apply"
- "-f"
- "$(workspaces.source.path)/source/deploy/$(params.deploy-config-file)"
6.2 RBAC
这里为了方便直接授权了ClusterRoleBinding
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: helloworld-admin
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: helloworld-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: helloworld-admin
namespace: default
此时在hello命名空间下创建了clusterrolebindings
# kubectl get clusterrolebindings.rbac.authorization.k8s.io -n hello|grep hello
helloworld-admin ClusterRole/cluster-admin 19m
6.3 PipelineRun
指定Task用哪个ServiceAccount权限运行
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: s2i-buildid-run-00002
spec:
serviceAccountName: default
# 指定Task用哪个ServiceAccount权限运行,ServiceAccount在07中定义
taskRunSpecs:
- pipelineTaskName: deploy-to-cluster
taskServiceAccountName: helloworld-admin
pipelineRef:
name: source-to-image
params:
- name: git-url
value: http://192.168.31.199/deploy/deploy.git
- name: image-url
value: kurtqiu1979/spring
- name: version
value: v
workspaces:
- name: codebase
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: nfs-csi
- name: docker-config
secret:
secretName: docker-config
6.4 部署并更新
运行pipeline之后通过参数传递及关键字替换最终实现通过kubectl实现deployment中的image替换及更新
可以看到Pod中的镜像版本就是刚才发布的这个版本.
# kubectl get pods -n hello
NAME READY STATUS RESTARTS AGE
spring-boot-helloworld-74dc778dd9-6s7m6 1/1 Running 0 72s
# kubectl get svc -n hello
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
spring-boot-helloworld NodePort 10.200.225.2 <none> 80:42272/TCP 80s
# kubectl describe pods -n hello spring-boot-helloworld-74dc778dd9-6s7m6 |grep Image:
Image: kurtqiu1979/spring:v-20221107-060809
