使用Tekton 创建CI/CD 流水线

什么是Tekton?

Tekton是用于构建CI / CD管道的云原生解决方案。它由提供构建模块的Tekton Pipelines和使Tekton成为完整生态系统的支持组件（如Tekton CLI和Tekton Catalog）组成。Tekton是CD Foundation（ Linux Foundation项目）的一部分。

谁在使用Tekton?

Tekton用户通常分为以下几类：

• 为组织中的开发人员构建CI / CD系统的平台工程师。
• 使用这些CI / CD系统进行工作的开发人员。

Tekton的优势？

• 可定制的。Tekton实体是完全可定制的，因此具有高度的灵活性。平台工程师可以定义非常详细的构建目录，以供开发人员在各种场景中使用。
• 可重复使用的。Tekton实体是完全可移植的，因此一旦定义，组织内的任何人都可以使用现有的管道并复用其组成模块。这使开发人员可以快速构建复杂的pipeline，而无需“重新造轮子”。
• 可扩展。Tekton Catalog是一个由社区驱动的用于存放tekton构建模块的仓库。使开发人员可以通过Tekton仓库中现有组件快速构建新的或者按需扩展已有的pipeline。
• 标准化。Tekton将在您的Kubernetes集群上安装并作为扩展运行，并使用完善的Kubernetes资源模型。Tekton工作负载在Kubernetes容器中执行。
• 可扩展 要增加工作负载容量，您可以简单地将节点添加到现有集群中。Tekton可以伴随着集群扩展，而无需重新定义资源分配或者对pipeline做任何其他修改。

Tekton的组成部分是什么？

• Tekton由以下组件组成：
  • Tekton Pipelines是Tekton的基础。它定义了一组KubernetesCustom Resources，它们充当构建模块的基础，开发者可以由此组装CI / CD pipelines。
  • Tekton Triggers允许您根据事件实例化pipelines。例如，每次PR与GitHub存储库合并时，您都可以触发pipeline的实例化和执行。您还可以构建一个用于启动特定Tekton触发器的用户界面。
  • Tekton CLItkn在Kubernetes CLI的基础上提供了一个名为的命令行界面，使您可以与Tekton进行交互。
  • Tekton Dashboard是Tekton Pipelines的Web服务，显示有关pipeline执行的信息。目前正在开发中。
  • Tekton Catalog是一个由社区提供的高质量Tekton构建模块-Tasks，Pipelines等等的存储库，它们可以在您自己的pipelines中使用。
  • Tekton Hub是用于访问Tekton目录的Web服务。
  • Tekton Operator是由tekton开发的Operator用于在集群中执行创建，更新和删除Tekton项目。

在所有组件中，管道提供Tekton的核心功能，并为其他组件奠定基础。安装触发器，CLI和仪表板是可选的;您可以将它们与管道一起设置，以创建最适合您的团队和项目的CI/CD工作流。

Tekton Pipelines

Tekton pipeline是一个Kubernetes扩展，可以安装和运行在Kubernetes集群上。它定义了一组Kubernetes自定义资源，作为构建块，您可以从中组装CI/CD管道。一旦安装完毕，Tekton pipeline就可以通过Kubernetes CLI (kubectl)和API调用使用，就像pod和其他资源一样。

Tekton Pipelines entities

Tekton Pipelines 定义了以下对象：

对象	描述
Task	定义了一系列步骤，这些步骤将启动特定的构建或交付工具。这些工具接收特定的输入以及生成特定的输出。
TaskRun	根据指定的输入输出以及执行参数实例化一个Task并执行，可以单独调用，也可以作为Pipeline的一部分调用。
Pipeline	定义一系列任务完成一个特定的构建或交付目标，可以由事件触发或从pipelineRun调用。
PipelineRun	根据指定的输入输出以及执行参数实例化一个Pipeline并执行。
PipelineResource	定义每一步Task中提取输入以及产生输出的位置。

部署

部署环境

k8s	v1.19.2
tekton cli	v0.18.0
tekton pipelines	v0.24.1
tekton triggers	v0.14.1
tekton dashboard	v0.17.0
tekton-triggers-core-interceptors	v0.14.1

由于gcr.io镜像拉取的问题，已经将所有tekton需要的镜像上传至阿里云仓库。所有的yaml也都做了替换存放到github。

Tekton Pipelines

kubectl apply -f https://raw.githubusercontent.com/gsakun/tekton-example/master/deploy/tekton-pipeline-release.yaml

Tekton Triggers

# triggers
kubectl apply -f https://raw.githubusercontent.com/gsakun/tekton-example/master/deploy/tekton-triggers-release.yaml

# interceptors
kubectl apply -f https://raw.githubusercontent.com/gsakun/tekton-example/master/deploy/tekton-core-interceptors.yaml

Tekton Dashboard

kubectl apply -f https://raw.githubusercontent.com/gsakun/tekton-example/master/deploy/tekton-dashboard-release.yaml

Tekton Cli

 wget https://github.com/tektoncd/cli/releases/download/v0.18.0/tkn_0.18.0_Linux_x86_64.tar.gz && tar xvzf tkn_0.18.0_Linux_x86_64.tar.gz -C /usr/local/bin/ tkn

[root@cilium-1 ~]# kubectl get po -n tekton-pipelines
NAME                                           READY   STATUS    RESTARTS   AGE
tekton-dashboard-544b4b699c-q66xg                    1/1     Running   0          2m
tekton-pipelines-controller-54674677dc-s6xrc         1/1     Running   0          2m
tekton-pipelines-webhook-6bbdc498c5-ktrl8            1/1     Running   0          2m
tekton-triggers-controller-6795d464b7-thvpc          1/1     Running   0          2m
tekton-triggers-core-interceptors-688b5f659d-95dtw   1/1     Running   0          2m
tekton-triggers-webhook-6f675db95d-q5tfv             1/1     Running   0          2m

[root@cilium-1 ~]# kubectl get svc -n tekton-pipelines
NAME                                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                              AGE
tekton-dashboard                    ClusterIP   10.99.173.100    <none>        9097/TCP                             3m
tekton-pipelines-controller         ClusterIP   10.99.212.178    <none>        9090/TCP,8080/TCP                    3m
tekton-pipelines-webhook            ClusterIP   10.109.237.232   <none>        9090/TCP,8008/TCP,443/TCP,8080/TCP   3m
tekton-triggers-controller          ClusterIP   10.105.248.88    <none>        9000/TCP                             3m
tekton-triggers-core-interceptors   ClusterIP   10.96.192.131    <none>        80/TCP                               3m
tekton-triggers-webhook             ClusterIP   10.96.119.238    <none>        443/TCP                              3m

[root@cilium-1 ~]# tkn version
Client version: 0.18.0
Pipeline version: v0.24.1
Triggers version: v0.14.1
Dashboard version: v0.17.0

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
  name: tekton-dashboard
  namespace: tekton-pipelines
spec:
  rules:
  - host: tekton.xxx.xxx
    http:
      paths:
      - backend:
          serviceName: tekton-dashboard
          servicePort: 9097
        path: /
        pathType: Prefix

示例

一、Hello World

• 创建一个Task

任务定义了一系列按所需顺序运行并完成一定数量的构建工作的步骤。每个任务都在Kubernetes集群上作为Pod运行，每个步骤都作为自己的容器。例如，以下任务输出“ Hello World”：

cat <<EOF | kubectl create -f - 
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: echo-hello-world
spec:
  steps:
    - name: echo
      image: ubuntu
      command:
        - echo
      args:
        - "Hello World"
EOF

[root@cilium-1 zk]# tkn task list
NAME               DESCRIPTION   AGE
echo-hello-world                 10 seconds ago

• 创建一个TaskRun

cat <<EOF | kubectl create -f -
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: echo-hello-world-task-run
spec:
  taskRef:
    name: echo-hello-world
EOF

[root@cilium-1 zk]# tkn taskrun list
NAME                        STARTED         DURATION   STATUS
echo-hello-world-task-run   6 seconds ago   ---        Running(Pending)
[root@cilium-1 zk]# tkn taskrun list
NAME                        STARTED          DURATION      STATUS
echo-hello-world-task-run   10 seconds ago   -12 seconds   Succeeded

二、创建任务从clone代码到编译生成镜像推送镜像仓库

该示例演示创建一个流水线从gitlab clone golang项目指定分支代码 编译并build image，上传到内网harbor镜像仓库。

创建pipeline resource

git-resource

# git-resource.yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: git
  namespace: tekton-test
spec:
  type: git
  params:
    - name: revision # 指定分支
      value: master
    - name: url # 指定git仓库
      value: https://git.xxx.xxx/xxxxx/xxxxxx.git
    - name: sslVerify # 配置是否跳过ssl验证 一般内网https私仓需要配置
      value: "false"

harbor-resource

# harbor-resource.yaml
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: image
  namespace: tekton-test
spec:
  type: image
  params:
    - name: url
      value: harbor.xxx.xxx/xxxxxx/xxxxxx:latest # 配置镜像名 镜像名按照harbor仓库规范格式化命名 harbor-url/project-name/imagename:tagname 需要注意project需要提前创建好

创建Secret

创建secret 存放gitlab 以及 harbor用户名密码

# git-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: git-user-pass
  namespace: tekton-test
  annotations:
    tekton.dev/git-0: https://git.xxx.xxx #通过注释关联指定的gitlab
type: kubernetes.io/basic-auth
stringData:
  username: username # 用户名
  password: password # 密码

# harbor-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: harbor-user-pass
  namespace: tekton-test
  annotations:
    tekton.dev/docker-0: https://harbor.xxx.xxx #通过注释关联指定的仓库
type: kubernetes.io/basic-auth
stringData:
  username: username # 用户名
  password: password # 密码

此处需要注意一点，由于本公司gitlab 以及harbor 独立于测试机器之外 容器内无法解析内网域名。可通过集群内coredns添加上游dns服务器代理。集体操作修改coredns configmap 然后删除所有的coredns容器等待重新启动。

kubectl edit cm coredns -n kube-system
# 修改后的文件如下
apiVersion: v1
data:
  Corefile: |
    .:53 {
        errors
        health {
           lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
           pods insecure
           fallthrough in-addr.arpa ip6.arpa
           ttl 30
        }
        prometheus :9153
        forward . /etc/resolv.conf {
           max_concurrent 1000
        }
        cache 30
        loop
        reload
        loadbalance
    }
    # 添加以xxx.xxx后缀的域名解析代理到192.180.0.1 如有需要此处根据自己公司的dns服务器地址填写
    xxx.xxx:53 {
       errors
       cache 30
       forward . 192.168.0.1
    }

创建serviceaccount

# sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tekton-test
  namespace: tekton-test
secrets:   # 关联创建的secret tekton会自动将相关配置挂载到容器中
  - name: git-user-pass 
  - name: harbor-user-pass 

创建Task

# task.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: build-docker-image-from-git-source
  namespace: tekton-test
spec:
  params: # 创建变量 为任务中提供参数
    - name: pathToDockerFile # 指定dockerfile路径
      type: string
      description: The path to the dockerfile to build
      default: $(resources.inputs.docker-source.path)/Dockerfile
    - name: pathToContext # 指定项目路径
      type: string
      description: |
        The build context used by Kaniko
      default: $(resources.inputs.docker-source.path)
  resources: # 创建resource
    inputs:
      - name: docker-source
        type: git
    outputs:
      - name: builtImage
        type: image
  steps:
    - name: build-and-push # 使用google 开源的镜像工具kaniko build docker image
      image: registry.cn-beijing.aliyuncs.com/tekton_cd/kaniko-project_executor:v0.16.0
      env:
        - name: "DOCKER_CONFIG"
          value: "/tekton/home/.docker/"
      command:
        - /kaniko/executor
      args:
        - --dockerfile=$(params.pathToDockerFile) # 指定dockerfile路径
        - --destination=$(resources.outputs.builtImage.url) # 指定镜像仓库
        - --context=$(params.pathToContext) # 指定项目workspace
        - --skip-tls-verify # 配置跳过ssl验证 适用于内网https环境

创建TaskRun

# taskrun.yaml
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: build-docker-image-from-git-source-task-run
  namespace: tekton-test
spec:
  serviceAccountName: tekton-test
  taskRef: # 关联task 
    name: build-docker-image-from-git-source
  params:
    - name: pathToDockerFile
      value: Dockerfile
    - name: pathToContext
      value: $(resources.inputs.docker-source.path)
  resources:
    inputs: # 关联输入资源
      - name: docker-source
        resourceRef:
          name: git
    outputs: # 关联输出资源
      - name: builtImage
        resourceRef:
          name: image

三、通过Trigger监听事件变化自动触发任务

配置RBAC

由于 EventListener 对象需要访问其他资源，所以需要为之前创建的serviceaccount配置rbac

# rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: tekton-triggers-gitlab-minimal
  namespace: tekton-test
rules:
# EventListeners need to be able to fetch all namespaced resources
- apiGroups: ["triggers.tekton.dev"]
  resources: ["eventlisteners", "triggerbindings", "triggertemplates", "triggers"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
# configmaps is needed for updating logging config
  resources: ["configmaps"]
  verbs: ["get", "list", "watch"]
# Permissions to create resources in associated TriggerTemplates
- apiGroups: ["tekton.dev"]
  resources: ["pipelineruns", "pipelineresources", "taskruns"]
  verbs: ["create"]
- apiGroups: [""]
  resources: ["serviceaccounts"]
  verbs: ["impersonate"]
- apiGroups: ["policy"]
  resources: ["podsecuritypolicies"]
  resourceNames: ["tekton-triggers"]
  verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: tekton-triggers-gitlab-binding
  namespace: tekton-test
subjects:
- kind: ServiceAccount
  name: tekton-test
  namespace: tekton-test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: tekton-triggers-gitlab-minimal
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: tekton-triggers-gitlab-clusterrole
rules:
- apiGroups: ["triggers.tekton.dev"]
  resources: ["clustertriggerbindings", "clusterinterceptors"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tekton-triggers-gitlab-clusterbinding
subjects:
- kind: ServiceAccount
  name: tekton-test
  namespace: tekton-test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: tekton-triggers-gitlab-clusterrole

创建TriggerBinding和TriggerTemplate

# triggerbinding.yaml
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
  name: gitlab-push-binding
  namespace: tekton-test
spec:
  params:
    - name: gitrevision
      value: $(body.checkout_sha)
    - name: gitrepositoryurl
      value: $(body.repository.git_http_url)
    - name: gitrepositoryname
      value: $(body.repository.name)

# triggertemplate.yaml
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
  name: gitlab-push-binding
  namespace: tekton-test
spec:
  params:
    - name: gitrevision
      value: $(body.checkout_sha)
    - name: gitrepositoryurl
      value: $(body.repository.git_http_url)
    - name: gitrepositoryname
      value: $(body.repository.name)

创建eventlistener

部署eventlistener之前要创建secret用于验证gitlab webhook,并更新serviceaccount

# secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: gitlab-secret
  namespace: tekton-test
type: Opaque
stringData:
  secretToken: "1234567"
# serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: tekton-test
  namespace: tekton-test
secrets:
  - name: gitlab-secret
  - name: gitlab-auth
  - name: harbor-user-pass

# eventlist.yaml
apiVersion: triggers.tekton.dev/v1alpha1
kind: EventListener
metadata:
  name: gitlab-listener
  namespace: tekton-test
spec:
  resources:
    kubernetesResource:
      serviceType: NodePort #定义创建的service类型 也可通过Ingress暴露，此处为了实验方便，直接使用nodeport
  serviceAccountName: tekton-triggers-gitlab-sa
  triggers:
    - name: gitlab-push-events-trigger
      interceptors:
        - name: "verify-gitlab-payload"
          ref:
            name: "gitlab"
            kind: ClusterInterceptor
          params:
            - name: secretRef
              value:
                secretName: "gitlab-secret"
                secretKey: "secretToken"
            - name: eventTypes
              value:
                - "Push Hook"
      bindings:
        - ref: gitlab-push-binding
      template:
        ref: gitlab-build-template

查看生成的service

[root@cilium-1 trigger]# kubectl get svc -n tekton-test
NAME                 TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
el-gitlab-listener   NodePort   10.110.197.67   <none>        8080:31954/TCP   3m

效果

更新代码并推送