【云原生 | Kubernetes 系列】--Gitops持续交付和持续Tekton Triggers
Tekton Triggers
Tekton 几个项目
-
Tekton Pipelines
-
Task
- Steps:执行具体操作,在容器中执行的命令或脚本
- command
- script
- Steps:执行具体操作,在容器中执行的命令或脚本
-
TaskRun
运行为一个pod,该pod中每个step都运行为一个容器,根据定义的次序,依次运行
那些由用户通过command或script指定要运行的脚本或程序,实际上是通过entrypoint启动的.
-
Pipeline
组合一个到多个Task
-
PipelineRun
根据指定的Pipeline创建一个特定的运行实例
将Pipeline中定义一到多个Task,根据用户配置的次序,运行TaskRun
-
-
Tekton Triggers
-
Tekton Dashboard
-
Tekton CLI
模块化Task的一个重要要素: Parameters,在task中定义Parameter,在step中引用.
在command或script中引用
数据传递:
Step to Step, Task to Task
Results: 其后端的存储是由pipeline自行创建和管理,仅能够传递小于4096字节的数据
Workspace: 工作空间,文件系统
其对应的后端存储,需要由用户在运行TaskRun或PipelineRun时予以定义
允许在Step或Task间共享使用
通常用语存储卷就是kubernetes的存储卷
emptyDir:生命周期同Pod
pvc/pv:k8s上独立的资源,生命周期独立,可以跨Task使用
pv的预配
Static pv
Dynamic pv: 要求后端存储服务允许PVC能够按对应PVC中声明的存储特性按需创建存储空间
Ceph RBD
通过csi接口对接到k8s
nfs-csi driver
PVC:Claim
隶属于namespace
ConfigMap:提供只读的配置信息
Secrets:提供加密信息
1. Pipeline和Task上的数据共享
1.1 Workspace
由Task进行声明,在TaskRun时以volume进行赋值,并作为文件系统使用
通常是comfigmap,secret,emptyDir,静态PVC或动态pvc
emptyDir的生命周期通Pod,只能在同一Task的各Step之间共享数据
跨Task数据共享,需要pvc
1.2 Results
由Task进行声明
将Task中某Step生成的结果保存在临时文件中(/token/results/),而后由同一Task中后面的Step引用,或者由后面其他Task中的Step引用.例如:“$(results..path)”
由Tekton的Results API负责实现,仅可用于共享小于4096字节规模的小数据片
2. 在Task上声明Workspace
定义在spec.workspace字段中
定义嵌套以下字段:
- name: 必选,workspace唯一标识符
- description: 描述信息,通常标明其使用目的
- readOnly: 是否为只读,默认false,configmap和secret为只读
- optional: 是否可选,默认false
- mountPath: 在各step中挂载路径,默认为/workspace/,name为workspace的名称
可用的workspace变量:
-
$(workspace..path)由name指定workspace挂载的路径,对于可选且TaskRun未声明时,其值为空
有2种引用方式:
- /workspace/
- $(workspace..path)
- /workspace/
-
$(workspace..bound):值为true或false,用于标识指定的workspace是否已经绑定,对于optional为false的workspace改变量始终为true
-
$(workspace..claim),由name标识的workspace使用的pvc名称,对于非pvc类型的存储卷改变量值为空
-
$(workspace..volume):由name标识的workspace所使用的存储卷的名称
3. EmptyDir单Step测试
# cat 01-task-workspace-demo.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: workspace-demo
spec:
params:
- name: target
type: string
default: pana
steps:
- name: write-message
image: alpine:3.16
script: |
#!/bin/sh
set -xe
if [ "$(workspaces.messages.bound)" == "true" ] ; then
echo "Hello $(params.target)" > $(workspaces.messages.path)/message
cat $(workspaces.messages.path)/message
fi
echo "Mount Path: $(workspaces.messages.path)"
echo "Volume Name: $(workspaces.messages.volume)"
workspaces:
- name: messages
description: |
The folder where we write the message to. If no workspace
is provided then the message will not be written.
optional: true
mountPath: /data
创建task
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# kubectl apply -f 01-task-workspace-demo.yaml
task.tekton.dev/workspace-demo created
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# tkn task ls
NAME DESCRIPTION AGE
hello 3 days ago
hello-params 1 day ago
logger 22 hours ago
multiple 23 hours ago
script 23 hours ago
workspace-demo 16 seconds ago
实例化task
root@k8s-master-01:~# tkn task start workspace-demo --showlog \
> -p target="www.baidu.com" \ ## 传参数给变量target,如果不传就是default的pana生效
> -w name=messages,emptyDir="" ## 指定workspace是messages,emptyDir只能是空,不能传参数所以用""
TaskRun started: workspace-demo-run-qwsnh
Waiting for logs to be available...
[write-message] + '[' true '==' true ]
[write-message] + echo 'Hello www.baidu.com'
[write-message] + cat /data/message
[write-message] Hello www.baidu.com
[write-message] + echo 'Mount Path: /data'
[write-message] Mount Path: /data
[write-message] + echo 'Volume Name: ws-4mljp' ## emptyDir的名字是ws-4mljp
[write-message] Volume Name: ws-4mljp
root@k8s-master-01:~# kubectl describe pods workspace-demo-run-qwsnh-pod|grep 4mljp
/data from ws-4mljp (rw)
ws-4mljp:
root@k8s-master-01:~#
4. Empty 多Step间文件共享
4.1 公开的git仓库
这里需要留意一点,仓库必须是公开,否则就要对仓库做免密或者http://用户名:密码@仓库
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# cat 02-task-with-workspace.yaml
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: source-lister
spec:
params:
- name: git-repo
type: string
description: Git repository to be cloned
workspaces:
- name: source
steps:
- name: git-clone
image: alpine/git:v2.36.1
script: git clone -v $(params.git-repo) $(workspaces.source.path)/source
- name: list-files
image: alpine:3.16
command:
- /bin/sh
args:
- '-c'
- 'cat $(workspaces.source.path)/source/issue'
创建task
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# kubectl apply -f 02-task-with-workspace.yaml
task.tekton.dev/source-lister created
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# tkn task ls |grep source-lister
source-lister 49 seconds ago
实例化task
root@k8s-master-01:~# tkn task start source-lister --showlog -p git-repo=http://192.168.31.199/tekton/app01.git -w name=source,emptyDir=""
TaskRun started: source-lister-run-b8n4d
Waiting for logs to be available...
[git-clone] Cloning into '/workspace/source/source'...
[git-clone] POST git-upload-pack (175 bytes)
[git-clone] POST git-upload-pack (217 bytes)
[list-files] Ubuntu 18.04.6 LTS \n \l
4.2 私有的git仓库
注意当密码中有特殊字符时需要转码
! # $ & ' ( ) * + , / : ; = ? @ [ ]
%21 %23 %24 %26 %27 %28 %29 %2A %2B %2C %2F %3A %3B %3D %3F %40 %5B %5D
效果如下
root@k8s-master-01:~# tkn task start source-lister --showlog \
> -p git-repo=http://root:Pana%[email protected]/wework/app1.git \
> -w name=source,emptyDir=""
TaskRun started: source-lister-run-jgrx6
Waiting for logs to be available...
[git-clone] Cloning into '/workspace/source/source'...
[git-clone] POST git-upload-pack (175 bytes)
[git-clone] POST git-upload-pack (217 bytes)
[list-files] Ubuntu 18.04.6 LTS \n \l
由于emptyDir每次都会被重建,这样如果需要将数据持久化的话就无法实现,可以看到每次执行volume都会发生改变
root@k8s-master-01:~# tkn task start source-lister --showlog -p git-repo=http://root:Pana%[email protected]/wework/app1.git -w name=source,emptyDir=""
TaskRun started: source-lister-run-h95rn
Waiting for logs to be available...
[git-clone] Cloning into '/workspace/source/source'...
[git-clone] POST git-upload-pack (175 bytes)
[git-clone] POST git-upload-pack (217 bytes)
[list-files] Volume Name: ws-pnk58
root@k8s-master-01:~# tkn task start source-lister --showlog -p git-repo=http://root:Pana%[email protected]/wework/app1.git -w name=source,emptyDir=""
TaskRun started: source-lister-run-kt2gc
Waiting for logs to be available...
[git-clone] Cloning into '/workspace/source/source'...
[git-clone] POST git-upload-pack (175 bytes)
[git-clone] POST git-upload-pack (217 bytes)
[list-files] Volume Name: ws-qr666
5. Pipeline示例
# cat 03-pipeline-workspace.yaml
apiVersion: tekton.dev/v1beta1
kind: Pipeline # 类型是pipeline
metadata:
name: pipeline-source-lister # pipeline的名字
spec:
workspaces:
- name: codebase # pipeline的workspace
params:
- name: git-url # pipeline的参数
type: string
description: Git repository url to be cloned
tasks:
- name: git-clone # pipeline对task的命名
taskRef: # 引用task
name: source-lister # 引用source-lister task
workspaces:
- name: source # source-lister 的workspace是source
workspace: codebase # 将pipeline的codebase传递给source-lister的source
params:
- name: git-repo
value: $(params.git-url) # 将pipeline参数git-url的值 传递给task的git-repo参数
部署pipeline
# kubectl apply -f 03-pipeline-workspace.yaml
pipeline.tekton.dev/pipeline-source-lister created
root@k8s-master-01:~# tkn pipeline ls
NAME AGE LAST RUN STARTED DURATION STATUS
pipeline-demo 1 day ago pipeline-demo-run-20221025 23 hours ago 17s Succeeded
pipeline-source-lister 11 seconds ago --- --- --- ---
pipeline-task-ordering 22 hours ago pipeline-task-ordering-run-fzb5j 22 hours ago 3m5s Succeeded
pipeline-with-params 23 hours ago pipeline-with-params-run-1025-2 22 hours ago 7s Succeeded
实例化pipeline
root@k8s-master-01:~# tkn pipeline start pipeline-source-lister --showlog \
> -p git-url=http://192.168.31.199/tekton/app01.git \
> -w name=codebase,emptyDir=""
PipelineRun started: pipeline-source-lister-run-blc9x
Waiting for logs to be available...
[git-clone : git-clone] Cloning into '/workspace/source/source'...
[git-clone : git-clone] POST git-upload-pack (175 bytes)
[git-clone : git-clone] POST git-upload-pack (217 bytes)
[git-clone : list-files] Ubuntu 18.04.6 LTS \n \l
pipelinerun
root@k8s-master-01:~# tkn pipeline start pipeline-source-lister --showlog -p git-url=http://192.168.31.199/tekton/app01.git -w name=codebase,emptyDir="" --dry-run
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
creationTimestamp: null
generateName: pipeline-source-lister-run-
namespace: default
spec:
params:
- name: git-url
value: http://192.168.31.199/tekton/app01.git
pipelineRef:
name: pipeline-source-lister
workspaces:
- emptyDir: {}
name: codebase
status: {}
应该使用volumeClaimTemplate来动态绑定pvc
6. 基于nfs-csi的PVC共享
# 项目地址https://github.com/kubernetes-csi/csi-driver-nfs
6.1 开启nfs服务
nfs服务器
# cat /etc/exports
/data/k8s *(rw,sync,no_root_squash)
root@haproxy-1:~# systemctl enable --now nfs-server.service
所有node节点上执行,或者给node打tag并使用nodeselect
# mkdir /nfs-vol &&mount -t nfs 192.168.31.109:/data/k8s/nfs /nfs-vol
6.2 在k8s cluster启用nfs server
root@k8s-master-01:~# kubectl create ns nfs
namespace/nfs created
root@k8s-master-01:~# kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/nfs-provisioner/nfs-server.yaml -n nfs
service/nfs-server created
deployment.apps/nfs-server created
root@k8s-master-01:~# kubectl get deployment -n nfs
NAME READY UP-TO-DATE AVAILABLE AGE
nfs-server 1/1 1 1 55s
root@k8s-master-01:~# kubectl get svc -n nfs
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nfs-server ClusterIP 10.200.73.254 <none> 2049/TCP,111/UDP 58s
root@k8s-master-01:~#
6.3 安装NFS CSI v3.1.0
克隆离线安装文件
# git clone https://github.com/kubernetes-csi/csi-driver-nfs.git
Cloning into 'csi-driver-nfs'...
remote: Enumerating objects: 21474, done.
remote: Counting objects: 100% (880/880), done.
remote: Compressing objects: 100% (484/484), done.
remote: Total 21474 (delta 423), reused 734 (delta 364), pack-reused 20594
Receiving objects: 100% (21474/21474), 22.10 MiB | 114.00 KiB/s, done.
Resolving deltas: 100% (11719/11719), done.
下载镜像(先从github到阿里转一道)
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:csi-provisioner
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:livenessprobe
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:nfsplugin
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:csi-node-driver-registrar
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:csi-provisioner harbor.intra.com/csi-driver-nfs/csi-provisioner:v2.2.2
docker push harbor.intra.com/csi-driver-nfs/csi-provisioner:v2.2.2
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:livenessprobe harbor.intra.com/csi-driver-nfs/livenessprobe:v2.5.0
docker push harbor.intra.com/csi-driver-nfs/livenessprobe:v2.5.0
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:nfsplugin harbor.intra.com/csi-driver-nfs/nfsplugin:v3.1.0
docker push harbor.intra.com/csi-driver-nfs/nfsplugin:v3.1.0
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:csi-node-driver-registrar harbor.intra.com/csi-driver-nfs/csi-node-driver-registrar:v2.4.0
docker push harbor.intra.com/csi-driver-nfs/csi-node-driver-registrar:v2.4.0
替换镜像
sed -i 's#registry.k8s.io/sig-storage#harbor.intra.com/csi-driver-nfs#g' *.yaml
替换后的结果
root@k8s-master-01:/data/csi-driver-nfs/deploy/v3.1.0# grep image *.yaml
csi-nfs-controller.yaml: image: registry.k8s.io/sig-storage/csi-provisioner:v2.2.2
csi-nfs-controller.yaml: image: registry.k8s.io/sig-storage/livenessprobe:v2.5.0
csi-nfs-controller.yaml: image: registry.k8s.io/sig-storage/nfsplugin:v3.1.0
csi-nfs-controller.yaml: imagePullPolicy: IfNotPresent
csi-nfs-node.yaml: image: registry.k8s.io/sig-storage/livenessprobe:v2.5.0
csi-nfs-node.yaml: image: registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.4.0
csi-nfs-node.yaml: image: registry.k8s.io/sig-storage/nfsplugin:v3.1.0
csi-nfs-node.yaml: imagePullPolicy: "IfNotPresent"
root@k8s-master-01:/data/csi-driver-nfs/deploy/v3.1.0# sed -i 's#registry.k8s.io/sig-storage#harbor.intra.com/csi-driver-nfs#g' *.yaml
root@k8s-master-01:/data/csi-driver-nfs/deploy/v3.1.0# grep image *.yaml
csi-nfs-controller.yaml: image: harbor.intra.com/csi-driver-nfs/csi-provisioner:v2.2.2
csi-nfs-controller.yaml: image: harbor.intra.com/csi-driver-nfs/livenessprobe:v2.5.0
csi-nfs-controller.yaml: image: harbor.intra.com/csi-driver-nfs/nfsplugin:v3.1.0
csi-nfs-controller.yaml: imagePullPolicy: IfNotPresent
csi-nfs-node.yaml: image: harbor.intra.com/csi-driver-nfs/livenessprobe:v2.5.0
csi-nfs-node.yaml: image: harbor.intra.com/csi-driver-nfs/csi-node-driver-registrar:v2.4.0
csi-nfs-node.yaml: image: harbor.intra.com/csi-driver-nfs/nfsplugin:v3.1.0
csi-nfs-node.yaml: imagePullPolicy: "IfNotPresent"
执行本地化部署,这里有个小坑,安装3.1.0时会提示./deploy/v3.1.0/rbac-csi-nfs.yaml文件没有,只需要将./deploy/v3.1.0/rbac-csi-nfs-controller.yaml复制过去就可以了.
root@k8s-master-01:/data/csi-driver-nfs# ./deploy/install-driver.sh v3.1.0 local
use local deploy
Installing NFS CSI driver, version: v3.1.0 ...
error: the path "./deploy/v3.1.0/rbac-csi-nfs.yaml" does not exist
root@k8s-master-01:/data/csi-driver-nfs# cp ./deploy/v3.1.0/rbac-csi-nfs-controller.yaml ./deploy/v3.1.0/rbac-csi-nfs.yaml
root@k8s-master-01:/data/csi-driver-nfs# ./deploy/install-driver.sh v3.1.0 local
use local deploy
Installing NFS CSI driver, version: v3.1.0 ...
serviceaccount/csi-nfs-controller-sa unchanged
clusterrole.rbac.authorization.k8s.io/nfs-external-provisioner-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/nfs-csi-provisioner-binding unchanged
csidriver.storage.k8s.io/nfs.csi.k8s.io created
deployment.apps/csi-nfs-controller created
daemonset.apps/csi-nfs-node configured
NFS CSI driver installed successfully.
此时在kube-system下的pod已经正常启动了
root@k8s-master-01:/data/csi-driver-nfs# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-76586bcfb6-tf6kg 1/1 Running 7 (4d9h ago) 43d
calico-node-48nd2 1/1 Running 4 (52d ago) 134d
calico-node-6csj7 1/1 Running 8 (4d22h ago) 134d
calico-node-brphr 1/1 Running 34 (107s ago) 182d
calico-node-jd8xr 1/1 Running 35 (4d22h ago) 134d
calico-node-pl8sz 1/1 Running 33 (95s ago) 85d
calico-node-szqd6 1/1 Running 11 (4d22h ago) 62d
calico-node-xzfgw 1/1 Running 35 (3d23h ago) 182d
coredns-55556fc4d7-92ptb 1/1 Running 18 (95s ago) 43d
csi-nfs-controller-7d758f5b6b-6847h 3/3 Running 0 5m35s
csi-nfs-controller-7d758f5b6b-k7m9z 3/3 Running 0 5m35s
kube-state-metrics-8444bbc459-mw4zk 1/1 Running 13 (4d9h ago) 43d
metrics-server-5b5f4d4dd4-m7xc9 1/1 Running 19 (4d9h ago) 40d
部署csi-nfs-node
先搞镜像
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:livenessprobe
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:nfsplugin
docker pull registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:csi-node-driver-registrar
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:livenessprobe harbor.intra.com/csi-driver-nfs/livenessprobe:v2.7.0
docker push harbor.intra.com/csi-driver-nfs/livenessprobe:v2.7.0
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:nfsplugin harbor.intra.com/csi-driver-nfs/nfsplugin:canary
docker push harbor.intra.com/csi-driver-nfs/nfsplugin:canary
docker tag registry.cn-shanghai.aliyuncs.com/qiuqin/csi-driver-nfs:csi-node-driver-registrar harbor.intra.com/csi-driver-nfs/csi-node-driver-registrar:v2.5.1
docker push harbor.intra.com/csi-driver-nfs/csi-node-driver-registrar:v2.5.1
替换镜像
sed -i 's#registry.k8s.io/sig-storage#harbor.intra.com/csi-driver-nfs#g' deploy/csi-nfs-node.yaml
sed -i 's#gcr.io/k8s-staging-sig-storage#harbor.intra.com/csi-driver-nfs#g' csi-nfs-node.yaml
这里又有一个坑了…
用csi-node-driver-registrar:v2.5.1会发现Pod状态是CrashLoopBackOff,故障原因是node-driver-registrar这个container引起,报错内容是flag provided but not defined: -csi-address
所以把node-driver-registra的镜像改成2.4.0就好了
# grep image deploy/csi-nfs-node.yaml
image: harbor.intra.com/csi-driver-nfs/livenessprobe:v2.7.0
image: harbor.intra.com/csi-driver-nfs/csi-node-driver-registrar:v2.4.0
image: harbor.intra.com/csi-driver-nfs/nfsplugin:canary
imagePullPolicy: "IfNotPresent"
部署生效
# kubectl apply -f deploy/rbac-csi-nfs.yaml
# kubectl apply -f deploy/csi-nfs-node.yaml
此时kube-system下csi-nfs-node也都创建好了.
# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-76586bcfb6-tf6kg 1/1 Running 8 (54m ago) 44d
calico-node-48nd2 1/1 Running 5 (29m ago) 135d
calico-node-6csj7 1/1 Running 9 (54m ago) 135d
calico-node-brphr 1/1 Running 34 (56m ago) 182d
calico-node-jd8xr 1/1 Running 36 (32m ago) 135d
calico-node-pl8sz 1/1 Running 33 (56m ago) 85d
calico-node-szqd6 1/1 Running 12 (32m ago) 62d
calico-node-xzfgw 1/1 Running 36 (54m ago) 182d
coredns-55556fc4d7-92ptb 1/1 Running 18 (56m ago) 44d
csi-nfs-controller-7d758f5b6b-6847h 3/3 Running 3 (29m ago) 60m
csi-nfs-controller-7d758f5b6b-k7m9z 3/3 Running 0 60m
csi-nfs-node-84g5n 3/3 Running 0 7m39s
csi-nfs-node-92gr2 3/3 Running 0 7m39s
csi-nfs-node-k9kwq 3/3 Running 0 7m39s
csi-nfs-node-klmb4 3/3 Running 0 7m39s
csi-nfs-node-mjm6r 3/3 Running 0 7m39s
csi-nfs-node-pxs8r 3/3 Running 0 7m39s
csi-nfs-node-r4j8f 3/3 Running 0 7m39s
kube-state-metrics-8444bbc459-mw4zk 1/1 Running 15 (53m ago) 44d
metrics-server-5b5f4d4dd4-m7xc9 1/1 Running 33 (53m ago) 40d
没外网有点太曲折了
有外网可以直接
curl -skSL https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/v3.1.0/deploy/install-driver.sh | bash -s v3.1.0 --
6.4 关联nfs-csi-driver与nfs-server
创建StorageClass
# cat nfs-csi-driver-sc.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-csi
provisioner: nfs.csi.k8s.io
parameters:
server: nfs-server.nfs.svc.cluster.local
share: /
# csi.storage.k8s.io/provisioner-secret is only needed for providing mountOptions in DeleteVolume
# csi.storage.k8s.io/provisioner-secret-name: "mount-options"
# csi.storage.k8s.io/provisioner-secret-namespace: "default"
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
- nconnect=8
- nfsvers=4.1
生效
root@k8s-master-01:/data/nfs# kubectl apply -f nfs-csi-driver-sc.yaml
storageclass.storage.k8s.io/nfs-csi created
root@k8s-master-01:/data/nfs# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-csi nfs.csi.k8s.io Delete Immediate false 4s
## 如果不希望删除pvc后数据立即回收可以将reclaimPolicy改为Retain
reclaimPolicy: Retain
root@k8s-master-01:/data/nfs# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-csi nfs.csi.k8s.io Retain Immediate false 2s
测试动态制备pvc
# kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/pvc-nfs-csi-dynamic.yaml
root@k8s-master-01:/data/nfs# cat pvc-nfs-csi-dynamic.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-nfs-dynamic
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 10Gi
storageClassName: nfs-csi
root@k8s-master-01:/data/nfs# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-nfs-dynamic Bound pvc-f0d481de-14a3-4428-92db-02736d895f4c 10Gi RWX nfs-csi 5s
root@k8s-master-01:/data/nfs# kubectl get pv|grep pvc-f0d481de-14a3-4428-92db-02736d895f4c
pvc-f0d481de-14a3-4428-92db-02736d895f4c 10Gi RWX Delete Bound default/pvc-nfs-dynamic nfs-csi 55s
## 清理
root@k8s-master-01:/data/nfs# kubectl delete pvc pvc-nfs-dynamic
persistentvolumeclaim "pvc-nfs-dynamic" deleted
root@k8s-master-01:/data/nfs# kubectl get pv|grep pvc-f0d481de-14a3-4428-92db-02736d895f4c
至此已经可以基于VolumeClaimTemplate动态请求pvc
6.5 PipelineRun使用pvc
配置文件
# cat pipelinerun-with-pvc-demo.yaml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: pipeline-source-lister-run-001
spec:
pipelineRef:
name: pipeline-source-lister
params:
- name: git-url
value: http://192.168.31.199/tekton/app01.git
workspaces:
- name: codebase
volumeClaimTemplate:
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 1Gi
storageClassName: nfs-csi
部署运行
# kubectl apply -f pipelinerun-with-pvc-demo.yaml
pipelinerun.tekton.dev/pipeline-source-lister-run-001 created
# tkn pipelinerun ls|grep pipeline-source-lister-run-001
pipeline-source-lister-run-001 1 minute ago 15s Succeeded
# tkn pipelinerun logs pipeline-source-lister-run-001
[git-clone : git-clone] Cloning into '/workspace/source/source'...
[git-clone : git-clone] POST git-upload-pack (175 bytes)
[git-clone : git-clone] POST git-upload-pack (217 bytes)
[git-clone : list-files] Ubuntu 18.04.6 LTS \n \l
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-05b2130fe7 Bound pvc-8b7a15ba-f38c-42b0-9b5d-986c4bb9fa6f 1Gi RWO nfs-csi 16m
此时pipelinerun创建的pod已经结束了生命周期,但pvc不会因此被删除
7. 综合示例
通过pvc实现在2个Task间数据共享
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: volume-share
spec:
params:
- name: git-url
type: string
workspaces:
- name: codebase
tasks:
- name: fetch-from-source
params:
- name: url
value: $(params.git-url)
taskSpec:
workspaces:
- name: source
params:
- name: url
steps:
- name: git-clone
image: alpine/git:v2.36.1
script: git clone -v $(params.url) $(workspaces.source.path)/source
workspaces:
- name: source
workspace: codebase
- name: source-lister
runAfter:
- fetch-from-source
taskSpec:
steps:
- name: list-files
image: alpine:3.16
script: ls $(workspaces.source.path)/source
workspaces:
- name: source
workspaces:
- name: source
workspace: codebase
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: volume-share-run-xxxx
spec:
pipelineRef:
name: volume-share
params:
- name: git-url
value: http://192.168.31.199/tekton/app01.git
workspaces:
- name: codebase
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: nfs-csi
执行
# kubectl apply -f 04-pipeline-worlspace-02.yaml
pipeline.tekton.dev/volume-share created
pipelinerun.tekton.dev/volume-share-run-xxxx created
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# tkn pipeline ls
NAME AGE LAST RUN STARTED DURATION STATUS
pipeline-demo 1 day ago pipeline-demo-run-20221025 1 day ago 17s Succeeded
pipeline-source-lister 22 hours ago pipeline-source-lister-run-001 1 hour ago 15s Succeeded
pipeline-task-ordering 1 day ago pipeline-task-ordering-run-fzb5j 1 day ago 3m5s Succeeded
pipeline-with-params 1 day ago pipeline-with-params-run-1025-2 1 day ago 7s Succeeded
volume-share 1 minute ago volume-share-run-xxxx 59 seconds ago 23s Succeeded
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# tkn pipelinerun ls
NAME STARTED DURATION STATUS
volume-share-run-xxxx 28 seconds ago 23s Succeeded
pipeline-source-lister-run-001 1 hour ago 15s Succeeded
pipeline-source-lister-run-blc9x 22 hours ago 6s Succeeded
pipeline-task-ordering-run-fzb5j 1 day ago 3m5s Succeeded
pipeline-with-params-run-1025-2 1 day ago 7s Succeeded
pipeline-with-params-run-1025 1 day ago 7s Succeeded
pipeline-with-params-run-22w2m 1 day ago 8s Succeeded
pipeline-with-params-run-rhp75 1 day ago 14s Succeeded
pipeline-with-params-run-xxxxx 1 day ago 8s Succeeded
pipeline-with-params-run-dwt8h 1 day ago 8s Succeeded
pipeline-with-params-run-jrvrh 1 day ago 1m22s Succeeded
pipeline-demo-run-20221025 1 day ago 17s Succeeded
pipeline-demo-run-p2g6t 1 day ago 20s Succeeded
pipeline-demo-run-7chp9 1 day ago 18s Succeeded
pipeline-demo-run-rlgzp 1 day ago 9s Succeeded
pipeline-demo-run-2tjlz 1 day ago 9s Succeeded
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-05b2130fe7 Bound pvc-8b7a15ba-f38c-42b0-9b5d-986c4bb9fa6f 1Gi RWO nfs-csi 73m
pvc-b496e3a247 Bound pvc-ea19b117-6406-400a-8004-45293fd4a23e 1Gi RWO nfs-csi 33s
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# tkn pipeline logs volume-share
Cloning into '/workspace/source/source'...
POST git-upload-pack (175 bytes)
POST git-upload-pack (217 bytes)
README.md
issue
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# tkn pipelinerun logs volume-share-run-xxxx
[fetch-from-source : git-clone] Cloning into '/workspace/source/source'...
[fetch-from-source : git-clone] POST git-upload-pack (175 bytes)
[fetch-from-source : git-clone] POST git-upload-pack (217 bytes)
[source-lister : list-files] README.md
[source-lister : list-files] issue
如果用emptyDir就无法实现不同task之间的传递,第二个task就会失败
root@k8s-master-01:/apps/tekton-and-argocd-in-practise/03-tekton-advanced# tkn pipeline start volume-share -p git-url=http://192.168.31.199/tekton/app01.git -w name=codebase,emptyDir="" --showlog
PipelineRun started: volume-share-run-ptgcm
Waiting for logs to be available...
[fetch-from-source : git-clone] Cloning into '/workspace/source/source'...
[fetch-from-source : git-clone] POST git-upload-pack (175 bytes)
[fetch-from-source : git-clone] POST git-upload-pack (217 bytes)
[source-lister : list-files] ls: /workspace/source/source: No such file or directory
failed to get logs for task source-lister : container step-list-files has failed : [{"key":"StartedAt","value":"2022-10-27T04:35:32.423Z","type":3}]