香橙派4和树莓派4B构建K8S集群实践之七： Jenkins
目录
1. 说明
2. 步骤
2.1 准备工作
2.2 安装
2.2.1 用jenkins原站for k8s的安装仓方法安装
2.2.2 Helm 安装
3. 相关命令
4. 遇到的问题
5. 参考
1. 说明
- 在k8s上部署jenkins，并用 jenkins.k8s-t2.com访问
- 在namespace为devops下安装在指定节点k8s-master-1，有指定持久化的PV/PVC/SC
- CI/DI 实践
2. 步骤
2.1 准备工作
设置代理，不然去git拿文件的时候会遇到麻烦
git config --global http.proxy 'socks5://192.168.0.108:1080'
git config --global https.proxy 'socks5://192.168.0.108:1080'
git config --global --unset http.proxy
git config --global --unset https.proxy编辑客户机hosts, 映射子域名
192.168.0.103 jenkins.k8s-t2.com
2.2 安装
2.2.1 用jenkins原站for k8s的安装仓方法安装
获取
git clone https://github.com/scriptcamp/kubernetes-jenkins建个namespace
kubectl create ns devops-tools编辑 volume.yaml，设置/data0/jenkins-volume为存储地, 节点 k8s-master-0
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: jenkins-pv-volume
labels:
type: local
spec:
storageClassName: local-storage
claimRef:
name: jenkins-pv-claim
namespace: devops-tools
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
local:
path: /data0/jenkins-volume
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- k8s-master-0
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-pv-claim
namespace: devops-tools
spec:
storageClassName: local-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi建serviceAccount.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: jenkins-admin
rules:
- apiGroups: [""]
resources: ["*"]
verbs: ["*"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins-admin
namespace: devops-tools
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: jenkins-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins-admin
subjects:
- kind: ServiceAccount
name: jenkins-admin
namespace: devops-tools建deployment.yaml ，为使得jenkins插件能科学安装，需进行deployment中的环境代理设置，否则牙痛 : )
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
namespace: devops-tools
spec:
replicas: 1
selector:
matchLabels:
app: jenkins-server
template:
metadata:
labels:
app: jenkins-server
spec:
securityContext:
fsGroup: 1000
runAsUser: 1000
serviceAccountName: jenkins-admin
containers:
- name: jenkins
image: jenkins/jenkins:lts
resources:
limits:
memory: "2Gi"
cpu: "1000m"
requests:
memory: "500Mi"
cpu: "500m"
ports:
- name: httpport
containerPort: 8080
- name: jnlpport
containerPort: 50000
livenessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 90
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
volumeMounts:
- name: jenkins-data
mountPath: /var/jenkins_home
env:
- name: http_proxy
value: http://192.168.0.108:1081
- name: https_proxy
value: http://192.168.0.108:1081
- name: no_proxy
value: aliyuncs.com,huaweicloud.com,k8s-master-0,k8s-master-1,k8s-worker-0,localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
volumes:
- name: jenkins-data
persistentVolumeClaim:
claimName: jenkins-pv-claim建service.yaml
apiVersion: v1
kind: Service
metadata:
name: jenkins-service
namespace: devops-tools
annotations:
prometheus.io/scrape: 'true'
prometheus.io/path: /
prometheus.io/port: '8080'
spec:
selector:
app: jenkins-server
type: NodePort
ports:
- port: 8080
targetPort: 8080
nodePort: 32000执行脚本k8s-jenkins.sh
#!/bin/bash
kubectl label node k8s-master-0 app=jenkins-server
kubectl create namespace devops-tools
kubectl apply -f /k8s_apps/kubernetes-jenkins/serviceAccount.yaml
kubectl create -f /k8s_apps/kubernetes-jenkins/volume.yaml
kubectl apply -f /k8s_apps/kubernetes-jenkins/deployment.yaml
kubectl apply -f /k8s_apps/kubernetes-jenkins/service.yaml
成功后可查pod日志获取admin密码
2.2.2 Helm 安装
添加jenkins来源:
helm repo add jenkinsci https://charts.jenkins.io
helm repo updatehelm search repo jenkinsci
可知当前版本为:
NAME CHART VERSION APP VERSION DESCRIPTION
jenkinsci/jenkins 4.3.26 2.401.1 Jenkins - Build great things at any scale! The ...
获取到本地，并解压
helm fetch jenkinsci/jenkins
tar -zxvf jenkins.tgz
编辑 values.yaml：
ingress:
#enabled: false
=>
ingress:
enabled: true
# See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
# 需注明用的是哪个ingress class，因为之前装的是ingress-nginx, 所以这里填nginx
ingressClassName: nginx
# Set this path to jenkinsUriPrefix above or use annotations to rewrite path
hostName: jenkins.k8s-t2.com注意ingress需指定对应的 ingressClassName
执行安装
helm upgrade --install jenkins --namespace default \
-f values.yaml \
jenkins/jenkins
# 过程大概要30分钟
NOTES:
1. 获取admin登录密码 Get your 'admin' user password by running:
kubectl exec --namespace default -it svc/jenkins -c jenkins -- /bin/cat /run/secrets/additional/chart-admin-password && echo
2. Visit http://jenkins.k8s-t2.com
3. Login with the password from step 1 and the username: admin
4. Configure security realm and authorization strategy
5. Use Jenkins Configuration as Code by specifying configScripts in your values.yaml file, see documentation: http://jenkins.k8s-t2.com/configuration-as-code and examples: https://github.com/jenkinsci/configuration-as-code-plugin/tree/master/demos
获知部署后的情况
orangepi@k8s-master-1:/k8s_apps/jenkins$ kubectl describe ingress jenkins
Name: jenkins
Labels: app.kubernetes.io/component=jenkins-controller
app.kubernetes.io/instance=jenkins
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=jenkins
helm.sh/chart=jenkins-4.3.23
Namespace: default
Address:
Ingress Class: nginx
Default backend:
Rules:
Host Path Backends
---- ---- --------
jenkins.k8s-t2.com
/jenkins jenkins:8080 (10.244.2.7:8080)
Annotations: kubernetes.io/ingress.class: nginx
meta.helm.sh/release-name: jenkins
meta.helm.sh/release-namespace: default
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 11s (x2 over 5m36s) nginx-ingress-controller Scheduled for sync
然后就可以欢快地访问 jenkins.k8s-t2.com了
3. 相关命令
(jenkins_url)/safeRestart – 允许所有正在运行的作业完成。 重新启动完成后，新作业将保留在队列中运行。
(jenkins_url)/restart – 强制重启，无需等待生成完成。
4. 遇到的问题
- 启动pod时出现
default-scheduler 0/3 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }
查找安装的目标Server是否有污点 "node-role.kubernetes.io/control-plane"，有则去掉或修改
- 在jenkins安装插件时，出现:
unable to find valid certification path to requested target
解决办法：手动安装 skip-certificate-check，到这里下载skip-certificate-check | Jenkins plugin安装最新版本，目前是1.1
设置git仓库时，提示 stderr: No ECDSA host key is known for and you have requested strict checking.
解决办法： Manage Jenkins => Security => Git-Host-Key-Verification 修改为 No verificationssh - stderr: No ECDSA host key is known for github.com and you have requested strict checking - Ask Ubuntu
5. 参考
Kubernetes
kubernetes(十四) 基于kubernetes的jenkins持续集成-腾讯云开发者社区-腾讯云
Managing Plugins
https://medium.com/javarevisited/deploying-a-spring-boot-application-on-kubernetes-using-jenkins-672961425a42