k8s学习之jenkins主从部署-pipeline
k8s–jenkins主从-pipeline
可以实现如下功能：
多集群管理
可以根据客户需求对开发，测试，生产环境部署多套kubernetes集群，每个环境使用独立的物理资源，相互之间避免影响
多环境一致性
Kubernetes是基于docker的容器编排工具，因为容器的镜像是不可变的，所以镜像把 OS、业务代码、运行环境、程序库、目录结构都包含在内，镜像保存在我们的私有仓库，只要用户从我们提供的私有仓库拉取镜像，就能保证环境的一致性
持续集成，持续部署，持续交付
可以让产品快速迭代，自动部署，根据客户的要求达到持续交付的能力
第一部分：在k8s集群安装jenkins
1.部署nfs服务
[root@master ~]# yum -y install nfs-utils
[root@master ~]# systemctl enable nfs --now
[root@master ~]# mkdir /data/v1 -p
[root@master ~]# echo "/data/v1 192.168.1.0/24(rw,no_root_squash)" > /etc/exports
[root@master ~]# exportfs -arv
exporting 192.168.1.0/24:/data/v1
[root@master ~]# showmount -e
Export list for master:
/data/v1 192.168.1.0/24
2.kubernetes中部署jenkins
创建pv
cat >pv.yaml<[root@master ~]# kubectl create namespace jenkins-k8s
[root@master ~]# kubectl apply -f pv.yaml
创建pvc
[root@master ~]# cat pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: jenkins-k8s-pvc
namespace: jenkins-k8s
spec:
resources:
requests:
storage: 10Gi
accessModes:
- ReadWriteMany
[root@master ~]# kubectl apply -f pv.yaml
[root@master ~]# kubectl get pv -n jenkins-k8s
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM
jenkins-k8s-pv 10Gi RWX Retain Bound jenkins-k8s/jenkins-k8s-pvc
[root@master ~]# kubectl get pvc -n jenkins-k8s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
jenkins-k8s-pvc Bound jenkins-k8s-pv 10Gi RWX 91s
创建一个sa账号,做rbac授权
[root@master ~]# kubectl create sa jenkins-k8s-sa -n jenkins-k8s
[root@master ~]# kubectl create clusterrolebinding jenkins-k8s-sa-cluster -n jenkins-k8s --clusterrole=cluster-admin --serviceaccount=jenkins-k8s:jenkins-k8s-sa
通过deployment部署jenkins
[root@master ~]# cat jenkins-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: jenkins
namespace: jenkins-k8s
spec:
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
serviceAccount: jenkins-k8s-sa
containers:
- name: jenkins
image: jenkins/jenkins:2.328
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: web
protocol: TCP
- containerPort: 50000
name: agent
protocol: TCP
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
livenessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
readinessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
volumeMounts:
- name: jenkins-volume
subPath: jenkins-home
mountPath: /var/jenkins_home
volumes:
- name: jenkins-volume
persistentVolumeClaim:
claimName: jenkins-k8s-pvc
[root@master ~]# kubectl apply -f jenkins-deploy.yaml
[root@master ~]# kubectl delete -f jenkins-deploy.yaml
[root@master ~]# chown -R 1000：1000 /data/v1 #否则无法写入，pod无法启动
[root@master ~]# kubectl apply -f jenkins-deploy.yaml
deployment.apps/jenkins created
[root@master ~]# kubectl get pod -n jenkins-k8s -w
NAME READY STATUS RESTARTS AGE
jenkins-75cf9cf9d5-9w5k9 0/1 Running 0 4s
jenkins-75cf9cf9d5-9w5k9 1/1 Running 0 63s
[root@master ~]# kubectl get pod -n jenkins-k8s -owide
NAME READY STATUS RESTARTS AGE IP
jenkins-75cf9cf9d5-9w5k9 1/1 Running 0 15m 10.244.1.105
[root@master ~]# curl 10.244.1.106:8080
把jenkins前端加上service，提供外部网络访问
[root@master ~]# cat jenkins-service.yaml
apiVersion: v1
kind: Service
metadata:
name: jenkins-service
namespace: jenkins-k8s
labels:
app: jenkins
spec:
selector:
app: jenkins
type: NodePort
ports:
- name: web
port: 8080
targetPort: web
nodePort: 30002
- name: agent
port: 50000
targetPort: agent
[root@master ~]# kubectl apply -f jenkins-service.yaml
service/jenkins-service created
[root@master ~]# kubectl get svc -n jenkins-k8s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins-service NodePort 10.104.176.186 8080:30002/TCP,50000:31857/TCP
在浏览器访问jenkins的web界面：192.168.1.12:30002
[root@master secrets]# cd /data/v1/jenkins-home/secrets/
[root@master secrets]# cat initialAdminPassword
7796fdb870804d6794d51cf9df409010
若无法正常配置，可尝试http://192.168.1.12:30002/restart
1.在Jenkins中安装kubernetes插件
http://192.168.1.12:30002/restart
配置k8s
添加pod模板…
下面的容器名test请换成jnlp
[root@master ~]# scp -r .kube/ node2:/root/
节点上需要有这个目录
在执行pipeline时，jenkins可以直接调用pod模板，跑一个从slave
2.添加自己的dockerhub凭据 ，也可以自己搭建hub仓库
pipeline脚本
node('testhan') {
//node()不写，不会指定jenkins标签的节点
stage('Clone') {
echo "1.Clone Stage"
git url: "https://github.com/luckylucky421/jenkins-sample.git"
//pipeline注释用//开头，上面是git仓库，拉取代码
script {
build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
}
}
stage('Test') {
echo "2.Test Stage"
}
stage('Build') {
echo "3.Build Docker Image Stage"
sh "docker build -t daydayup9527/jenkins-demo:${build_tag} ."
}
//仓库名/镜像名:标签变量
stage('Push') {
echo "4.Push Docker Image Stage"
withCredentials([usernamePassword(credentialsId: 'dockerhub', passwordVariable: 'dockerHubPassword', usernameVariable: 'dockerHubUser')]) {
//变量调用配置好的仓库信息
sh "docker login -u ${dockerHubUser} -p ${dockerHubPassword}"
sh "docker push daydayup9527/jenkins-demo:${build_tag}"
}
}
stage('Deploy to dev') {
echo "5. Deploy DEV"
sh "sed -i 's//${build_tag}/' k8s-dev.yaml"
//这里的文件是仓库里的文件，一般固定
sh "sed -i 's//${env.BRANCH_NAME}/' k8s-dev.yaml"
// sh "bash running-devlopment.sh"
sh "kubectl apply -f k8s-dev.yaml --validate=false"
}
stage('Promote to qa') {
def userInput = input(
//定义一个输入函数
id: 'userInput',
message: 'Promote to qa?',
parameters: [
[
$class: 'ChoiceParameterDefinition',
choices: "YES\nNO",
name: 'Env'
]
]
)
echo "This is a deploy step to ${userInput}"
if (userInput == "YES") {
//输入YES开始部署
sh "sed -i 's//${build_tag}/' k8s-qa.yaml"
sh "sed -i 's//${env.BRANCH_NAME}/' k8s-qa.yaml"
// sh "bash running-qa.sh"
sh "kubectl apply -f k8s-qa.yaml --validate=false"
sh "sleep 6"
sh "kubectl get pods -n qa"
} else {
//exit
}
}
stage('Promote to pro') {
def userInput = input(
id: 'userInput',
message: 'Promote to pro?',
parameters: [
[
$class: 'ChoiceParameterDefinition',
choices: "YES\nNO",
name: 'Env'
]
]
)
echo "This is a deploy step to ${userInput}"
if (userInput == "YES") {
sh "sed -i 's//${build_tag}/' k8s-prod.yaml"
sh "sed -i 's//${env.BRANCH_NAME}/' k8s-prod.yaml"
// sh "bash running-production.sh"
sh "cat k8s-prod.yaml"
sh "kubectl apply -f k8s-prod.yaml --record --validate=false"
}
}
}
[root@master ~]# kubectl create namespace deployment
[root@master ~]# kubectl create namespace deployment
[root@master ~]# kubectl create namespace qatest
这里是根据git仓库里的文件配置，按需要创建
这里的运行，相当于前面的构建
可以一步一步的进行构建，
[root@master ~]# kubectl get service -n deployment
[root@master ~]# kubectl get service -n production
[root@master ~]# kubectl get service -n qatest
根据相应的端口信息访问测试验证
回滚操作
当修改代码，再次构建时
kubectl rollout history deploy -n production |awk '{print $1}' |grep -v deploy|grep -v REVISION >version.csv
cat version.csv
1
2
node('testhan') {
stage('git clone') {
git url: "https://github.com/luckylucky421/jenkins-rollout"
sh "ls -al"
sh "pwd" }
stage('select env') {
def envInput = input( id: 'envInput',
message: 'Choose a deploy environment',
parameters: [
[
$class: 'ChoiceParameterDefinition',
choices: "devlopment\nqatest\nproduction",
//实际环境的命名空间
name: 'Env'
]
]
)
echo "This is a deploy step to ${envInput}"
sh "sed -i 's//${envInput}/' getVersion.sh"
sh "sed -i 's//${envInput}/' rollout.sh"
sh "bash getVersion.sh"
// env.WORKSPACE = pwd()
// def version = readFile "${env.WORKSPACE}/version.csv"
// println version
}
stage('select version') {
env.WORKSPACE = pwd()
def version = readFile "${env.WORKSPACE}/version.csv"
println version
def userInput = input(
id: 'userInput',
message: '选择回滚版本',
parameters: [
[
$class: 'ChoiceParameterDefinition',
choices: "$version\n",
name: 'Version'
]
]
)
sh "sed -i 's//${userInput}/' rollout.sh"
}
stage('rollout deploy') {
sh "bash rollout.sh"
}
}
