逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送

目录

一、软件交付流程介绍

1、瀑布式流程

2、敏捷开发

3、DevOps

二、Kubernetes集群部署Jenkins

1、jenkins的部署启动

​2、jenkins初始化的一些操作

三、Kubernetes集群部署gitlab

1、部署 postgres依赖

2、部署 redis依赖

3、部署 gitlab组件

4、测试项目推送gitlab库 

四、jenkins联动gitlab测试

1、安装gitlab plugin

2、配置gitlab的访问token 

3、配置host解析

4、jenkins配置Gitlab插件

5、创建自由风格项目

6、gitlab配置webhook

7、测试能否正常推送钉钉

五、Master-Slaves(agent)模式

1、创建slave节点

2、执行java命令启动agent服务

3、配置构建任务指向slave节点


        本章基于k8s集群部署gitlab、sonarQube、Jenkins等工具,并把上述工具集成到Jenkins中,以Django项目和SpringBoot项目为例,通过多分支流水线及Jenkinsfile实现项目代码提交到不同的仓库分支,实现自动代码扫描、单元测试、docker容器构建、k8s服务的自动部署。

一、软件交付流程介绍

        一个软件从零开始到最终交付,大概包括以下几个阶段:规划、编码、构建、测试、发布、部署和维护,基于这些阶段,我们的软件交付模型大致经历了几个阶段:

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第1张图片

1、瀑布式流程

        前期需求确立之后,软件开发人员花费数周和数月编写代码,把所有需求一次性开发完,然后将代码交给QA(质量保障)团队进行测试,然后将最终的发布版交给运维团队去部署。瀑布模型,简单来说,就是等一个阶段所有工作完成之后,再进入下一个阶段。这种模式的问题也很明显,产品迭代周期长,灵活性差。一个周期动辄几周几个月,适应不了当下产品需要快速迭代的场景。

2、敏捷开发

        任务由大拆小,开发、测试协同工作,注重开发敏捷,不重视交付敏捷

3、DevOps

        开发、测试、运维协同工作, 持续开发+持续交付。

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第2张图片

DevOps的核心就是自动化

devops可以看作一种提倡开发、测试、运维协同工作来实现持续开发、持续交付的一种软件交付模式 + 基于工具和技术支撑的自动化流程的落地实践。devops不是某一个具体的技术,而是一种思想+自动化能力,来使得构建、测试、发布软件能够更加地便捷、频繁和可靠的落地实践

二、Kubernetes集群部署Jenkins

        一个可扩展的持续集成引擎,用于自动化各种任务,包括构建、测试和部署软件

官网:https://www.jenkins.io/zh/doc/book/installing/https://www.jenkins.io/zh/doc/book/installing/

1、jenkins的部署启动

注意点:

  1. 第一次启动很慢

  2. 因为后面Jenkins会与kubernetes集群进行集成,会需要调用kubernetes集群的api,因此安装的时候创建了ServiceAccount并赋予了cluster-admin的权限

  3. 默认部署到jenkins=true的节点

  4. 初始化容器来设置权限

  5. ingress来外部访问

  6. 数据存储通过hostpath挂载到宿主机中

[root@k8s-master jenkins]# cat jenkins-all.yaml 
apiVersion: v1
kind: Namespace
metadata:
  name: jenkins
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: jenkins-crb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: jenkins
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-master
  namespace: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      devops: jenkins-master
  template:
    metadata:
      labels:
        devops: jenkins-master
    spec:
      nodeSelector:
        jenkins: "true"
      serviceAccount: jenkins #Pod 需要使用的服务账号
      initContainers:
      - name: fix-permissions
        image: busybox
        command: ["sh", "-c", "chown -R 1000:1000 /var/jenkins_home"]
        securityContext:
          privileged: true
        volumeMounts:
        - name: jenkinshome
          mountPath: /var/jenkins_home
      containers:
      - name: jenkins
        image: jenkinsci/blueocean:1.23.2
        imagePullPolicy: IfNotPresent
        ports:
        - name: http #Jenkins Master Web 服务端口
          containerPort: 8080
        - name: slavelistener #Jenkins Master 供未来 Slave 连接的端口
          containerPort: 50000
        volumeMounts:
        - name: jenkinshome
          mountPath: /var/jenkins_home
        env:
        - name: JAVA_OPTS
          value: "-Xms4096m -Xmx5120m -Duser.timezone=Asia/Shanghai -Dhudson.model.DirectoryBrowserSupport.CSP="
      volumes:
      - name: jenkinshome
        hostPath:
          path: /var/jenkins_home/
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: jenkins
spec:
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: slavelistener
    port: 50000
    targetPort: 50000
  type: ClusterIP
  selector:
    devops: jenkins-master
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jenkins-web
  namespace: jenkins
spec:
  rules:
  - host: jenkins.haha.com
    http:
      paths:
      - backend:
          serviceName: jenkins
          servicePort: 8080
        path: /
创建服务:

## 为k8s-node1打标签,将jenkins-master部署在k8s-node1节点
$ kubectl label node k8s-node1 jenkins=true
## 部署服务
$ kubectl create -f jenkins-all.yaml
## 查看服务
[root@k8s-master jenkins]# kubectl get po -n jenkins
NAME                             READY   STATUS    RESTARTS   AGE
jenkins-master-6b4c77dcc-hjmtv   1/1     Running   2          4d15h
# 查看日志,第一次启动提示需要完成初始化设置
$ kubectl -n jenkins logs -f jenkins-master-767df9b574-lgdr5
......
*************************************************************
​
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
​
5396b4e1c395450f8360efd8ee641b18
​
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
​
*************************************************************

​2、jenkins初始化的一些操作

访问服务:

配置hosts解析,192.168.0.122 jenkins.rui.com,然后使用浏览器域名访问服务。第一次访问需要大概几分钟的初始化时间。

注意:

部署jenkins和gitlab是配置的linux的hosts、windows的hosts,容器coredns的cm域名解析,都是配置192.168.0.122,这个地址不是容器实际所在节点,而是k8s集群ingress部署的节点。

使用jenkins启动日志中的密码,或者执行下面的命令获取解锁的管理员密码:

$ kubectl -n jenkins exec jenkins-master-767df9b574-lgdr5 bash 
/ # cat /var/jenkins_home/secrets/initialAdminPassword
35b083de1d25409eaef57255e0da481a
​

点击叉号,跳过选择安装推荐的插件环节,直接进入Jenkins。由于默认的插件地址安装非常慢,我们可以替换成国内清华的源,进入 jenkins 工作目录,目录下面有一个 updates 的目录,下面有一个 default.json 文件,我们执行下面的命令替换插件地址:

$ cd /var/jenkins_home/updates
$ sed -i 's/http:\/\/updates.jenkins-ci.org\/download/https:\/\/mirrors.tuna.tsinghua.edu.cn\/jenkins/g' default.json 
$ sed -i 's/http:\/\/www.google.com/https:\/\/www.baidu.com/g' default.json
​暂时先不用重新启动pod,汉化后一起重启。

选择右上角admin->configure->password重新设置管理员密码,设置完后,会退出要求重新登录,使用admin/xxxxxx(新密码),登录即可。

安装汉化插件

Jenkins -> manage Jenkins -> Plugin Manager -> Avaliable,搜索 chinese关键字

选中后,选择[Install without restart],等待下载完成,然后点击[ Restart Jenkins when installation is complete and no jobs are running ],让Jenkins自动重启

启动后,界面默认变成中文。

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第3张图片

三、Kubernetes集群部署gitlab

        代码仓库,企业内部使用最多的代码版本管理工具。

gitlab代码仓库搭建

GitHub - sameersbn/docker-gitlab: Dockerized GitLab

## 全量部署的组件
$ gitlab-ctl status
run: alertmanager: (pid 1987) 27s; run: log: (pid 1986) 27s
run: gitaly: (pid 1950) 28s; run: log: (pid 1949) 28s
run: gitlab-exporter: (pid 1985) 27s; run: log: (pid 1984) 27s
run: gitlab-workhorse: (pid 1956) 28s; run: log: (pid 1955) 28s
run: logrotate: (pid 1960) 28s; run: log: (pid 1959) 28s
run: nginx: (pid 2439) 1s; run: log: (pid 1990) 27s
run: node-exporter: (pid 1963) 28s; run: log: (pid 1962) 28s
run: postgres-exporter: (pid 1989) 27s; run: log: (pid 1988) 27s
run: postgresql: (pid 1945) 28s; run: log: (pid 1944) 28s
run: prometheus: (pid 1973) 28s; run: log: (pid 1972) 28s
run: puma: (pid 1968) 28s; run: log: (pid 1966) 28s
run: redis: (pid 1952) 28s; run: log: (pid 1951) 28s
run: redis-exporter: (pid 1971) 28s; run: log: (pid 1964) 28s
run: sidekiq: (pid 1969) 28s; run: log: (pid 1967) 28s

因为全量部署gitlab默认添加很多组建,本次采用精简部署,只部署必要依赖postgres和redis

1、部署 postgres依赖

把PostgreSQL的账号密码以secret形式存储
[root@k8s-master gitlab]# cat gitlab-secret.txt 
postgres.user.root=root
postgres.pwd.root=123456

$ kubectl -n jenkins create secret generic gitlab-secret --from-env-file=gitlab-secret.txt
[root@k8s-master gitlab]# cat postgres.yaml 
apiVersion: v1
kind: Service
metadata:
  name: postgres
  labels:
    app: postgres
  namespace: jenkins
spec:
  ports:
  - name: server
    port: 5432
    targetPort: 5432
    protocol: TCP
  selector:
    app: postgres
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: jenkins
  name: postgres
  labels:
    app: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      nodeSelector:
        postgres: "true"
      # 容忍所有节点
      tolerations:
      - operator: "Exists"
      containers:
      - name: postgres
        image:  postgres:11.4
        imagePullPolicy: "IfNotPresent"
        ports:
        - containerPort: 5432
        # 调用刚才创建的secret
        env:
        - name: POSTGRES_USER           #PostgreSQL 用户名
          valueFrom:
            secretKeyRef:
              name: gitlab-secret
              key: postgres.user.root
        - name: POSTGRES_PASSWORD       #PostgreSQL 密码
          valueFrom:
            secretKeyRef:
              name: gitlab-secret
              key: postgres.pwd.root
        resources:
          limits:
            cpu: 1000m
            memory: 2048Mi
          requests:
            cpu: 50m
            memory: 100Mi
        volumeMounts:
        - mountPath: /var/lib/postgresql/data
          name: postgredb
      volumes:
      - name: postgredb
        hostPath:
          path: /var/lib/postgres/
#部署到k8s-slave2节点
$ kubectl label node k8s-node2 postgres=true

#创建postgres
$ kubectl create -f postgres.yaml

# 创建数据库gitlab,为后面部署gitlab组件使用
$ kubectl -n jenkins exec -ti postgres-7ff9b49f4c-nt8zh bash
root@postgres-7ff9b49f4c-nt8zh:/# psql
root=# create database gitlab;
CREATE DATABASE

2、部署 redis依赖

[root@k8s-master gitlab]# cat redis.yaml 
apiVersion: v1
kind: Service
metadata:
  name: redis
  labels:
    app: redis
  namespace: jenkins
spec:
  ports:
  - name: server
    port: 6379
    targetPort: 6379
    protocol: TCP
  selector:
    app: redis
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: jenkins
  name: redis
  labels:
    app: redis
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      tolerations:
      - operator: "Exists"
      containers:
      - name: redis
        image:  sameersbn/redis:4.0.9-2
        imagePullPolicy: "IfNotPresent"
        ports:
        - containerPort: 6379
        resources:
          limits:
            cpu: 1000m
            memory: 2048Mi
          requests:
            cpu: 50m
            memory: 100Mi

3、部署 gitlab组件

        注意点:

  • 使用ingress暴漏服务

  • 添加annotation,指定nginx端上传大小限制,否则推送代码时会默认被限制1m大小,相当于给nginx设置client_max_body_size的限制大小

  • 使用gitlab=true来选择节点

  • 使用服务发现地址来访问postgres和redis

  • 在secret中引用数据库账户和密码

  • 数据库名称为gitlab

[root@k8s-master gitlab]# cat gitlab.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gitlab
  namespace: jenkins
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
spec:
  rules:
  - host: gitlab.haha.com
    http:
      paths:
      - backend:
          serviceName: gitlab
          servicePort: 80
        path: /
---
apiVersion: v1
kind: Service
metadata:
  name: gitlab
  labels:
    app: gitlab
  namespace: jenkins
spec:
  ports:
  - name: server
    port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: gitlab
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: jenkins
  name: gitlab
  labels:
    app: gitlab
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitlab
  template:
    metadata:
      labels:
        app: gitlab
    spec:
      nodeSelector:
        gitlab: "true"
      tolerations:
      - operator: "Exists"
      containers:
      - name: gitlab
        image:  sameersbn/gitlab:13.2.2
        imagePullPolicy: "IfNotPresent"
        env:
        - name: GITLAB_HOST
          value: "gitlab.haha.com"
        - name: GITLAB_PORT
          value: "80"
        - name: GITLAB_SECRETS_DB_KEY_BASE
          value: "long-and-random-alpha-numeric-string"
        - name: GITLAB_SECRETS_DB_KEY_BASE
          value: "long-and-random-alpha-numeric-string"
        - name: GITLAB_SECRETS_SECRET_KEY_BASE
          value: "long-and-random-alpha-numeric-string"
        - name: GITLAB_SECRETS_OTP_KEY_BASE
          value: "long-and-random-alpha-numeric-string"
        # 服务发现配置
        - name: DB_HOST
          value: "postgres"
        - name: DB_NAME
          value: "gitlab"
        - name: DB_USER
        # 数据库账号密码读取
          valueFrom:
            secretKeyRef:
              name: gitlab-secret
              key: postgres.user.root
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: gitlab-secret
              key: postgres.pwd.root
        - name: REDIS_HOST
          value: "redis"
        - name: REDIS_PORT
          value: "6379"
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 2000m
            memory: 5048Mi
          requests:
            cpu: 100m
            memory: 500Mi
        volumeMounts:
        - mountPath: /home/git/data
          name: data
      volumes:
      - name: data
        hostPath:
          path: /var/lib/gitlab/
linux:
[root@k8s-master gitlab]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.0.121 k8s-master 
192.168.0.122 k8s-node1 gitlab.haha.com
192.168.0.123 k8s-node2


[root@k8s-master python-demo]# git push -u origin --all
fatal: unable to access 'http://gitlab.haha.com/root/myblog.git/': Failed connect to gitlab.haha.com:80; Connection refused

如果报错,尝试将每个ip都解析gitlab.haha.com

windows:
C:\Windows\System32\drivers\etc\hosts
192.168.0.122  gitlab.haha.com

        gitlab第一次启动较慢,需要初始化数据库等操作,具体日志如下,等到日志刷完,页面才能正常访问。

Configuring gitlab::gitlab-workhorse...
Configuring gitlab::puma...
Configuring gitlab::timezone...
Configuring gitlab::rack_attack...
Configuring gitlab::ci...
Configuring gitlab::artifacts...
Configuring gitlab::lfs...
Configuring gitlab::uploads...
Configuring gitlab::mattermost...
Configuring gitlab::project_features...
Configuring gitlab::oauth...
Configuring gitlab::ldap...
Configuring gitlab::cron_jobs...
Configuring gitlab::backups...
Configuring gitlab::registry...
Configuring gitlab::pages...
Configuring gitlab::sentry...
Configuring gitlab-shell...
Configuring nginx...
Configuring nginx::gitlab...
Setting up GitLab for firstrun. Please be patient, this could take a while...
2021-12-09 13:02:06,576 CRIT Supervisor running as root (no user in config file)
2021-12-09 13:02:06,578 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2021-12-09 13:02:06,580 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
2021-12-09 13:02:06,580 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
2021-12-09 13:02:06,582 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
2021-12-09 13:02:06,584 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
2021-12-09 13:02:06,585 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2021-12-09 13:02:06,585 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
2021-12-09 13:02:06,585 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
2021-12-09 13:02:06,586 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
2021-12-09 13:02:06,637 INFO RPC interface 'supervisor' initialized
2021-12-09 13:02:06,639 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2021-12-09 13:02:06,642 INFO supervisord started with pid 607
2021-12-09 13:02:07,646 INFO spawned: 'gitaly' with pid 621
2021-12-09 13:02:07,650 INFO spawned: 'puma' with pid 622
2021-12-09 13:02:07,653 INFO spawned: 'gitlab-workhorse' with pid 623
2021-12-09 13:02:07,669 INFO spawned: 'sidekiq' with pid 624
2021-12-09 13:02:07,672 INFO spawned: 'nginx' with pid 625
2021-12-09 13:02:07,681 INFO spawned: 'sshd' with pid 626
2021-12-09 13:02:07,684 INFO spawned: 'cron' with pid 627
2021-12-09 13:02:09,366 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:02:09,390 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:02:09,390 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:02:09,391 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:02:09,392 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:02:09,392 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:02:09,394 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:03:13,664 INFO exited: sidekiq (exit status 1; not expected)
2021-12-09 13:03:14,678 INFO spawned: 'sidekiq' with pid 744
2021-12-09 13:03:15,683 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:03:16,245 INFO exited: puma (exit status 1; not expected)
2021-12-09 13:03:17,250 INFO spawned: 'puma' with pid 748
2021-12-09 13:03:18,252 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Database 'gitlab' already exists
Migrating database...
Clearing cache...
2021-12-09 13:06:41,063 WARN received SIGTERM indicating exit request
2021-12-09 13:06:41,064 INFO waiting for gitaly, puma, gitlab-workhorse, sidekiq, nginx, sshd, cron to die
2021-12-09 13:06:41,069 INFO stopped: cron (terminated by SIGTERM)
2021-12-09 13:06:41,070 INFO stopped: sshd (exit status 0)
2021-12-09 13:06:41,086 INFO stopped: nginx (exit status 0)
2021-12-09 13:06:43,709 INFO stopped: sidekiq (exit status 0)
2021-12-09 13:06:43,712 INFO stopped: gitlab-workhorse (terminated by SIGTERM)
2021-12-09 13:06:44,793 INFO waiting for gitaly, puma to die
2021-12-09 13:06:47,797 INFO waiting for gitaly, puma to die
2021-12-09 13:06:50,800 INFO waiting for gitaly, puma to die
2021-12-09 13:06:53,805 WARN killing 'puma' (748) with SIGKILL
2021-12-09 13:06:53,806 INFO waiting for gitaly, puma to die
2021-12-09 13:06:53,852 INFO stopped: puma (terminated by SIGKILL)
2021-12-09 13:06:53,893 INFO stopped: gitaly (exit status 0)
2021-12-09 13:06:55,079 CRIT Supervisor running as root (no user in config file)
2021-12-09 13:06:55,080 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
2021-12-09 13:06:55,081 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
2021-12-09 13:06:55,081 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
2021-12-09 13:06:55,082 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
2021-12-09 13:06:55,082 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
2021-12-09 13:06:55,083 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
2021-12-09 13:06:55,084 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
2021-12-09 13:06:55,085 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
2021-12-09 13:06:55,085 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
2021-12-09 13:06:55,174 INFO RPC interface 'supervisor' initialized
2021-12-09 13:06:55,174 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2021-12-09 13:06:55,176 INFO supervisord started with pid 1
2021-12-09 13:06:56,181 INFO spawned: 'gitaly' with pid 921
2021-12-09 13:06:56,184 INFO spawned: 'puma' with pid 922
2021-12-09 13:06:56,188 INFO spawned: 'gitlab-workhorse' with pid 923
2021-12-09 13:06:56,195 INFO spawned: 'sidekiq' with pid 924
2021-12-09 13:06:56,199 INFO spawned: 'nginx' with pid 925
2021-12-09 13:06:56,208 INFO spawned: 'sshd' with pid 926
2021-12-09 13:06:56,217 INFO spawned: 'cron' with pid 927
2021-12-09 13:06:57,693 INFO success: gitaly entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:06:57,694 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:06:57,695 INFO success: gitlab-workhorse entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:06:57,695 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:06:57,696 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:06:57,696 INFO success: sshd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-12-09 13:06:57,701 INFO success: cron entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

 启动完成后访问页面:

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第4张图片

4、测试项目推送gitlab库 

        把测试的demo推送至gitlab库,myblog项目推送到gitlab

         django项目介绍

        - 项目地址:https://gitee.com/agagin/python-demo.git
        - python3 + django + uwsgi + nginx + mysql

        - 内部服务端口8002

mkdir demo
cp -r myblog demo/
cd demo/myblog
yum install git -y
git config --global user.name "Administrator"
git config --global user.email "[email protected]"
git init
git remote add origin http://gitlab.rui.com/root/myblog.git
git add .
git commit -m "Initial commit"
git push -u origin master
​

 推送成功,gitlab部署完成,推送正常

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第5张图片

四、jenkins联动gitlab测试

演示目标

- 代码提交gitlab,自动触发Jenkins任务
- Jenkins任务完成后发送钉钉消息通知

 逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第6张图片

1、安装gitlab plugin

插件中心搜索并安装gitlab,直接安装即可

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第7张图片

2、配置gitlab的访问token 

获取AccessToken

登录gitlab,选择user->Settings->access tokens新建一个访问token

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第8张图片

3、配置host解析

由于我们的Jenkins和gitlab域名是本地解析,因此需要让gitlab和Jenkins服务可以解析到对方的域名。两种方式:

  • 在容器内配置hosts(不推荐)

  • 配置coredns的静态解析,122为hosts配置的gitlab的主机地址

    [root@k8s-master gitlab]# kubectl edit cm coredns  -n kube-system
    
    apiVersion: v1
    data:
      Corefile: |
        .:53 {
            errors
            health {
               lameduck 5s
            }
            hosts {
                192.168.0.122 jenkins.haha.com  gitlab.haha.com
                fallthrough
            }
            ready
            kubernetes cluster.local in-addr.arpa ip6.arpa {
               pods insecure
               fallthrough in-addr.arpa ip6.arpa
               ttl 30
            }
    稍等,等coredns热加载配置文件
    

4、jenkins配置Gitlab插件

系统管理->系统配置->Gitlab,配置之前gitlab生成的token并text通过

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第9张图片

5、创建自由风格项目

gitlab connection 选择为刚创建的gitlab

源码管理选择Git,填项项目地址:http://gitlab.haha.com/root/myblog.git

新建一个 Credentials 认证,使用用户名密码方式,配置gitlab的用户和密码

构建触发器选择 Build when a change is pushed to GitLab

点高级生成一个Secret token供gitlab的webhook使用

增加构建步骤推送钉钉:

curl 'https://oapi.dingtalk.com/robot/send?access_token=d7d8a3e17742d2153de5223ec6b9ba7ea24fe6616495c9318e11f82751cb0d09' \
   -H 'Content-Type: application/json' \
   -d '{"msgtype": "text", 
        "text": {
             "content": "gitlab推送成功"
        }
      }'

保存

6、gitlab配置webhook

进入项目下settings->Integrations

URL: http://jenkins.haha.com/project/ding01  注意是http,jenkins默认地址是https

Secret Token 填入在Jenkins端生成的token

Add webhook

test push events,报错:Requests to the local network are not allowed

设置gitlab允许向本地网络发送webhook请求

访问 小扳手Admin Aera -> Settings -> Network ,展开Outbound requests

Collapse,勾选第一项即可。再次test push events,成功。

7、测试能否正常推送钉钉

  • 提交代码到gitlab仓库或者直接text push
  • gitlab的webhook会根据url和token自动推送消息至jenkins的ding01自由风格任务
  • jenkins接到消息后自动执行构建步骤配置的shell脚本
  • jenkins控制线显示构建任务执行,执行完成
  • 钉钉收到消息

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第10张图片

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第11张图片

五、Master-Slaves(agent)模式

        上面演示的任务,默认都是在master节点执行的,多个任务都在master节点执行,对master节点的性能会造成一定影响,如何将任务分散到不同的节点,做成多slave的方式?

        目前slave采用固定节点模式,以后对接kubernetes后,可以配置创建临时slave容器进行构建,待更新。

1、创建slave节点

  • 系统管理 -> 节点管理 -> 新建节点

  • 比如添加192.168.0.123,选择固定节点,保存

  • 远程工作目录/opt/jenkins_jobs

  • 标签为任务选择节点的依据,如192.168.0.123

  • 启动方式选择通过java web启动代理,代理是运行jar包,通过JNLP(是一种允许客户端启动托管在远程Web服务器上的应用程序的协议 )启动连接到master节点服务中

2、执行java命令启动agent服务

逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第12张图片

## 登录192.168.0.123,下载agent.jar
$ wget http://jenkins.luffy.com/jnlpJars/agent.jar
## 会提示找不到agent错误,因为没有配置地址解析,由于连接jenkins master会通过50000端口,直接使用cluster-ip
$ kubectl -n jenkins get svc #在master节点执行查询cluster-ip地址
NAME      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)              AGE
jenkins   ClusterIP   10.99.204.208           8080/TCP,50000/TCP   4h8m
​
## 再次回到68节点
$ wget 10.99.204.208:8080/jnlpJars/agent.jar
​
[root@k8s-node2 jenkins-job]# java -jar agent.jar -jnlpUrl http://10.1.120.64:8080/computer/23/slave-agent.jnlp -secret @secret-file -workDir "/opt/jenkins-job"
十二月 18, 2021 7:25:16 下午 org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
信息: Using /opt/jenkins-job/remoting as a remoting work directory
十二月 18, 2021 7:25:16 下午 org.jenkinsci.remoting.engine.WorkDirManager setupLogging
信息: Both error and output logs will be printed to /opt/jenkins-job/remoting
十二月 18, 2021 7:25:16 下午 hudson.remoting.jnlp.Main createEngine
信息: Setting up agent: 23
十二月 18, 2021 7:25:16 下午 hudson.remoting.jnlp.Main$CuiListener 
信息: Jenkins agent is running in headless mode.
十二月 18, 2021 7:25:16 下午 hudson.remoting.Engine startEngine
信息: Using Remoting version: 4.5
十二月 18, 2021 7:25:16 下午 org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
信息: Using /opt/jenkins-job/remoting as a remoting work directory
十二月 18, 2021 7:25:16 下午 hudson.remoting.jnlp.Main$CuiListener status
信息: Locating server among [http://jenkins.rui.com/]
十二月 18, 2021 7:25:16 下午 org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
信息: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
十二月 18, 2021 7:25:16 下午 org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
信息: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check
十二月 18, 2021 7:25:16 下午 hudson.remoting.jnlp.Main$CuiListener status
信息: Agent discovery successful
  Agent address: 10.1.120.64
  Agent port:    50000
  Identity:      5c:c5:fa:97:e5:25:0b:78:fd:57:21:b7:1d:a5:2f:d5
十二月 18, 2021 7:25:16 下午 hudson.remoting.jnlp.Main$CuiListener status
信息: Handshaking
十二月 18, 2021 7:25:16 下午 hudson.remoting.jnlp.Main$CuiListener status
信息: Connecting to 10.1.120.64:50000
十二月 18, 2021 7:25:16 下午 hudson.remoting.jnlp.Main$CuiListener status
信息: Trying protocol: JNLP4-connect
十二月 18, 2021 7:25:17 下午 hudson.remoting.jnlp.Main$CuiListener status
信息: Remote identity confirmed: 5c:c5:fa:97:e5:25:0b:78:fd:57:21:b7:1d:a5:2f:d5
十二月 18, 2021 7:25:18 下午 hudson.remoting.jnlp.Main$CuiListener status
信息: Connected

​
若出现如下错误:

警告: 拒绝连接
十二月 18, 2021 7:21:47 下午 hudson.remoting.jnlp.Main$CuiListener error
严重: http://jenkins.rui.com/ provided port:50000 is not reachable
java.io.IOException: http://jenkins.rui.com/ provided port:50000 is not reachable
	at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:314)
	at hudson.remoting.Engine.innerRun(Engine.java:694)
	at hudson.remoting.Engine.run(Engine.java:519)

可以选择: 配置从节点 -> 高级 -> Tunnel连接位置,进行设置:

3、配置构建任务指向slave节点

  1. 查看Jenkins节点列表,新节点已经处于可用状态

    逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第13张图片

  2. 测试使用新节点执行任务

    • 配置free项目

    • 限制项目的运行节点 ,标签表达式选择slave的标签逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第14张图片

    • 立即构建

    • 查看构建日志逃脱只会部署集群系列 —— jenkins和gitlab的部署联动推送_第15张图片

你可能感兴趣的:(kubernetes,Jenkins,jenkins,devops,gitlab,kubernetes,k8s)