基于Kubernetes的DevOps平台企业级实践（一）

概况

基于k8s集群部署gitlab、Jenkins等工具，并把上述工具集成到Jenkins中，通过pipline及Jenkinsfile实现项目代码提交、自动代码扫描、单元测试、docker容器构建、k8s服务的自动部署。演示的架构是以gitlab+jenkins（pipline流水线）+kubernetes+钉钉（结果通知）。

kubernetes环境部署jenkins

注意点

1.因为后面Jenkins会与kubernetes集群进行集成，会需要调用kubernetes集群的api，因此安装的 时候创建了ServiceAccount并赋予了cluster-admin的权限;

2.由于每次部署Jenkins环境，均需要安装很多必要的插件，因此考虑把插件提前做到镜像中。

Jenkins定制化容器
Dockerfile

FROM  jenkinsci/blueocean
LABEL  maintainer="[email protected]"

## 用最新的插件列表文件替换默认插件文件
COPY  plugins.txt  /usr/share/jenkins/ref/

## 执行插件安装
RUN  /usr/local/bin/install-plugins.sh  <  /usr/share/jenkins/ref/plugins.txt

plugins.txt

ace-editor:1.1
allure-jenkins-plugin:2.28.1 ant:1.10
antisamy-markup-formatter:1.6
apache-httpcomponents-client-4-api:4.5.10-1.0 

#执行构建定制的jenkins容器
$ docker build -t wanglei/jenkins:v20200814 ./

部署jenkins

#vim jenkins-master.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: jenkins
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: jenkins-crb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: jenkins
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins-master
  namespace: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      devops: jenkins-master
  template:
    metadata:
      labels:
        devops: jenkins-master
    spec:
      nodeSelector:
        jenkins: "true"
#      tolerations:
#      - operator: "Exists"
      serviceAccount: jenkins #Pod 需要使用的服务账号 initContainers:
      initContainers:
      - name: fix-permissions
        image: busybox
        command: ["sh", "-c", "chown -R 1000:1000 /var/jenkins_home"]
        securityContext:
          privileged: true
        volumeMounts:
        - name: jenkinshome
          mountPath: /var/jenkins_home
      containers:
      - name: jenkins
        image: jenkinsci/blueocean:latest
        imagePullPolicy: IfNotPresent
        ports:
        - name: http #Jenkins Master Web 服务端口
          containerPort: 8080
        - name: slavelistener #Jenkins Master 供未来 Slave 连接的端口
          containerPort: 50000
        volumeMounts:
        - name: jenkinshome
          mountPath: /var/jenkins_home
        env:
        - name: JAVA_OPTS
          value: "-Xms4096m -Xmx5120m -Duser.timezone=Asia/Shanghai -Dhudson.model.DirectoryBrowserSupport.CSP="
      volumes:
      - name: jenkinshome
        hostPath:
          path: /var/jenkins_home/
---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: jenkins
spec:
  type: NodePort
  ports:
  - name: http
    port: 8080
    targetPort: 8080
  - name: slavelistener
    port: 50000
    targetPort: 50000
  selector:
    devops: jenkins-master

部署jenkins的pod

#kubectl apply -f jenkins-master.yaml
访问jenkins web界面

gitlab与jenkins集成

gitlab代码仓库搭建

# docker run -d  --hostname 10.248.190.7 -p 8443:443 -p 80:80 -p 8022:22 --name gitlab  --restart always -v /opt/gitlab/config:/etc/gitlab -v /opt/gitlab/logs:/var/log/gitlab -v /opt/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:latest

流程示意图

未完待续。。。