gitlab-runner使用kubernetes executor

gitlabrunner使用k8s executor

gitlab-runner使用docker executor

所需镜像

• gitlab-runner:latest
• gitlab-runner-helper:x86_64-f100a208 这个由gitlab-runner版本和服务器版本决定
• maven:3-jdk-8 这个根据部署项目语言等决定
• busybox:latest 默认镜像

第一步：在K8S中部署gitlab-runner

在gitlab上找到url和token

1. 打开你想要配置CICD的项目或项目组

2. runner配置地址：设置->CICD->Runner->展开

3. 找到设置runner需要的url和token，拿小本本记下来

4. 生成toml

docker run -it gitlab-runner /bin/bash
gitlab-runner register

大致需要以下信息

• url：小本本上有
• token：小本本上有
• description：myrunner with docker executor
• tags：docker,node1
• executor：docker

根据gitlab-runner容器内的config.toml，生成以下yml

kind: ConfigMap
apiVersion: v1
metadata:
  name: gitlab-runner
  namespace: gitlab-runner
data:
  config.toml: |
    concurrent = 4
    check_interval = 0

    [session_server]
      session_timeout = 1800

    [[runners]]
      name = "first test"
      url = "http://$(gitlabIP)"
      token ="......"
      executor = "kubernetes"
      [runners.kubernetes]
        host = ""
        bearer_token_overwrite_allowed = true
        privileged = true
        #这是runner的默认镜像；具体镜像maven:3-jdk-8在.gitlab-ci.yml中配置
        image = "$(registryIP)/gitlab-runner/busybox:latest"  
        namespace = "gitlab-runner"
        namespace_overwrite_allowed = ""
        cpu_limit = "1"
        memory_limit = "2Gi"
        cpu_request = "1"
        memory_request = "2Gi"
        service_cpu_limit = "1"
        service_memory_limit = "2Gi"
        service_cpu_request = "1"
        service_memory_request = "2Gi"
        helper_cpu_limit = "1"
        helper_memory_limit = "2Gi"
        helper_cpu_request = "1"
        helper_memory_request = "2Gi"
        service_account = "gitlab"
        helper_image = "$(registryIP)/gitlab-runner/gitlab-runner-helper:x86_64-f100a208"
        [[runners.kubernetes.volumes.config_map]]
          name = "gitlab-runner-maven"
          mount_path = "/usr/share/maven/conf2/"

创建configmap：gitlab-runner-maven，保存maven的setting文件

kind: ConfigMap
apiVersion: v1
metadata:
  name: gitlab-runner-maven
  namespace: gitlab-runner
data:
  settings.xml: "此处是maven的settings.xml的内容"

创建serviceaccount.yml 实现权限，否则提示无法在namespace中创建pod

apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab
  namespace: gitlab-runner
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: gitlab
  namespace: gitlab-runner
rules:
- apiGroups: [""]
  resources: ["pods","pods/exec"]
  verbs: ["get","list","watch","create","update","patch","delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: gitlab
  namespace: gitlab-runner
subjects:
- kind: ServiceAccount
  name: gitlab
  apiGroup: ""
roleRef:
  kind: Role
  name: gitlab
  apiGroup: rbac.authorization.k8s.io

通过deployment.yml部署gitlab-runner

1. 注意写入serviceaccountname gitlab
2. 注意导入configmap，即gitlab-runner的/etc/gitlab-runner/config.toml
3. configmap通过[[runners.kubernetes.volumes.config_map]]，将maven的settings挂载到了maven:3-jdk-8容器
4. 注意config.toml中token和cicd/runner的token并不一样

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      serviceAccountName: gitlab
      containers:
      - args:
        - '--debug'
        - run
        image: $(registryIP)/gitlab-runner/gitlab-runner:latest
        imagePullPolicy: IfNotPresent
        name: gitlab-runner
        volumeMounts:
        - mountPath: /etc/gitlab-runner
          name: config
        - mountPath: /etc/ssl/certs
          name: cacerts
          readOnly: true
        resources:
          limits:
            cpu: "2"
            memory: "4Gi"
          requests:
            cpu: "2"
            memory: "4Gi"
      restartPolicy: Always
      volumes:
      - configMap:
          name: gitlab-runner
        name: config
      - hostPath:
          path: /usr/share/ca-certificates/mozilla
        name: cacerts

第二步：配置.gitlab-ci.yml

• 选择maven模板
• 在 MVAEN_CLI_OPTS 中加入"–settings /usr/share/maven/conf2/settings.xml"
• 简单示例：

variables:
  MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
  MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true --settings /usr/share/maven/conf2/settings.xml"

compile:
  stage: build
  script:
    - 'mvn $MAVEN_CLI_OPTS compile'
  image: $(registryIP)/gitlab-runner/maven:3-jdk-8
  artifacts:
    name: "jars"
    expire_in: 60 mins
    paths:
      - target/*.jar

deploy:
  stage: deploy
  script:
    - 'echo deploy_caas is not set yet'
  only:
    - develop
  image: $(registryIP)/gitlab-runner/maven:3-jdk-8
  services:
  - name: $(registryIP)/gitlab-runner/mysql
    alias: mysql
  - name: $(registryIP)/gitlab-runner/redis
    alias: redis
  dependencies:
    - compile

第三步：测试

打开runner配置地址，可以看到刚才创建的runner，以及连接状态，修改为无需指定tag。

• 如果你的commit信息中包含 [ci skip] 或者 [skip ci] ，不论大小写，将会跳过jobs。
• gitlab -> 项目（组）xxxxx -> CI/CD -> pipelines
  • 直接create pipeline就可以了
• 提示"exceeded quota"，需要在config.toml中设定limit、request等参数
• 提示"pod limits=0"，serviceaccount没设置好

第四步：Auto-Devops

• 修改gitlab-CI模板
  • 找到模板目录，进行相应的修改。
  • 建议修改所有的模板，并在通用模板中提示用户改变模板。不同gitlab版本，地址略有差异：
  • vi /opt/gitlab/embedded/service/gitlab-rails/vendor/gitlab-ci-yml/Auto-DevOps.gitlab-ci.yml。
• vi /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml