Kubernetes----helm部署gitlab-runner至k8s集群

一、Helm安装使用

请参考我的另一篇博客：k8s安装helm包管理器

二、gitlab官方部署方案

1、配置values.yaml文件

各个字段配置说明

image: #指定gitlab-runner镜像
imagePullPolicy: #镜像拉取策略
gitlabUrl: #gitlab地址
runnerRegistrationToken: #gitlab-runner注册用到的tocken
concurrent: #设置同行运行的runner个数
checkInterval: #定义检查gitlab新构建的频率
rbac: #角色设置
  create: true
  clusterWideAccess: true
metrics: #prometheus metrics数据暴露
  enabled: true
runners: #runners配置
  image:
  imagePullSecrets:
  imagePullPolicy:
  locked: #是否设置为特定的runner
  tags: #设置标签
  privileged: true
  secret: 
  namespace: 
  cache: {}
  builds: #构建资源限制
    cpuLimit: 200m
    memoryLimit: 256Mi
    cpuRequests: 100m
    memoryRequests: 128Mi
  services: {}
  helpers: {}
  serviceAccountName:
  nodeSelector: #worker调度选择器
resources: {} #资源限制
affinity: {} #节点亲和性
nodeSelector: {} #节点调度选择器
tolerations: [] #污点容忍度
envVars: #环境变量设置
  - name: RUNNER_EXECUTOR
    value: kubernetes

2、添加chart源

helm repo add gitlab https://charts.gitlab.io

3、部署gitlab-runner

安装命令如下：

helm install --namespace <NAMESPACE> --name gitlab-runner -f <CONFIG_VALUES_FILE> gitlab/gitlab-runner

更新命令如下：

helm upgrade --namespace <NAMESPACE> -f <CONFIG_VALUES_FILE> <RELEASE-NAME> gitlab/gitlab-runner

删除命令如下：

helm delete --namespace <NAMESPACE> <RELEASE-NAME>

三、阿里云k8s集群部署方案

1、下载GitLab Runner的Helm Chart

git clone https://github.com/haoshuwei/ack-gitlab-runner.git

目录结构如下

├── Chart.yaml
├── README.md
├── templates
│   ├── _cache_s3.tpl
│   ├── configmap.yaml
│   ├── deployment.yaml
│   ├── _env_vars.tpl
│   ├── _helpers.tpl
│   ├── NOTES.txt
│   ├── pvc.yaml
│   ├── role-binding.yaml
│   ├── role.yaml
│   ├── secrets.yaml
│   └── service-account.yaml
└── values.yaml

2、templates目录下的文件按需分配，values.yaml示例如下：

image: gitlab/gitlab-runner:alpine-v12.1.0
imagePullPolicy: IfNotPresent
init:
  image: busybox
  tag: latest
gitlabUrl: "https://example.gitlab.com/"
runnerRegistrationToken: "V3b-Q9LpSfzMGkMNqZQx"
unregisterRunners: true
concurrent: 8
checkInterval: 10
rbac:
  create: true
  clusterWideAccess: false
metrics:
  enabled: true
runners:
  image: ubuntu:16.04
  tags: "k8s"
  privileged: true
  namespace: gitlab
  cachePath: "/opt/cache"
  cache: {}
  builds: #资源限制
    cpuLimit: 3000m
    memoryLimit: 4096Mi
    cpuRequests: 100m
    memoryRequests: 512Mi
  services: {}
  helpers: {}
  nodeSelector: #worker调度节点选择器
    runner: "gitlab-runner-only" 
resources: {}
nodeSelector: #gitlab-runner节点选择器           
  runner: "gitlab-runner-only"
tolerations: #污点容忍配置
- key: "runner"
  operator: "Exists"

3、打包发布

helm package .
helm install --namespace gitlab --name gitlab-runner *.tgz

参考文章：https://www.alibabacloud.com/help/zh/doc-detail/106968.htm

三、遇到的问题以及解决方案

1、命名空间指定

helm install --namespace gitlab --name gitlab-runner -f values.yaml gitlab/gitlab-runner，使用其他命名空间会有如下错误

2、docker守护进程的错误

错误如下：

解决方案：修改configmap配置如下参数

    cat >>/home/gitlab-runner/.gitlab-runner/config.toml <<EOF
            [[runners.kubernetes.volumes.host_path]]
              name = "docker"
              mount_path = "/var/run/docker.sock"
              read_only = false
              host_path = "/var/run/docker.sock"
    EOF

3、gitlab-runner和worker调度到指定节点

添加标签选择器，添加节点标签，修改values.yaml
kubectl label node node1 runner=gitlab-runner-only

runners:
.....
  nodeSelector:
    runner: gitlab-runner-only
nodeSelector:
  runner: gitlab-runner-only
....

4、指定节点只运行gitlab-runner和worker

添加污点配置，kubectl taint node node1 runner=gitlab-runner-only:NoSchedule，并修改configmap配置:

    cat >>/home/gitlab-runner/.gitlab-runner/config.toml <<EOF
        [runners.kubernetes.node_tolerations]
          "runner=gitlab-runner-only" = "NoSchedule"
    EOF

进入gitlab-runner容器查看/home/gitlab-runner/.gitlab-runner/config.toml文件格式是否正确

[[runners]]
.....
  [runners.kubernetes]
.....
    [runners.kubernetes.node_selector]
      runner = "gitlab-runner-only"
    [runners.kubernetes.node_tolerations]
      "runner=gitlab-runner-only" = "NoSchedule"

参考资料

gitlab-runner install on kubernetes

使用GitLab CI在Kubernetes服务上运行GitLab Runner并执行Pipeline