Helm一键部署gitlab runner

本文需要掌握的技术

基础

• k8s
• helm charts
• gitlab runner

可选

• kubeseal
• sealed-secret
• S3

官方charts部署runner

添加repo

#国外
helm repo add gitlab https://charts.gitlab.io/

#国内
helm repo add gitlab-jh https://charts.gitlab.cn/

#查看helm库
helm repo list

确保能通过kubeconfig访问K8S集群

kubectl get pod -A

部署gitlab runner

#列出可用版本
helm search repo -l runner

#部署runner
helm install k8s-runner gitlab/gitlab-runner -n k8s-gitlab-runner --create-namespace

自定义chart部署runner

引用官方的charts，做自定义修改

初始化helm chart文件夹

#生成一个空chart
helm create chart-test

修改chart内容

apiVersion: v2
name: gitlab-runner
description: A Helm chart for Kubernetes

# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"

#对应runner的官方版本和名称以及仓库名
dependencies:
  - name: gitlab-runner
    repository: https://charts.gitlab.io/
    version: 0.40.1

自定义templates

telmplates中的文件如果和官方charts里面对应目录文件名重合，那么会覆盖官方的，以自定义文件为准；
templates中可以新增一些部署，比如部署sealed-secret或者是configmap

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: {{ .Release.Name }}-gitlab-runner
  namespace: {{ default .Release.Namespace .Values.namespace | quote }}
  labels:
    app: runner
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/weight": "1"
spec:
  encryptedData:
    runner-registration-token: {{ default "" .Values.runnerSealedRegistrationToken | quote }}
    runner-token: {{ default "" .Values.runnersSealedToken | quote }}
  template:
    data: null
    metadata:
      name: {{ .Release.Name }}-gitlab-runner
      namespace: {{ default .Release.Namespace .Values.namespace | quote }}
    type: Opaque

配置Values

这里runnerSealedRegistrationToken和runnersSealedToken为kubeseal加密的字符串
对应的镜像换成私有仓库的地址会让pod启动更快：image和helper_image

runnerSealedRegistrationToken: "wefwefewfwefwef"
runnersSealedToken: "wefwefwefwefwef"
namespace: veevaorion-runner-large
gitlab-runner:
  image: "repo/proxy/gitlab-runner:alpine-v14.10.1"
  imagePullPolicy: IfNotPresent
  unregisterRunners: true
  terminationGracePeriodSeconds: 3600
  concurrent: 10
  checkInterval: 30
  logLevel: debug
  rbac:
    create: true
    rules:
    - resources: ["configmaps", "pods", "pods/attach", "secrets", "services"]
      apiGroups: [""]
      verbs: ["get", "list", "watch", "create", "patch", "delete", "update"]
    - apiGroups: [""]
      resources: ["pods/exec"]
      verbs: ["create", "patch", "delete"]
    clusterWideAccess: false
    podSecurityPolicy:
      enabled: false
      resourceNames:
      - gitlab-runner
  replicas: 1
  gitlabUrl: https://gitlab.test.com/
  nodeSelector:
    usage: "runner-large"
  runners:
    tags: "veevaorion-group-large"
    config: |
      [[runners]]
        [runners.kubernetes]
          namespace = "{{.Release.Namespace}}"
          image = "ubuntu:16.04"
          pull_policy = ["if-not-present"]
          helper_image = "repo/proxy/gitlab-runner-helper:x86_64-f761588f"
          [runners.kubernetes.node_selector]
            usage = "runner-large"
    cache: 
      cacheType: s3
      cachePath: "runner-cache"
      cacheShared: true
      s3ServerAddress: s3.amazonaws.com
      s3BucketName: test-runner
      s3BucketLocation: cn-northwest-1 
      s3CacheInsecure: false

参考链接

gitlab

github

AWS