kubernetes gitlab

kubernetes configMap

1. PVC 用于每次启动job 会自动挂在PVC，pvc可以加速打包，用户maven的缓存文件，也可以用作编译后的包存放路径。挂在路径为maven的缓存路径
2. host_path 主要用于kubernetes里面的job跑docker命令
3. 节点亲和力用于那些kubernetes node节点运行runner

apiVersion: v1
kind: ConfigMap
metadata:
  name: gitlab-runner
  namespace: gitlab
data:
  config.toml: |
    concurrent = 1
    check_interval = 0

    [session_server]
      session_timeout = 1800

    [[runners]]
      name = "gitlab.handpay.com.cn"
      url = "http://gitlab.handpay.com.cn/"
      token = "*******"
      executor = "kubernetes"
      [runners.cache]
        [runners.cache.s3]
        [runners.cache.gcs]
      [runners.kubernetes]
        host = "https://10.148.181.221:8443"
        cert_file = "/etc/ssl/certs/admin.pem"
        key_file = "/etc/ssl/certs/admin-key.pem"
        ca_file = "/etc/ssl/certs/ca.pem"
        namespace = "gitlab"
        bearer_token_overwrite_allowed = true
        privileged = true
        disable_cache = true
        cpu_limit = "1"
        memory_limit = "1Gi"
        service_cpu_limit = "1"
        service_memory_limit = "1Gi"
        helper_cpu_limit = "500m"
        helper_memory_limit = "100Mi"
        poll_interval = 5
        poll_timeout = 3600
        [[runners.kubernetes.volumes.pvc]]
          name = "gitlab-cache"
          mount_path = "/root/.m2/repository"
        [[runners.kubernetes.volumes.host_path]]
          name = "docker-demo"
          mount_path = "/var/run/docker.sock"
          read_only = true
          host_path = "/var/run/docker.sock"
        [runners.kubernetes.node_selector]
          gitlab = "true"

runner

1. kubernetes的证书用于对接，gitlab 有全套的对接kubernetes，被强，按照官方文档值安装了runner

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gitlab-runner
  namespace: gitlab
spec:
  replicas: 1
  selector:
    matchLabels:
      name: gitlab-runner
  template:
    metadata:
      labels:
        name: gitlab-runner
    spec:
      containers:
      - args:
        - run
        image: gitlab/gitlab-runner:latest
        name: gitlab-runner
        volumeMounts:
        - mountPath: /etc/gitlab-runner
          name: config
        - mountPath: /etc/ssl/certs
          name: cacerts
          readOnly: true
      restartPolicy: Always
      volumes:
      - configMap:
          name: gitlab-runner
        name: config
      - hostPath:
          path: /etc/kubernetes/ssl
        name: cacerts

maven 编译缓存地址

gitlab-ci.yaml 指定maven缓存文件目录，全局变量，maven打包的时候会读取，也可以自行设置maven的setting文件

variables:
  MAVEN_OPTS: "-Djava.awt.headless=true -Dmaven.repo.local=/root/.m2/repository"

job 之间共享cache

1. 在build阶段，打包出来的包使用artifacts来进行cache，用于job之间访问cache
2. 使用docker命令镜像，来进行读取编译cache打包好的package，进行docker build和docker push

  artifacts:
    paths:
      - target/*.jar

运行结果，每次运行会在kubernetes生成一个job任务，

gitlab-ci.yaml

简单跑通了，后面发布那块比较简陋，一个开发一个测试环境用ns做隔离,后面deployment、svc、ing做成模板，svc和deployment进行拆分，

stages:
  - build
  - image
  - deploy


image: harbor.handpay.com.cn/handpay-srp/maven:3.6.0-jdk-8-hpa

variables:
   MAVEN_OPTS: "-Djava.awt.headless=true -Dmaven.repo.local=/root/.m2/repository"

before_script:
  - echo  harbor.handpay.com.cn/handpay-risk/$CI_PROJECT_NAME:$CI_COMMIT_SHA
   
after_script:
  - echo "After script section"

   
build-pkg:
  stage: build
  script:
    - mvn install
  artifacts:
    paths:
      - target/*.jar
   
build-image:
  stage: image
  image: docker:latest
  script: 
    - sh docker.sh
    - docker build -t harbor.handpay.com.cn/handpay-risk/$CI_PROJECT_NAME:$CI_COMMIT_SHA .

push-image:
  stage: image
  image: docker:latest
  script: 
    # export DOCKER_HOST="tcp://localhost:2375"
    - docker login  -u $HARBOR_USER -p $HARBOR_PASS harbor.handpay.com.cn
    - docker push harbor.handpay.com.cn/handpay-risk/$CI_PROJECT_NAME:$CI_COMMIT_SHA
   
deploy:
  stage: deploy
  image: harbor.handpay.com.cn/handpay-ops/kubelet:1.12.4
  script:
    - echo harbor.handpay.com.cn/handpay-risk/$CI_PROJECT_NAME:$CI_COMMIT_SHA
    - kubectl apply -f deploy/service.yaml -n $NAMESPACES
    - kubectl apply -f deploy/deployment.yaml -n $NAMESPACES
    - kubectl apply -f  deploy/ingress.yaml -n $NAMESPACES
    - kubectl set image deployment/handpay-dashboard-sxt handpay-dashboard-sxt=harbor.handpay.com.cn/handpay-risk/$CI_PROJECT_NAME:$CI_COMMIT_SHA  -n $NAMESPACES
    - kubectl get pods,svc -n $NAMESPACES