云原生下的企业级前端构建实践

前言

容器、k8s、云原生现在越来越流行，在云原生时代下的前端项目构建发布有什么变化呢？相比传统的构建部署平台，云原生有哪些区别？如果对传统构建部署平台下的企业级项目构建感兴趣，可以先看看[统一部署平台下的企业级前端构建实践]。

今天这里主要讲的是基于开源轮子的云原生构建方案（docker/k8s/rancher/gitlab ci）

完整流程

完整流程如下图所示：

1、代码托管在gitlab，借用gitlab 的ci cd；

2、gitlab的ci主要负责工程自身的build打包，以及对docker镜像的打包build-docker；

3、将打包好的docker镜像推到私有源harbor中；

4、区分环境，调用rancher的http接口，将指定集群的指定项目，下载harbor上的docker镜像进行部署。rancher也是通过kubectl来控制k8s集群

    ┌──────────────┐               ┌───────────────┐
    │              │               │               │
    │    harbor    ├──────────────▲│     rancher   │
    │              │▼──────────────┤               ├──────────────┐
    │              │  docker pull  │               │              │
    └──────────────┘               └───────────────┘       kubectl│
           ▲                               ▲                      │
           │                               │                      │
           │                               │                      │
           │                               │                      │
           │build-docker                   deploy-test1           │
           │                               deploy-release         │
           │                               ......                 │
           │                               │                      │
     build │                               │                      │                         
           │                               │                      │
┌──────────┴────────────┐                  │         ┌────────────┴─────────────┐
│                       │                  │         │                          │
│                       │                  │         │                          │
│                       │                  │         │                          │
│        gitlab         ├──────────────────┘         │       k8s cluster        │
│                       │                            │                          │
│                       │                            │                          │
│                       │                            │                          │
└───────────────────────┘                            └──────────────────────────┘

CI、CD完整流程

我以某一个工程的gitlab ci yml文件为例进行细节介绍。

1、首先是定义了需要的变量，包括了rancher的、harbor的；

2、执行前置脚本before_script，通过分支来决定环境变量以及打包镜像的名字，普通分支的镜像名是不会带上hash的，因为基本都是测试环境不需要回滚。环境变量为RUNTIME_ENV，会通过后面的docker build 带到容器内。

3、定义整个阶段分为build、build-docker、deploy-test1、deploy-online。分别进行工程打包、镜像打包、部署测试环境和部署线上环境。

4、定义每个阶段执行的job，例如deploy就是调用rancher的接口，传递不同的变量。

image: docker:18.06
 
 
variables:
  # rancher 变量
  RANCHER_NAMESPACE: kef2e
  RANCHER_PROJECT: kef2e
  RANCHER_WORKLOAD: ke-cms
  # harbor 变量
  PROJECT: kef2e
  IMAGE_NAME: ke-cms
  # 部署重要变量
  RANCHER_URL: https://someurl
  RANCHER_URL_ONLINE: https://someurl
  RANCHER_TOKEN: yourtoken
  RANCHER_TOKEN_ONLINE: yourtoken
  RANCHER_PROJECT_ID: yourid
  RANCHER_PROJECT_ID_ONLINE: yourid

 
services:
  - name: docker:18.06-dind
    alias: docker
 
stages:
  - build-spa
  - build-docker
  - deploy
  - deploy_online
 

  before_script:
    - echo exec before_script
    # 现在gitlab版本没有 CI_COMMIT_SHORT_SHA 变量(v>11.7)，进行兼容
    - if [ "$CI_COMMIT_SHORT_SHA" = "" ]; then CI_COMMIT_SHORT_SHA=${CI_COMMIT_SHA:0:8} ;fi;
    # 将branch name里的/转换为tag name支持的-
    - CI_COMMIT_REF_SAFE_NAME=$(echo $CI_COMMIT_REF_NAME | sed 's/\//-/')
    # 镜像的仓库地址
    - IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" != "release" ]]; then RUNTIME_ENV=development ; fi
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" == "release" ]] || [[ "$CI_COMMIT_REF_SAFE_NAME" == "pre" ]]; then IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME-$CI_COMMIT_SHORT_SHA; fi
    - echo image url:$IMAGE_URL
    - echo image RUNTIME_ENV:$RUNTIME_ENV

.deploy_tpl: &deploy_def
  script:
    - CONTAINER_URL=$RANCHER_URL/v3/projects/$RANCHER_PROJECT_ID/workloads/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD
    - curl -u $RANCHER_TOKEN -X GET $CONTAINER_URL -o deployment.json -k
    - TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
    - sed -i 's#image\":[^,]*,#image\":\"'$IMAGE_URL'\",#' deployment.json
    - sed -i 's/"cattle.io\/timestamp":"[^"]*"/"cattle.io\/timestamp":"'${TIMESTAMP}'"/g' deployment.json
    - curl -u $RANCHER_TOKEN -XPUT -H "Accept:application/json" -H "Content-Type:application/json" -[email protected] $CONTAINER_URL -k
    - echo RANCHER_URL:$RANCHER_URL
    - echo RANCHER_TOKEN:$RANCHER_TOKEN
    - echo RANCHER_PROJECT_ID:$RANCHER_PROJECT_ID
    - echo CONTAINER_URL:$CONTAINER_URL


# 打包spa
build-spa:
  stage: build-spa
  tags:
    - k8s
  image: harbor-registry.inner.youdao.com/ke-test/agent-base:node12
  script:
    - echo "package build directory"
    - yarn config set strict-ssl false
    - yarn config set registry http://f2enpm.inner.youdao.com/
    # https://github.com/cnpm/cnpmjs.org/issues/1246， puppeteer在CI可能install有问题
    - yarn config set puppeteer_download_host https://nexus3.corp.youdao.com/repository/npm-all/
    - yarn config set puppeteer_skip_chromium_download true
    - yarn install
    # 这里为项目线上的构建命令，根据项目自行修改
    - yarn release
  cache:
    key: "$CI_COMMIT_REF_SLUG"
    paths:
      - node_modules/
  artifacts:
    name: "$CI_COMMIT_REF_SLUG"
    paths:
      - dist
    when: always
    expire_in: 6 mos
 
# 构建镜像
build-docker:
  retry: 2
  stage: build-docker
  tags:
    - k8s
  script:
    # 登陆 dockerhub
    - echo $DOCKER_REG_PASSWD | docker login $DOCKER_REG -u $DOCKER_REG_USER --password-stdin
    # 镜像地址
    - echo $IMAGE_URL
    - docker build -t $IMAGE_URL --build-arg RUNTIME_ENV=$RUNTIME_ENV .
    - docker push $IMAGE_URL
    - docker logout $DOCKER_REG
 
 
# 部署rancher
deploy_rancher:
  stage: deploy
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  when: manual
  
# 部署rancher online
deploy_rancher_online:
  stage: deploy_online
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    RANCHER_URL: $RANCHER_URL_ONLINE
    RANCHER_PROJECT_ID: $RANCHER_PROJECT_ID_ONLINE
    RANCHER_TOKEN: $RANCHER_TOKEN_ONLINE
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  only:
    - release
  when: manual
  

通过上面的流程，就可以完成一个前端项目的构建部署了，但是也存在一些问题。企业级的含义就是追求至高的可维护性和尽可能的降低个性化。而如果我们多达二十几个工程，都需要这样一个个的配置yml，岂不是乱套了？所以抽象统一配置势在必行。

统一配置后的CI、CD

通过分析可以得出，除了build阶段是有可能不一致的，其他阶段都是相同的，只不过是变量名如rancher的项目id等不一样而已。

通过统一配置后的yml文件如下：

除了build阶段，其他阶段和before_script均已抽出到远程去管理，通过include来加载。

如果极端能够保证build阶段做的事情一致，甚至可以全部抽出只留下变量名配置。由于我们工程数量极多，个性化改造性价比不高，所以build暂时保存。

image: docker:18.06
 
 
variables:
  # rancher 变量
  RANCHER_NAMESPACE: kef2e
  RANCHER_PROJECT: kef2e
  RANCHER_WORKLOAD: ke-cms
  # harbor 变量
  PROJECT: kef2e
  IMAGE_NAME: ke-cms
  # 部署重要变量
  RANCHER_URL: https://someurl
  RANCHER_URL_ONLINE: https://someurl
  RANCHER_TOKEN: yourtoken
  RANCHER_TOKEN_ONLINE: yourtoken
  RANCHER_PROJECT_ID: yourid
  RANCHER_PROJECT_ID_ONLINE: yourid

 
services:
  - name: docker:18.06-dind
    alias: docker
 
stages:
  - build
  - build-docker
  - deploy-test1
  - deploy-online

include: 
    - 'https://g.hz.netease.com/api/v4/projects/49427/repository/files/before_script.yml/raw?ref=master&private_token=YOUR_TOKEN&ext=.yml'
    - 'https://g.hz.netease.com/api/v4/projects/49427/repository/files/job.yml/raw?ref=master&private_token=YOUR_TOKEN&ext=.yml'

# 打包spa
build-spa:
  stage: build
  tags:
    - k8s
  image: harbor-registry.inner.youdao.com/ke-test/agent-base:node12
  script:
    - echo "package build directory"
    - yarn config set strict-ssl false
    - yarn config set registry http://f2enpm.inner.youdao.com/
    # https://github.com/cnpm/cnpmjs.org/issues/1246， puppeteer在CI可能install有问题
    - yarn config set puppeteer_download_host https://nexus3.corp.youdao.com/repository/npm-all/
    - yarn config set puppeteer_skip_chromium_download true
    - yarn install
    # 这里为项目线上的构建命令，根据项目自行修改
    - yarn release
  cache:
    key: "$CI_COMMIT_REF_SLUG"
    paths:
      - node_modules/
  artifacts:
    name: "$CI_COMMIT_REF_SLUG"
    paths:
      - dist
    when: always
    expire_in: 6 mos
 

  

抽象出来的两个远程脚本如下：

before_script.yml用来做一些前置操作，比如根据条件定义镜像名：

default:
  before_script:
    - echo exec before_script
    # 现在gitlab版本没有 CI_COMMIT_SHORT_SHA 变量(v>11.7)，进行兼容
    - if [ "$CI_COMMIT_SHORT_SHA" = "" ]; then CI_COMMIT_SHORT_SHA=${CI_COMMIT_SHA:0:8} ;fi;
    # 将branch name里的/转换为tag name支持的-
    - CI_COMMIT_REF_SAFE_NAME=$(echo $CI_COMMIT_REF_NAME | sed 's/\//-/')
    # 镜像的仓库地址
    - IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" != "release" ]]; then RUNTIME_ENV=development ; fi
    - if [[ "$CI_COMMIT_REF_SAFE_NAME" == "release" ]] || [[ "$CI_COMMIT_REF_SAFE_NAME" == "pre" ]]; then IMAGE_URL=$DOCKER_REG/$PROJECT/$IMAGE_NAME:$CI_COMMIT_REF_SAFE_NAME-$CI_COMMIT_SHORT_SHA; fi
    - echo image url:$IMAGE_URL
    - echo image RUNTIME_ENV:$RUNTIME_ENV

而job.yml则用来存放一些公共的任务：

.deploy_tpl: &deploy_def
  script:
    - CONTAINER_URL=$RANCHER_URL/v3/projects/$RANCHER_PROJECT_ID/workloads/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD
    - curl -u $RANCHER_TOKEN -X GET $CONTAINER_URL -o deployment.json -k
    - TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
    - sed -i 's#image\":[^,]*,#image\":\"'$IMAGE_URL'\",#' deployment.json
    - sed -i 's/"cattle.io\/timestamp":"[^"]*"/"cattle.io\/timestamp":"'${TIMESTAMP}'"/g' deployment.json
    - curl -u $RANCHER_TOKEN -XPUT -H "Accept:application/json" -H "Content-Type:application/json" -d@deployment.json $CONTAINER_URL -k
    - echo RANCHER_URL:$RANCHER_URL
    - echo RANCHER_TOKEN:$RANCHER_TOKEN
    - echo RANCHER_PROJECT_ID:$RANCHER_PROJECT_ID
    - echo CONTAINER_URL:$CONTAINER_URL

variables: 
  RANCHER_URL: 
  RANCHER_URL_ONLINE: 
  RANCHER_TOKEN: 
  RANCHER_TOKEN_ONLINE: 

# 构建镜像
build-docker:
  retry: 2
  stage: build-docker
  tags:
    - k8s
  script:
    # 登陆 dockerhub
    - echo $DOCKER_REG_PASSWD | docker login $DOCKER_REG -u $DOCKER_REG_USER --password-stdin
    # 镜像地址
    - echo $IMAGE_URL
    - docker build -t $IMAGE_URL --build-arg RUNTIME_ENV=$RUNTIME_ENV .
    - docker push $IMAGE_URL
    - docker logout $DOCKER_REG
 
 

# 部署rancher
deploy_rancher_test1:
  stage: deploy-test1
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  when: manual

  # 部署rancher online
deploy_rancher_online:
  stage: deploy-online
  image: $CURL_IMAGE
  tags:
    - k8s
  variables:
    RANCHER_URL: $RANCHER_URL_ONLINE
    RANCHER_PROJECT_ID: $RANCHER_PROJECT_ID_ONLINE
    RANCHER_TOKEN: $RANCHER_TOKEN_ONLINE
    MSG: "正式部署到 $RANCHER_CLUSTER:$RANCHER_NAMESPACE \n部署地址：$RANCHER_URL/p/$RANCHER_PROJECT_ID/workload/deployment:$RANCHER_NAMESPACE:$RANCHER_WORKLOAD-$CI_COMMIT_REF_NAME \n"
  script:
  <<: *deploy_def
  # 手动触发rancher部署
  only:
    - release
  when: manual
  

总结

在完成了上面的统一化配置后，每个工程在创建ci过程时，只需要配置自己工程的独特参数，和自己的build阶段任务，即可完成前端构建发布。

未来新增环境及集群时，只需要在同一配置的脚本里维护，工程只需要加一个stage和对应的变量即可。