如何快速搭建前端技术团队基础设施

简介

随着前端领域的快速发展，横向上，跨PC、移动端甚至嵌入式设备多平台，而且脱离浏览器走向混合桌面应用和移动应用，
纵向上，Nodejs逐渐渗透服务端，服务端渲染、微服务、业务应用开发，各种前端框架此起彼伏，各种理念、模式大行其道，
前端开发早已摆脱学学HTML、CSS、JavaScript写几个网页界面的简单场景，开发环境、流程、模式、架构设计、工程化各层面日趋复杂，这些都离不开一套基础设施的支撑，本文将结合通用研发流程和相关开源工具实践如何快速搭建前端技术团队基础设施。

本文源于一次内部分享，刚开始只是为了学习持续集成 Jenkins 软件的使用及在前端领域的应用，
随着其他开源工具的了解学习，发现这是一项系统工程，所以汇总很多开源工具的知识，总结了这篇文章。
文中总结的这些开源工具基本涵盖最简化前端项目研发流程，能够快速实施项目研发，根据项目选用的技术栈，
工具可以扩展很多，要想深入掌握，必须深入实践，希望借由这篇文章，为后续深入实践建立开端。

项目流程

一般技术项目实施主要包括以下几个大方面：

需求及项目管理
技术研发
- 版本管理
- 代码质量审查
- 测试
- 持续集成
- 持续部署
- 性能监控

而对于前端来说，因为主要涉及应用GUI界面层开发，更为细致划分还会涉及: 视觉还原、交互还原，
就测试而言，包括：功能测试、代码单元测试、UI自动化、兼容性等等。就具体技术栈、项目和业务而言，可能会涉及多项整合，简言之，因人而已，后续具体实践再表。本文主要涉及上述大方面流程梳理总结相关开源工具运用，很多工具其实属于通用项目开发，已经在传统服务端研发领域成熟应用，网上各种教程也很多，扩展了部分前端工具，同时收集很多其他工具实践链接地址，也可以根据自己当前项目技术栈进行实践。

FE-infrastructure - 前端技术团队基础设施

参考

http://tech.adroll.com/blog/frontend/2017/08/29/how-to-run-a-front-end-infrastructure-team.html
https://segmentfault.com/a/1190000014720175?utm_source=tag-newest

搭建工具

ansible

ansible是一款基于Python的自动化运维工具，可以通过内置各种模块快速部署各种应用，核心包括：inventory配置主机、playbook 任务剧本，主要使用yaml语法编写；

参考
https://www.ansible.com/
https://galaxy.ansible.com/
https://docs.ansible.com/ansible/latest/search.html?q=docker&check_keywords=yes&area=default
https://books.google.com/books?id=lNp6DwAAQBAJ&printsec=frontcover&dq=Ansible%E5%BE%B9%E5%BA%95%E5%85%A5%E9%96%80%EF%BD%9C%E9%9B%B2%E7%AB%AF%E6%99%82%E4%BB%A3%E7%9A%84%E7%B5%84%E6%85%8B%E7%AE%A1%E7%90%86&hl=zh-CN&sa=X&ved=0ahUKEwiI9I-Kq5TfAhXdFzQIHZ_3DHsQ6AEIKDAA#v=onepage&q=Ansible%E5%BE%B9%E5%BA%95%E5%85%A5%E9%96%80%EF%BD%9C%E9%9B%B2%E7%AB%AF%E6%99%82%E4%BB%A3%E7%9A%84%E7%B5%84%E6%85%8B%E7%AE%A1%E7%90%86&f=false

docker

Docker 是一个开源的应用容器引擎，让开发者可以打包他们的应用以及依赖包到一个可移植的容器中，然后发布到任何流行的 Linux 机器上，也可以实现虚拟化。容器是完全使用沙箱机制，相互之间不会有任何接口。 - 百度百科
用来承载各种开源工具应用，主要概念包括: image镜像、container运行容器。

参考
https://docs.docker.com/
http://www.runoob.com/docker/docker-tutorial.html
https://github.com/hangyan/docker-resources

搭建方式
- 利用 ansible 的 docker-image/docker-container 模块快速创建并管理容器；
- 利用 ansible 自带 ansible-galaxy模块下载对应的 roles 实现快速部署；
- 手动编写 shell 脚本进行分发部署；

这里主要使用第一种，获取官方镜像文件，直接通过内置容器模块管理应用部署。

以前端为例搭建基础设施

前端基础设施应用Docker容器分布

项目及需求管理 PM/RM

Redmine是用Ruby开发的基于web的项目管理软件，是用ROR框架开发的一套跨平台项目管理系统，据说是源于Basecamp的ror版而来，支持多种数据库，有不少自己独特的功能，例如提供wiki、新闻台等，还可以集成其他版本管理系统和BUG跟踪系统，例如Perforce、SVN、CVS、TD等等。这种 Web 形式的项目管理系统通过“项目（Project）”的形式把成员、任务（问题）、文档、讨论以及各种形式的资源组织在一起，大家参与更新任务、文档等内容来推动项目的进度，同时系统利用时间线索和各种动态的报表形式来自动给成员汇报项目进度。

# pm server
- hosts: pm_server
  vars_files:
    - ../vars.yml
  tasks:
    - name: pull redmine image
      docker_image:
        state: present
        name: "redmine:{{ redmine_version }}"
    - name: start pm server
      docker_container:
        name: "{{ pm_container_name }}"
        image: "redmine:{{ redmine_version }}"
        state: started
        ports:
          - "{{ pm_ports }}"

redmine项目管理

参考

http://www.redmine.org/
https://github.com/docker-library/redmine
https://hub.docker.com/_/redmine/
http://www.redmine.org.cn/manual

版本控制 VCS

版本控制系统（version control system）,是一种记录一个或若干文件内容变化，以便将来查阅特定版本修订情况的系统。版本控制系统不仅可以应用于软件源代码的文本文件，而且可以对任何类型的文件进行版本控制。

svn server
gitlab

- hosts: vcs_server
  become: yes
  vars_files:
    - ../vars.yml
  tasks:
    - name: pull gitlab image
      docker_image:
        state: present
        name: "gitlab/gitlab-ce:{{ gitlab_version }}"
    - name: start vcs server
      docker_container:
        name: "{{ vcs_container_name }}"
        image: "gitlab/gitlab-ce:{{ gitlab_version }}"
        state: started
        ports:
          - "{{ vcs_ports }}"
        volumes:
          - "{{ docker_data_dir }}/gitlab:/var/opt/gitlab:rw"
    - wait_for: path={{ docker_data_dir }}/gitlab/gitlab-rails/etc/gitlab.yml
    - name: copy config file
      copy:
        src: "{{ home_dir }}/backup/gitlab/gitlab_etc.yml"
        dest: "{{ docker_data_dir }}/gitlab/gitlab-rails/etc/gitlab.yml"
    - name: restart server
      shell: "docker exec {{ vcs_container_name }} gitlab-ctl restart"

gitlab版本控制

参考

https://hub.docker.com/r/garethflowers/svn-server/
https://docs.gitlab.com/ee/install/docker.html
https://docs.gitlab.com/runner/install/

私有包管理 CNPM

NPM的全称是Node Package Manager [1] ，是一个NodeJS包管理和分发工具，已经成为了非官方的发布Node模块（包）的标准。

# repo server
- hosts: repo_server
  become: yes
  vars_files:
    - ../vars.yml
  tasks:
    - name: clone cnpm repo
      git:
        repo: "{{ repo_url }}"
        dest: "{{ repo_path }}"
        accept_hostkey: yes
    - docker_service:
        project_src: "{{ repo_path }}"
        state: present

cnpm私有包管理

参考

https://github.com/cnpm/cnpmjs.org
https://github.com/verdaccio/verdaccio
https://github.com/rlidwka/sinopia

代码质量审查

SONAR QUBE

SONAR QUBE是代码质量管理平台，可以关联代码仓库，进行代码扫描，快速定位代码中潜在或明显错误，支持扫描多种语言，同时可以结合CI加入持续集成流水线。

# inspect server
- hosts: inspect_server
  vars_files:
    - ../vars.yml
  tasks:
    - name: pull sonar image
      docker_image:
        state: present
        name: "sonarqube:{{ sonar_version }}"
    - name: start inspect server
      docker_container:
        name: "{{ inspect_container_name }}"
        image: "sonarqube:{{ sonar_version }}"
        state: started
        ports:
          - "{{ inspect_ports1 }}"
          - "{{ inspect_ports2 }}"

sonar代码质量审查

参考

https://www.sonarqube.org/
https://www.cnblogs.com/qiumingcheng/p/7253917.html
https://blog.csdn.net/czy3y/article/details/81233543
https://lyj86.iteye.com/blog/2427381
https://blog.csdn.net/rainflood/article/details/77619693
https://alternativeto.net/software/sonarqube/

持续集成 CI

持续集成CI是一种软件开发实践，即团队开发成员经常集成他们的工作，通常每个成员每天至少集成一次，也就意味着每天可能会发生多次集成。每次集成都通过自动化的构建（包括编译，发布，自动化测试）来验证，从而尽早地发现集成错误。

jenkins
gitlab-runner

- hosts: ci_server
  become: yes
  vars_files:
    - ../vars.yml
  tasks:
    - name: pull ci image
      docker_image:
        state: present
        name: "jenkins/jenkins:{{ jenkins_version }}"
        debug: yes
    - name: start ci server
      docker_container:
        name: "{{ ci_container_name }}"
        image: "jenkins/jenkins:{{ jenkins_version }}"
        debug: yes
        state: started
        restart: yes
        ports:
          - "{{ ci_ports }}"
        volumes:
          - "{{ docker_data_dir }}/jenkins:/var/jenkins_home:rw"

Jenkins持续集成

参考

https://jenkins.io/zh/doc/
https://www.cnblogs.com/ceshi2016/p/6529557.html
https://www.mindtheproduct.com/2016/02/what-the-hell-are-ci-cd-and-devops-a-cheatsheet-for-the-rest-of-us/
https://www.martinfowler.com/articles/continuousIntegration.html
http://itrevolution.com/book/the-phoenix-project/

持续部署 CD

持续部署（continuous deployment）是通过自动化的构建、测试和部署循环来快速交付高质量的产品。

ftp/scp
ansible/puppet 脚本
drone
gocd
基于开源二次开发
热更新、版本升级

参考

https://martinfowler.com/books/continuousDelivery.html
https://www.cnblogs.com/wchonge/p/8447139.html
https://www.gocd.org/download/
https://gocd.org.cn/
https://hub.docker.com/r/drone/drone/
http://docs.drone.io/
https://linux.cn/article-7236-1.html

性能监控 APM

APM = Application Performance Management，应用性能管理对企业系统即时监控以实现对应用程序性能管理和故障管理的系统化的解决方案。 - 百度百科
简言之实时监控前端应用各项性能指标，并进行持续完善。

Sitespeedio + Graphite + Grafana

Sitespeedio 页面性能检测工具，针对网页生成详细的页面性能检测数据，同时向 Graphite 传递性能数据
Graphite 一款流行指标数据采集系统，能够收集存储各种监控数据
Grafana 监控数据可视化分析与监控工具，支持多种数据源、异常指标告警、丰富精美图表界面
Grafana-bootstrap sitespeedio 官方提供定制化性能数据展示图表模板，也可以导入自定义展示图表

docker-compose 方式安装及使用

curl -O https://raw.githubusercontent.com/sitespeedio/sitespeed.io/master/docker/docker-compose.yml
docker-compose up -d
docker-compose run sitespeed.io https://www.baidu.com --graphite.host=graphite
docker-compose stop && docker-compose rm
docker volume rm performancedashboard_graphite performancedashboard_grafana

ansible 模块化安装

# perf server
- hosts: perf_server
  vars_files:
    - ../vars.yml
  vars:
    sitespeed_version: "latest"
    container_name: "perf_server"
    graphite_name: "graphite_server"
    grafana_name: "grafana_server"
    grafana_bootstrap_name: "grafana_bootstrap_server"
    grafana_host: 192.168.33.10
  tasks:
    - name: pull grafana image
      docker_image:
        state: present
        name: "grafana/grafana:latest"
    - name: pull grafana-bootstrap image
      docker_image:
        state: present
        name: "sitespeedio/grafana-bootstrap:latest"
    - name: pull graphite image
      docker_image:
        state: present
        name: "sitespeedio/graphite"
    - name: pull sitespeedio image
      docker_image:
        state: present
        name: "sitespeedio/sitespeed.io:{{ sitespeed_version }}"

    - name: copy backup file
      copy:
        src: "{{ home_dir }}/backup/graphite/"
        dest: "{{ docker_data_dir }}/graphite/"
    - name: start graphite server
      docker_container:
        name: "{{ graphite_name }}"
        image: "sitespeedio/graphite:latest"
        state: started
        ports:
          - "6000:80"
          - "6001:2003"
        restart: yes
        volumes:
          - "{{ docker_data_dir }}/graphite/whisper:/opt/graphite/storage/whisper:rw"
          - "{{ docker_data_dir }}/graphite/graphite.db:/opt/graphite/storage/graphite.db"
          - "{{ docker_data_dir }}/graphite/conf/storage-schemas.conf:/opt/graphite/conf/storage-schemas.conf"
          - "{{ docker_data_dir }}/graphite/conf/storage-aggregation.conf:/opt/graphite/conf/storage-aggregation.conf"
          - "{{ docker_data_dir }}/graphite/conf/carbon.conf:/opt/graphite/conf/carbon.conf"

    - name: start grafana server
      docker_container:
        name: "{{ grafana_name }}"
        image: "grafana/grafana:latest"
        state: started
        ports:
          - "6002:3000"
        links:
          - "{{ graphite_name }}"
        env: GF_SECURITY_ADMIN_PASSWORD=hdeAga76VG6ga7plZ1
          GF_SECURITY_ADMIN_USER=sitespeedio
          GF_AUTH_ANONYMOUS_ENABLED=true
          GF_USERS_ALLOW_SIGN_UP=true
          GF_USERS_ALLOW_ORG_CREATE=true
        volumes:
          - "{{ docker_data_dir }}/grafana:/var/lib/grafana:rw"

    - name: start grafana-bootstrap server
      docker_container:
        name: "{{ grafana_bootstrap_name }}"
        image: "sitespeedio/grafana-bootstrap:latest"
        state: started
        links:
          - "{{ grafana_name }}:grafana"
        networks:
          - name: "bridge"
            links:
              - "{{ grafana_name }}:grafana"
              - "{{ container_name }}"
        env: GF_API=http://{{ grafana_host }}:6002/api
          GF_PASSWORD=hdeAga76VG6ga7plZ1
          GF_USER=sitespeedio

    - name: start perf server
      docker_container:
        name: "{{ container_name }}"
        image: "sitespeedio/sitespeed.io:{{ sitespeed_version }}"
        state: started
        shm_size: 1g
        command: "https://www.baidu.com -b firefox -n 10 --graphite.host={{ grafana_host }} --graphite.port=6001 --graphite.annotationMessag --graphite.annotationTag --graphite.annotationScreenshot"
        volumes:
          - "{{ docker_data_dir }}/sitespeed-result/:/sitespeed.io/sitespeed-result:rw"

sitespeed页面性能监控

其他方案

https://www.w3.org/TR/navigation-timing-2/#processing-model
https://developer.mozilla.org/zh-CN/docs/Web/API/Performance
http://javascript.ruanyifeng.com/bom/performance.html
https://www.oneapm.com/bi/feature.html

参考

https://www.sitespeed.io/
http://graphiteapp.org/
https://grafana.com/
http://www.ywjt.org/index.php/archives/1802
https://hub.docker.com/r/sitespeedio/sitespeed.io/
https://hub.docker.com/r/sitespeedio/graphite/
https://raw.githubusercontent.com/sitespeedio/sitespeed.io/master/docker/docker-compose.yml
https://www.peterhedenskog.com/blog/2015/04/open-source-performance-dashboard/
https://www.slideshare.net/PhilippeDonon/intro-to-sitespeedio
https://qiita.com/Jxck_/items/abfa9f3dd15c5572cbfd
https://www.jianshu.com/p/ebf7f590839d
https://blog.csdn.net/u012326462/article/details/81038446
http://www.cnblogs.com/zhengyun_ustc/p/55solution9.html
http://www.perf-tooling.today
https://gtmetrix.com/
https://medium.com/@addyosmani/the-cost-of-javascript-in-2018-7d8950fbb5d4
https://insights.project-a.com/the-business-value-of-site-speed-and-how-to-analyze-it-step-by-step-b01658e126cb
https://developers.google.com/web/fundamentals/performance/critical-rendering-path/
https://cloud.tencent.com/developer/article/1005339

异常监控 Error Tracking

异常监控主要针对前端应用运行时发生错误或故障进行实时收集和告警、处置。

Sentry

Sentry 是一款开源异常跟踪系统，支持服务端、客户端等多平台、多语言错误跟踪。

支持 Vue、React、Angular、RN 等多种前端框架，同时支持上传 sourcemap 定位代码错误具体位置, 可以关联 gitlab 创建 issue 修复问题。

ansible-galaxy install stouts.sentry

- hosts: track_server
  vars_files:
    - ../vars.yml
  tasks:
    - name: pull sentry image
      docker_image:
        state: present
        name: "sentry:{{ sentry_version }}"

    - name: get SENTRY_SECRET_KEY
      command: docker run --rm sentry config generate-secret-key
      register: SENTRY_SECRET_KEY

    - name: upgrade sentry server
      docker_container:
        name: "track_upgrade_tmp"
        image: "sentry:{{ sentry_version }}"
        state: started
        interactive: no
        command: "upgrade"
        cleanup: yes
        auto_remove: yes
        links:
          - "{{ redis_container_name }}:redis"
          - "{{ postgres_container_name }}:postgres"
        env:
          SENTRY_DB_USER: "{{ docker_user }}"
          SENTRY_DB_PASSWORD: "{{ docker_pass }}"
          SENTRY_SECRET_KEY: "{{ SENTRY_SECRET_KEY.stdout }}"

    - name: create sentry superuser
      docker_container:
        name: "track_super_tmp"
        image: "sentry:{{ sentry_version }}"
        state: started
        interactive: yes
        command: "createuser --email [email protected] --password admin --superuser"
        auto_remove: yes
        links:
          - "{{ redis_container_name }}:redis"
          - "{{ postgres_container_name }}:postgres"
        env:
          SENTRY_DB_USER: "{{ docker_user }}"
          SENTRY_DB_PASSWORD: "{{ docker_pass }}"
          SENTRY_SECRET_KEY: "{{ SENTRY_SECRET_KEY.stdout }}"

    - name: start sentry server
      docker_container:
        name: "{{ track_container_name }}"
        image: "sentry:{{ sentry_version }}"
        state: started
        links:
          - "{{ redis_container_name }}:redis"
          - "{{ postgres_container_name }}:postgres"
        ports:
          - "{{ track_ports }}"
        env:
          SENTRY_DB_USER: "{{ docker_user }}"
          SENTRY_DB_PASSWORD: "{{ docker_pass }}"
          SENTRY_SECRET_KEY: "{{ SENTRY_SECRET_KEY.stdout }}"
        volumes:
          - "{{ docker_data_dir }}/sentry:/var/lib/sentry:rw"

    - name: start sentry cron server
      docker_container:
        name: "track_cron_server"
        image: "sentry:{{ sentry_version }}"
        state: started
        command: "run cron"
        auto_remove: yes
        links:
          - "{{ redis_container_name }}:redis"
          - "{{ postgres_container_name }}:postgres"
        env:
          SENTRY_DB_USER: "{{ docker_user }}"
          SENTRY_DB_PASSWORD: "{{ docker_pass }}"
          SENTRY_SECRET_KEY: "{{ SENTRY_SECRET_KEY.stdout }}"

    - name: start sentry worker server
      docker_container:
        name: "track_worker_server"
        image: "sentry:{{ sentry_version }}"
        state: started
        command: "run worker"
        auto_remove: yes
        links:
          - "{{ redis_container_name }}:redis"
          - "{{ postgres_container_name }}:postgres"
        env:
          SENTRY_DB_USER: "{{ docker_user }}"
          SENTRY_DB_PASSWORD: "{{ docker_pass }}"
          SENTRY_SECRET_KEY: "{{ SENTRY_SECRET_KEY.stdout }}"

Sentry JS-SDK发送错误日志

Sentry-JavaScript异常日志

Sentry后台

参考

https://sentry.io/welcome/
http://www.ywjt.org/index.php/archives/1528
https://blog.didiyun.com/index.php/2018/11/29/monitor/
https://segmentfault.com/a/1190000012051620?utm_source=tag-newest
https://stackoverflow.com/questions/43263045/ansible-get-docker-container-output-during-container-creation
https://docs.fundebug.com/

私有监控方案

https://www.frontjs.com/doc/view/what
https://github.com/BetterJS/badjs-report
http://taobaofed.org/blog/2015/10/28/jstracker-how-to-collect-data/
https://github.com/saijs/wiki
http://www.aliued.cn/2012/10/27/%E6%9E%84%E5%BB%BAweb%E5%89%8D%E7%AB%AF%E5%BC%82%E5%B8%B8%E7%9B%91%E6%8E%A7%E7%B3%BB%E7%BB%9F-fdsafe.html

总结

工具本身只是提升开发效率和开发质量的手段，结合不同项目研发流程实际场景，自由定制符合团队需求的基础设施组合，才是工具使用的王道;
另外测试环节涉及的理念和工具很多，后续将针对实践持续总结。前面提到的工具也有很多其他好的替代方案，建议根据需要进行尝试;
文中涉及配置仅供参考，实际部署建议参考详细官方文档。