lighthouse_如何为CI / CD设置自动Lighthouse测试

lighthouse

Lighthouse by Google has become increasingly more popular in recent years — even to the point where companies start seeing measurable increase in profit when their performance is improved.

近年来，Google的Lighthouse变得越来越流行-甚至当公司的业绩提高时，公司开始看到利润的可衡量的增长。

Some colleagues and I thought to ourselves — why not have a way for our developers to see how their projects perform in Lighthouse. Preferably we wanted the test to run in our CI environment Azure Pipelines. We set some goals for the project, and I went to work:

我和一些同事对自己进行了思考-为什么不让开发人员了解他们的项目在Lighthouse中的表现。 最好我们希望该测试在我们的CI环境Azure管道中运行。 我们为该项目设定了一些目标，然后我开始工作：

1. A continuous measurement of several Lighthouse categories every time the code changes
   每次代码更改时，连续测量几个Lighthouse类别
2. Each project should be able to define their own goals and have them measured
   每个项目都应该能够定义自己的目标并进行衡量
3. The Lighthouse score should be tracked on a timeline to see the historic trends
   应该在时间轴上跟踪Lighthouse分数，以查看历史趋势

完美的工具 (The perfect tool)

The Google Chrome team already published a CLI for Node, that can run the tests, generate a report, and upload that report to a reporting server.

Google Chrome团队已经发布了Node的CLI ，该CLI可以运行测试，生成报告并将该报告上传到报告服务器 。

We did some research and found an extension to Azure Pipelines that wraps the CLI from Google, including an article explaining how to use it, and I discovered a Docker container wrapper for the Lighthouse CI reporting server. That was pretty much everything I needed.

我们进行了一些研究，发现了Azure Pipelines的扩展，该扩展包装了Google的CLI，其中包括一篇介绍如何使用 CLI 的文章 ，我发现了 Lighthouse CI报告服务器的Docker容器包装 。 那几乎就是我需要的一切。

The architecture for the Azure Pipelines build could then look like this:

然后，Azure Pipelines构建的体系结构应如下所示：

Architecture of the Azure Pipelines setup Azure Pipelines安装的体系结构

积分 (Integration)

With the extension in hand, it was pretty simple to integrate the tasks in the existing build pipeline. I ran into a dilemma here, though.

有了扩展，将任务集成到现有构建管道中非常简单。 不过，我在这里陷入了困境。

我是否要在内部版本或发布管道中或同时在两者中运行Lighthouse测试？ (Do I run the Lighthouse test in the build or in the release pipeline, or both?)

To make a long story short, I ended up adding an extra deploy stage to the build pipeline using Azure’s Multi-stage YAML Pipeline which deployed the project to a dedicated CI environment before running the Lighthouse test — everything happens inside the build pipeline. This allowed first and foremost to see the result of the build in connected applications, like BitBucket, but also to actually mark the build as failed, if the Lighthouse test did not pass — just like any other unit or integration test.

长话短说，我最终使用Azure的多阶段YAML管道在构建管道中添加了一个额外的部署阶段，该阶段在运行Lighthouse测试之前将项目部署到了​​专用的CI环境中- 一切都发生在构建管道中 。 首先，这可以看到连接的应用程序(例如BitBucket)中构建的结果，而且还可以在Lighthouse测试未通过的情况下实际将构建标记为失败(就像其他任何单元或集成测试一样)。

It was simple to add an extra stage for the Lighthouse test after that, which is dependant on the deploy stage:

在此之后，为Lighthouse测试添加一个额外的阶段很简单，这取决于部署阶段：

- stage: test_ci
  displayName: Test CI
  dependsOn: deploy
  jobs:
    - job: lhci
      variables:
        # Set the branch name to variables.Build.SourceBranch because the lighthouse-ci task is using the full name by default
        LHCI_BUILD_CONTEXT__CURRENT_BRANCH: '$(Build.SourceBranch)'
        # Append build id to the commit message shown in LHCI
        LHCI_BUILD_CONTEXT__COMMIT_MESSAGE: 'Build #$(Build.BuildId): $(Build.SourceVersionMessage)'
      steps:
        - task: lighthouse-ci@1
          displayName: Run Lighthouse CI
          # We don't want the build to fail yet - the devs should have a chance to fix the code first
          continueOnError: true
          inputs:
            # Use the 'autorun' command to invoke healthcheck, assets, collect, and upload tasks automatically
            command: 'autorun'
            configFilePath: '$(System.DefaultWorkingDirectory)/lighthouserc.json'
            # Override some parameters with variables stored in Azure Pipelines
            parameters: '--collect.url=$(lhci-url-dev) --upload.token=$(lhci-build-token-dev) --upload.serverBaseUrl=$(lhci-server-url)'

I added a Lighthouse CI configuration file with some presets to the Git repository:

我向Git存储库添加了带有一些预设的Lighthouse CI配置文件：

{
    "ci": {
        "collect": {
            "method": "node",
            "additive": false,
            "headful": false,
            "numberOfRuns": 3,
            "chromePath": false,
            "settings": {
                "chromeFlags": "--ignore-certificate-errors"
            }
        },
        "assert": {
            "preset": "lighthouse:no-pwa",
            "assertions": {
                "categories:performance": [
                    "warn",
                    {
                        "aggregationMethod": "optimistic",
                        "minScore": 0.7
                    }
                ],
                "categories:accessibility": [
                    "warn",
                    {
                        "aggregationMethod": "optimistic",
                        "minScore": 0.9
                    }
                ]
            }
        },
        "upload": {
            "target": "lhci"
        }
    }
}

CollectI wanted to run the test 3 times in a row using a headless version of Chrome (Puppeteer), because the first run would probably be slower than the following runs, because the website would still be in a state of rebuilding caches and so on, so it was neccessary to have at least a few test runs.

收集我 想要使用无头版本的Chrome(Puppeteer)连续运行该测试3次，因为第一次运行可能会比随后的运行慢，因为该网站仍处于重建缓存的状态，依此类推，因此至少要进行一些测试。

AssertI wanted to use the preset called “lighthouse:no-pwa”, which includes all of the default tests except PWA tests (I am testing just a server-side rendered Angular app), and I wanted to include a few assertions of my own to illustrate how to set it up.

断言我想使用名为“ lighthouse：no-pwa”的预设，它包括除 PWA测试(我仅测试服务器端渲染的Angular应用) 之外的所有默认测试，并且我想包含一些关于我的断言自己来说明如何设置它。

Note how I was actually a bit too aggressive in the scoring of “performance”, which I should probably adjust at some point — after all I was running the test from a build server on a B1 app service plan. The aggregation method optimistic meant, that the report included only the best result of the 3 runs.

请注意，我实际上对“性能”的评分有点过于激进，我可能应该在某个时候进行调整，毕竟我是在B1应用服务计划的构建服务器上运行测试的。 optimistic的汇总方法意味着该报告仅包含3次运行的最佳结果。

UploadUploads the report to an “lhci” target server. The rest of the parameters, like url and token, were provided in the YAML file using build variables.

上载将报告上载到 “ lhci”目标服务器。 其余参数(例如url和token )是使用构建变量在YAML文件中提供的。

The build then looked like this:

然后，构建看起来像这样：

Notice how the “lhci” task was marked with an exclamation mark? That was because it failed the assertions, that I set up in the configuration file, but it still allowed the build to continue because of the continueOnError: true property in the build task.

请注意，“ lhci”任务是如何用感叹号标记的？ 那是因为它使我在配置文件中设置的断言失败了，但是由于构建任务中的continueOnError: true属性，它仍然允许构建继续进行。

That should make the developers react ASAP :-)

这应该使开发人员尽快做出React：-)

The build log already started to look interesting, where you might notice stuff that probably should be looked in to:

构建日志已经开始看起来很有趣，您可能会在这里注意到应该注意的内容：

Log of the Lighthouse CI output in Azure Pipelines Azure Pipelines中Lighthouse CI输出的日志

The output of the tool was a report of all assertions, which could be uploaded to Lighthouse itself or, in my case, to the Lighthouse CI server.

该工具的输出是所有断言的报告，可以将其上载到Lighthouse本身，或者就我而言，上载到Lighthouse CI服务器。

Lighthouse CI报告服务器 (Lighthouse CI reporting server)

The architecture for the reporting server was pretty straightforward — an app service backed by a Postgres database:

报表服务器的架构非常简单-由Postgres数据库支持的应用程序服务：

Architecture of the Lighthouse CI reporting server Lighthouse CI报告服务器的体系结构

The reporting server had a web UI, which is neatly built and focuses on a comparison chart between two reports:

报告服务器具有一个Web UI，该UI整洁地构建，并着重于两个报告之间的比较表：

Comparing latest build to base build — do not trust the performance scores on a CI environment 将最新版本与基本版本进行比较-不要相信CI环境下的性能得分

It also had a timeline for several measurements:

它也有一个时间表来进行几个测量：

Timeline of the accessibility measurement group 辅助功能评估组的时间表

Each report could be opened in Google’s Lighthouse Viewer for even more information as well.

每个报告都可以在Google的Lighthouse Viewer中打开，以获取更多信息。

So it all looked great and fit neatly together, and the developers now had a nice timeline and distribution of each measured category.

因此，它们看起来都很棒，并且整齐地结合在一起，并且开发人员现在有了一个不错的时间表和每个测量类别的分布。

我学到了什么？ (What have I learned?)

TimeoutsIt turned out to be pretty easy to integrate the Lighthouse CI tool in the build pipeline, but after a month or so of tests, it would seem, that the test tool simply fails once in a while due to Puppeteer timeout, but not a big deal.

超时事实证明，将Lighthouse CI工具集成到构建管道中非常容易，但是经过一个月左右的测试，似乎该测试工具由于Puppeteer超时而偶尔会失败，但并非如此。没关系

Where to run the testI am still unsure if running the test in the build pipeline is the best solution, because it does take a while to run. I considered making the build pipeline trigger another pipeline to run the test side-by-side, but that would not be feasible if the test was running on the CI environment, because another build might overwrite the environment before the test finished. And I would also lose the possibility to mark the build as failing.

在哪里运行测试我仍然不确定在构建管道中运行测试是否是最佳解决方案，因为它确实需要一段时间才能运行。 我考虑过让构建管道触发另一个管道来并行运行测试，但是如果测试在CI环境上运行，那将是不可行的，因为另一个构建可能会在测试完成之前覆盖环境。 而且我也将失去将构建标记为失败的可能性。

Where is the best place to target the testAzure has a feature called ARM (Azure Resource Manager) deployments, which enables you to spin whole environments up and down using predefined templates. If we wrote templates for the project, we could simply spin up an entire environment for the sole purpose of running the test, then taking it down again, or repurpose it for the next test. That would at least make sure the test is running on a pure environment, but the setup of this is complex.

针对测试的最佳位置 Azure拥有一项称为ARM(Azure资源管理器)部署的功能，通过该功能，您可以使用预定义的模板来上下旋转整个环境。 如果我们为项目编写模板，则可以仅出于运行测试的目的而旋转整个环境，然后再次删除它，或将其重新用于下一个测试。 至少可以确保测试在纯环境中运行，但是此设置很复杂。

Which is the best URL to testI spent some time considering which URL should be tested; for ease of use, I chose to test only the frontpage, because I assume that is a hotspot. But the frontpage of our CI environment pulls content from an Umbraco Cloud dev environment, which is currently full of QA test content, so I am not sure that is the best page to test. Another consideration could be to have a dedicated test page containing various content and widgets would have been better suited.

我花了一些时间考虑应该测试哪个URL，这才是测试的最佳URL 。 为了易于使用，我选择仅测试首页，因为我认为这是一个热点。 但是我们CI环境的首页从Umbraco Cloud开发环境中提取内容，该环境目前充满了QA测试内容，因此我不确定这是测试的最佳页面。 另一个考虑因素是拥有包含各种内容的专用测试页面，而小部件将更适合。

Thoughts for the futureIf the project is containerized, a fun thought would be to spin it up directly on the build server and test it there. Hardware-wise that would not have been worse, than our base CI server, but I did not go down this road for sake of simplicity.

对未来的思考如果将项目容器化，一个有趣的想法是将其直接旋转到构建服务器上并在其中进行测试。 在硬件方面，这不会比我们的基础CI服务器更糟糕，但是为了简单起见，我没有走这条路。

资源资源 (Resources)

1. Thanks to Gurucharan Subramani for a great guide and Azure tool: https://www.gurucharan.in/web/nodejs/lighthouse-ci-the-complete-guide-part-2/

   感谢Gurucharan Subramani提供了出色的指南和Azure工具： https : //www.gurucharan.in/web/nodejs/lighthouse-ci-the-complete-guide-part-2/

2. Here’s the Azure tool I used: https://marketplace.visualstudio.com/items?itemName=gurucharan.lighthouse-ci

   这是我使用的Azure工具： https : //marketplace.visualstudio.com/items?itemName=gurucharan.lighthouse-ci

3. The Lighthouse CI client package: https://www.npmjs.com/package/@lhci/cli

   Lighthouse CI客户端软件包： https : //www.npmjs.com/package/@lhci/cli

4. The Lighthouse CI server package: https://www.npmjs.com/package/@lhci/server

   Lighthouse CI服务器软件包： https : //www.npmjs.com/package/@lhci/server

5. Thanks to Patrick Hulce for building and maintaining the Lighthouse CI Docker wrapper: https://hub.docker.com/r/patrickhulce/lhci-server/

   感谢Patrick Hulce构建和维护Lighthouse CI Docker包装器： https : //hub.docker.com/r/patrickhulce/lhci-server/

翻译自: https://medium.com/impact-developers/how-to-set-up-automatic-lighthouse-testing-for-ci-cd-236cb955e968

lighthouse