如何使用Postman和Newman在CI环境中自动化REST API端到端测试
Postman is a great tool to explore REST APIs. You can build requests and try them out to get quick feedback. Then you can persist them as collections to make sure that the knowledge doesn't get lost.
邮递员是探索REST API的好工具。 您可以构建请求并进行尝试，以获得快速反馈。 然后，您可以将它们持久保存为集合，以确保知识不会丢失。
Newman, the CLI version of Postman, allows you to take it to the next level and transform a collection into a suite of automated end-to-end tests. This suite will run then in your CI tool of choice. In this article I will explore the benefits of doing so and show you how to set it up.
Newman (Postman的CLI版本)使您可以将其带入一个新的水平，并将集合转换为一套自动化的端到端测试套件。 然后，该套件将在您选择的CI工具中运行。 在本文中，我将探讨这样做的好处，并向您展示如何进行设置。
什么是API上下文中的端到端测试？ (What is an End-to-End test in the context of an API?)
Testing nomenclature is a tricky thing. Keeping the testing pyramid in mind, we can picture them as very high level tests. These tests confirm that a particular REST API works as intended, treating the internals as a black box. We don't involve any UI in the process, which helps reduce the flakiness.
测试术语是一件棘手的事情。 牢记测试金字塔 ，我们可以将它们描述为非常高级的测试。 这些测试确认特定的REST API可以按预期工作，并将内部部件视为黑匣子。 我们在此过程中不涉及任何用户界面，这有助于减少脆弱性。
通过极客＆戳/ CC BY
Flaky tests are extremely annoying, as every developer has experienced at some point. Instead of banging our head against the wall trying to fix the unfixable, we can mitigate the problem by using lower level tests.
易碎的测试非常烦人，因为每个开发人员在某个时候都经历过。 我们可以通过使用较低级别的测试来减轻问题，而不必用力气砸墙以解决无法修复的问题。
为什么要进行这些测试？ (Why should I have these tests?)
There are two different scenarios I'd like to cover:
我想介绍两种不同的情况：
The first is testing your own REST APIs. These tests add an extra layer of confidence. Surely, you are using a healthy mix of different tests (unit, integration, functional, …). End-to-end tests can be the final confirmation that everything looks fine.
首先是测试您自己的REST API。 这些测试增加了额外的信心。 当然，您在使用各种测试(单元测试，集成测试，功能测试……)的健康组合。 端到端测试可以最终确定一切都很好。
The second case is testing APIs that you don't control. In my last projects most of the data we consumed came from APIs served by other teams. More than once I spent half a day debugging an error in my app, only to notice that a downstream API was borked all along. Automated tests cover that integration, and help isolate issues.
第二种情况是测试您无法控制的API。 在我的上一个项目中，我们消耗的大多数数据来自其他团队提供的API。 我花了半天多的时间调试应用程序中的错误，才发现下游API一直很闷。 自动化测试涵盖了该集成，并有助于隔离问题。
生活文件 (Living documentation)
A collection of tests that are being regularly executed serve as the best documentation for an API. Have you searched for something in any corporate wiki lately? If you find anything at all you should be happy. It will be probably incomplete. Or just flat out wrong. Fun times.
定期执行的测试的集合是API的最佳文档。 您最近是否在任何公司Wiki中搜索过什么？ 如果您发现任何东西，您应该会感到高兴。 这可能是不完整的。 或者只是完全错误。 娱乐时间。
监控方式 (Monitoring)
In both cases, these tests can morph from a gateway in the building process to an active monitoring tool. By constantly running them, you make sure that the API is still behaving as you expect. Otherwise, the right alarms will be raised. You don't want to realize something is wrong just when a customer complains.
在这两种情况下，这些测试都可以从构建过程中的网关转变为活动的监视工具。 通过不断运行它们，可以确保API仍按预期运行。 否则，将发出正确的警报。 您不希望仅在客户抱怨时才发现问题。
为什么不使用消费者驱动的合同测试呢？ (Why not use consumer driven contract tests instead?)
Great question, if I may say so myself. CDCs are an excellent way to ensure that an API conforms to what a client expects from it. If you can set them up properly, they will replace end-to-end tests almost completely. Remember, keep pushing the tests to a lower level whenever you can.
很好的问题，如果我自己说。 CDC是确保API符合客户期望的绝佳方法。 如果可以正确设置它们，它们将几乎完全替代端到端测试。 请记住，只要有可能，就将测试降低到更低的水平。
They don't work in every situation, though. If you don't control both the provider and the consumer, you have to rely on another party. If they don't fulfill their part of the contract the tests will be useless. Some teams are just not in the position of continuously running tests against a contract. Running your own tests could be your best bet.
但是，它们并非在每种情况下都有效。 如果您不能同时控制提供者和使用者，则必须依靠另一方。 如果他们不履行合同的一部分，则测试将毫无用处。 有些团队只是无法按照合同进行持续测试。 最好运行自己的测试。
Anyways, having laid out the rationale, it's time for some code.
无论如何，在阐明了基本原理之后，就该编写一些代码了 。
创建邮递员收藏 (Creating a Postman collection)
集合 (The collection)
We are defining a number of calls that will be executed sequentially inside our CI. Each call executes a request against the API. Then it runs some tests to check that the request was successful, checking the status code and the body as well.
我们定义了一些将在CI中按顺序执行的调用。 每个调用都会针对API执行一个请求。 然后，它运行一些测试以检查请求是否成功，并检查状态代码和正文。
In order to create the collection, I tend to use the Postman app. I like to extract things like URLs and parameters to an environment. Then configuring it becomes easier, and you don't have any sensitive information in the collection itself. Your history is a convenient place to start building this collection.
为了创建收藏集，我倾向于使用Postman应用程序。 我喜欢将诸如URL和参数之类的内容提取到环境中 。 然后，配置它变得更加容易，并且集合本身中没有任何敏感信息。 您的历史记录是开始构建此收藏集的便利位置 。
Once you are satisfied with the collection, you can export it as a JSON file. That file can be committed in source control to serve as a base for the pipeline that will run the tests. There is a Pro and Enterprise version that helps managing collections, which I haven't really tried. Still, a good ol' git repository is more than enough to get rolling.
对集合满意后，可以将其导出为JSON文件。 可以在源代码管理中提交该文件，以作为将运行测试的管道的基础。 有一个Pro和Enterprise版本可以帮助管理集合，但我还没有尝试过。 尽管如此，一个好的git仓库足以使之滚动。
运行收藏 (Running the collection)
Until now we have been using regular Postman and nothing else. Now it's the time for newman to shine. What am I talking about, anyways? I'll quote the official docs directly:
到现在为止，我们一直在使用常规邮递员，而没有其他东西。 现在是纽曼大放异彩的时候了。 我到底在说什么？ 我将直接引用官方文档 ：
Newman is a command line Collection Runner for Postman. It allows you to run and test a Postman Collection directly from the command line.
Newman是Postman的命令行Collection Runner。 它使您可以直接从命令行运行和测试Postman Collection。
Good that we clarified that! It is installed as a npm package, which can result in a package.json as simple as this:
好，我们澄清了！ 它以npm软件包的形式安装，这可能导致package.json如此简单：
{
"name": "postman-utils",
"version": "0.0.1",
"private": true,
"description": "Postman utilities",
"scripts": {
"newman": "node_modules/.bin/newman run"
},
"dependencies": {
"newman": "^4.4.1"
}
}as mentioned before, you don't want to hardcode variables like URLs, parameters or, God forbid, passwords in that collection. It's not flexible, and it's not safe. Instead, I like to use a configuration file which includes all these values. But if we want to commit that file, we still need to figure out a way to avoid putting secrets in there. I use it as a template and replace values at runtime with envsubst. The configuration file looks like this
如前所述，您不想硬编码URL，参数或该集合中的密码之类的变量。 它不灵活，也不安全。 相反，我喜欢使用包含所有这些值的配置文件。 但是，如果要提交该文件，我们仍然需要找出一种避免在其中放置秘密的方法。 我将其用作模板，并在运行时使用envsubst替换值。 配置文件如下所示
{
"id": "425cf4df-d994-4d91-9efb-41eba1ead456",
"name": "echo",
"values": [
{
"key": "host",
"value": "${HOST}",
"enabled": true
}
]
}You can orchestrate this with a simple bash script. The script injects the variables into the template, runs newman, and deletes the files to avoid leaks. It goes very well with gopass, where you can safely store your secrets and fetch them through the script.
您可以使用简单的bash脚本进行编排。 该脚本将变量注入模板，运行newman，并删除文件以避免泄漏。 与gopass一起使用非常好，您可以在其中安全地存储您的秘密，并通过脚本来获取它们。
setup-newman() {
settings=/tmp/settings.json.$$
result=/tmp/variables.json.$$
# shellcheck disable=SC2064
trap "rm -f \"$settings\" \"$result\"" EXIT
}
run-newman() {
local service=${1?You need to provide the service to check}
envsubst < "$service.environment.json.template" > "$settings"
npx newman run "$service.json" \
-e "${settings}" \
--export-environment "${result}"
}that helper can be called with the collection that you want to test. Exported variables will be picked by envsubst. npx gives us a little bit more of flexibility finding the newman binary, in case you don't want to use a package.json but have it globally installed.
可以使用您要测试的集合来调用该帮助程序。 导出的变量将由envsubst 。 NPX给我们一点点的灵活性寻找newman二进制文件，如果你不想使用package.json ，但有它全球的装机量。
goal_check-service() {
setup
export SERVICE_PASSWORD=${SERVICE_PASSWORD:-$(gopass store/service/password)}
run_newman service
}测验 (Tests)
Doing a request is but the first step. Remember, we aim to build a test suite. We have a convenient test tab in Postman that we can use to write our tests.
发出请求只是第一步。 记住，我们的目标是建立一个测试套件。 我们在Postman中有一个方便的测试选项卡，可用于编写测试。
Our tests are written in JavaScript, using Chai. Let's say I want to test that my call delivered a list of results, I could do it like this:
我们的测试使用Chai用JavaScript编写。 假设我要测试我的通话是否传递了结果列表，可以这样进行：
var getResults = function() {
var jsonData = pm.response.json();
return jsonData['results'];
};
pm.test("Request was successful", function () {
pm.response.to.have.status(200);
});
pm.test("There are results", function () {
pm.expect(getResults().length).to.be.above(0);
});More details can be found here
更多细节可以在这里找到
建筑流量 (Building flows)
All the calls in a collection get executed sequentially. This offers us the opportunity to test whole flows instead of just single calls. One such a flow for a /posts resource is:
集合中的所有调用将顺序执行。 这为我们提供了测试整个流程的机会，而不仅仅是单个调用。 /posts资源的这种流之一是：
Get a list of all
posts获取所有
posts的列表Fetch the first
postin the list获取列表中的第一篇
postUpdate the
post更新
post
We'll build a suite of parametrized tests that will continue to work over time, not just the first time that you ran it. An important part of this is modifying the environment in a request. That is our way of transmitting parameters between requests. Let's say our first request was successful, as corroborated by our tests. Then we store the id on a variable that will be used to fetch a particular entity.
我们将构建一套参数化测试，这些测试将随着时间的推移而继续有效，而不仅仅是您第一次运行它。 其中的重要部分是修改请求中的环境。 这是我们在请求之间传递参数的方式。 假设我们的测试证实了我们的第一个请求成功。 然后，我们将ID存储在一个变量中，该变量将用于获取特定实体。
// First result in the list
var post = getResults()[0];
// Pass variables to other stages
pm.environment.set("id", post.id)The next request can use that parameter as any that we set manually.
下一个请求可以使用该参数作为我们手动设置的任何参数。
根据条件忽略呼叫 (Ignoring calls based on a condition)
Flows might need also need some logic to skip certain requests. Let's say you have a request that is creating a new entity through a POST. You want to have that request, but you may not want to run it on every commit. Maybe you just want do it once per day. In that case, we'll skip the test based on a certain variable.
流可能还需要一些逻辑来跳过某些请求。 假设您有一个通过POST创建新实体的请求。 您想拥有该请求，但是您可能不想在每次提交时都运行它。 也许您只想每天做一次。 在这种情况下，我们将基于某个变量跳过测试。
// Do not run create request in sequence, unless executeCreate is set to true
if(!pm.environment.get("executeCreate")) {
postman.setNextRequest('Get other posts')
}The variable goes into the configuration file, and is set to a environment variable that gets injected through our script, as I showed above.
如上所示，该变量进入配置文件，并设置为通过我们的脚本注入的环境变量。
持续集成的时间 (Time for some continuous integration)
At this point you should have a collection that runs locally. Running this once is fine, but why not run it for every commit? Or maybe every hour, if you want to check an API that you don't control?
此时，您应该有一个在本地运行的集合。 运行一次即可，但是为什么不为每次提交运行一次呢？ 或者，也许每个小时，如果您想检查您无法控制的API？
Your CI pipeline is a perfect place to do this. I'm going to use CircleCI for my example, but any CI will do. I run the tests inside a docker image that I built which includes all the required dependencies. There is an official Docker image provided by Postman already. However, it does not contain envsubst and it uses an older NodeJS version.
您的CI管道是执行此操作的理想场所。 我将以CircleCI为例，但是任何CI都可以。 我在构建的docker映像中运行测试，该映像包含所有必需的依赖项。 Postman已经提供了一个官方的Docker镜像。 但是，它不包含envsubst ，它使用一个旧版本的NodeJS。
The helper script that we built in the step before will work without any changes inside CircleCI. We just have to provide the required secrets as variables. This is the job:
我们在上一步中构建的帮助程序脚本将可以在CircleCI内部进行任何更改。 我们只需要提供所需的机密作为变量即可 。 这是工作：
healthcheck:
docker:
- image: sirech/newman-executor:12.6
steps:
- checkout
- run: ./go test-e2ewhich will produce a report similar to this:
这将产生类似于以下内容的报告：
那替代品呢？ (What about the alternatives?)
Many frameworks provide their own way of running tests against a running API. In Spring Boot, for instance, you can use MockMvc to test controllers. You can use both, in my view. First the native tests, so to speak, and then layer Postman Tests on top.
许多框架提供了自己的针对运行中的API运行测试的方式。 例如，在Spring Boot中 ，您可以使用MockMvc来测试控制器。 在我看来，您可以同时使用两者。 可以这么说，首先是本机测试，然后是Postman测试。
And let's not forget about good ol' curl. I had a huge collection of curl commands with which I tested an API that was needed for my last project. However, managing that becomes more and more tedious over time. If you want to use send complex requests, like certificates or cookies, Postman is way more convenient to use. Moreover, you can use JavaScript instead of bash, which can make things a bit easier to read and maintain.
而且，别忘了良好的卷曲度 。 我收集了大量curl命令，并用它们测试了上一个项目所需的API。 但是，随着时间的推移，对其进行管理变得越来越乏味。 如果要使用发送复杂的请求(如证书或cookie)，Postman的使用方式将更加方便。 此外，您可以使用JavaScript代替bash，这会使事情更易于阅读和维护。
还有什么？ (What else?)
This is already quite a lot and it's just the beginning. Anything that you do with an API you can also automate. For instance, in my previous project we had a collection that ran an OAuth Flow. That got us a token that we could use to make requests against an authorized endpoint.
这已经很多了，这仅仅是开始。 您使用API所做的任何事情都可以实现自动化。 例如，在我之前的项目中，我们有一个运行OAuth Flow的集合。 这为我们提供了一个令牌，我们可以使用该令牌向授权端点发出请求。
回购示例 (A repo to use as an example)
Here is a repository for a Kotlin application that runs a Postman collection as an e2e test. It can serve as a starter kit to get going with high quality End-to-End API Tests.
这是Kotlin应用程序的存储库，该应用程序运行Postman集合作为e2e测试。 它可以用作入门套件，以进行高质量的端到端API测试。
翻译自: https://www.freecodecamp.org/news/how-to-automate-rest-api-end-to-end-tests/