terraform_从terraform云状态查询输出

terraform

Typical disclaimer: While I do work for Google, my tech blog is not official Google documentation, nor does it necessarily follow Google technical or product direction. This blog reflects my own thinking, study and work. I have not received any input, feedback, compensation or anything (not even a sticker!) from any Terraform representative.

典型的免责声明：虽然我确实在Google工作，但我的技术博客不是Google的正式文档，也不一定遵循Google的技术或产品指导。 这个博客反映了我自己的思想，学习和工作。 我没有从任何Terraform代表那里收到任何输入，反馈，补偿或任何东西(甚至没有贴纸！)。

tl; dr (tl;dr)

• Create or acquire a Terraform Cloud account and User Token
  创建或获取Terraform Cloud帐户和用户令牌
• Upload existing state file into Terraform Cloud
  将现有状态文件上传到Terraform Cloud
• Use Terraform API to query for state outputs
  使用Terraform API查询状态输出

• Parse output with jq to obtain value

  用jq解析输出以获得值

• Example demonstration is located within the https://gitlab.com/mike-ensor/terraform-query-remote-state repository

  示例演示位于https://gitlab.com/mike-ensor/terraform-query-remote-state存储库中

问题(The problem)

Output variables are commonly used in Terraform to store dynamically generated values across multiple layers of infrastructure involving isolated state files. The challenge comes when access to dynamic values is not exposed via traditional methods such as key-value stores, secrets management or databases, and the value is needed outside the context of Terraform.

输出变量通常在Terraform中使用，以在涉及隔离状态文件的多层基础结构中存储动态生成的值。 当没有通过传统方法(例如键值存储，机密管理或数据库)暴露对动态值的访问并且在Terraform上下文之外需要该值时，挑战就来了。

The use of multiple state files is a good practice to not only reduce the blast radius of changes, but also to encourage small changes, isolate environments, adhere to compliance, restrict access to critical or important resources and reduce coupling between infrastructure layers leading to developer self-service with application, team or division-level resources. Terraform provides an interface from one module to pull data from a previous or upstream state file during the plan and apply phases. This blog post will not go into why a best practice is to use multiple state files (perhaps in a later blog since the internets don’t seem to be highly opinionated). Details on using terraform_remote_state can be found in the official documentation.

使用多个状态文件是一种良好的做法，不仅可以减少更改的爆炸半径，而且可以鼓励进行较小的更改，隔离环境，遵守法规，限制对关键或重要资源的访问并减少基础结构层之间的耦合，从而导致开发人员使用应用程序，团队或部门级别的资源进行自助服务。 Terraform提供了一个模块的接口，可以在plan和apply阶段从上一个或上游状态文件中提取数据。 这篇博客文章不会讨论为什么最佳实践是使用多个状态文件(也许在以后的博客中，因为互联网似乎没有得到很高的评价)。 有关使用terraform_remote_state详细信息，请参见官方文档。

When in the ecosystem of Terraform, remote state queries are straight forward, but what happens when you’re no longer in the context of Terraform, say during configuration of an application and the value you need is in a Terraform state file? A concrete example is when you use random_string or random_petto produce unique names and then need to use the name later in application configuration, bash script or in a CICD pipeline.

在Terraform生态系统中时，远程状态查询很简单，但是当您不再处于Terraform上下文中时(例如，在配置应用程序期间并且所需的值在Terraform状态文件中)会发生什么？ 一个具体的示例是，当您使用random_string或random_pet生成唯一的名称，然后稍后需要在应用程序配置，bash脚本或CICD管道中使用该名称时。

传统查询Terraform输出 (Traditional query of Terraform outputs)

Querying for outputs is rather simple when using the CLI tool, simply use terraform outputs or terraform show and view the contents of the variable. However, this only applies when in a context where the terraform CLI binary is available and authentication to the state is possible. In many cases the binary may not be available such as within an application or CICD pipeline. The rest of this blog post will focus on the scenario where the value of an output is desired, but access to the Terraform CLI is not possible.

使用CLI工具时，查询输出非常简单，只需使用terraform outputs 或terraform show并查看变量的内容即可。 但是，这仅在可使用terraform CLI二进制文件且可以进行状态验证的上下文中适用。 在许多情况下，例如在应用程序或CICD管道中，二进制文件可能不可用。 本博客文章的其余部分将重点讨论需要输出值但无法访问Terraform CLI的场景。

暴露状态 (Exposing the state)

The first step in querying Terraform state outputs is to place the state in a location where APIs can access the contents. While there are some OSS and vendor software that can visualize state file contents from Google Cloud, AWS, Azure storage buckets, Terraform Cloud exposes an easy API to access state file contents and this blog will focus on the simplest solution.

查询Terraform状态输出的第一步是将状态放置在API可以访问内容的位置。 尽管有一些OSS和供应商软件可以可视化来自Google Cloud，AWS，Azure存储桶的状态文件内容，但是Terraform Cloud公开了一个简单的API来访问状态文件内容，并且本博客将重点介绍最简单的解决方案。

First step is to acquire a Terraform Cloud account. If your company does not have access, signing up for a free account is simple. You will need to create a token. From your profile icon (top-right) click on “User Settings” then “Tokens”. Create a token and save the value somewhere safe, the contents are never displayed again. Use the token value as the bearer value in the API calls unless noted.

第一步是获取Terraform Cloud帐户。 如果您的公司没有访问权限，那么注册免费帐户很简单。 您将需要创建一个令牌。 在您的个人资料图标(右上)中，单击“用户设置”，然后单击“令牌”。 创建令牌并将值保存在安全的地方，不会再显示内容。 除非另有说明，否则将令牌值用作API调用中的bearer值。

NOTE: companies often create tokens at a team or organization level to represent continuous delivery agents, this blog is focused on an individual querying the API and will use personal tokens.

注意：公司通常会在团队或组织级别创建令牌来表示连续的交付代理，此博客着重于查询API的个人并将使用个人令牌。

Next, gather the workspace_idinformation from the cloud workspace. If this is a new project or new to Terraform Cloud, create a Workspace using the “CLI driven workspace”. Follow the instructions given and place the backend code into your .tf file. The workspace_id is located in the “general settings” tab (top-right) from the Workspace UI screen. Copy the ID and store for use later.

接下来，从云工作区中收集workspace_id信息。 如果这是一个新项目或Terraform Cloud的新项目，请使用“ CLI驱动的工作空间”创建一个工作空间。 请按照给出的说明进行操作，然后将backend代码放入.tf文件中。 workspace_id位于工作区用户界面屏幕的“常规设置”选项卡(右上方)中。 复制ID并存储以备后用。

NOTE: Cloud workspaces are similar to the CLI workspace, but are not the same thing

注意：云工作区与CLI工作区类似，但不是同一件事

At this point, you should have the following values:

此时，您应该具有以下值：

• workspace-id — the ID of the workspace where the state is/will exist

  workspace-id —状态将/将存在的工作区的ID

• user-token- token used to query the API based on your user’s authentication

  user-token -用于根据用户的身份验证查询API的令牌

迁移现有的远程状态(Migrating Existing Remote State)

If the target project has is new and has no existing state or only has local state files, then skip the this section.

如果目标项目是新项目，没有现有状态，或者只有本地状态文件，请跳过本节。

The typical remote state is implemented using an Object Storage mechanism such as a Google Cloud Storage bucket. Since remote state is stored, well, remotely, the state needs to be pulled down locally in order to push up into the Terraform Cloud workspace. This code block shows current configuration using a GCS bucket, but has commented out the cut-copy-pasted (and redacted) Terraform Cloud backend.

典型的远程状态是使用对象存储机制(例如Google Cloud Storage存储桶)实现的。 由于很好地远程存储了远程状态，因此需要在本地将状态下拉，以便向上推送到Terraform Cloud工作空间中。 该代码块显示了使用GCS存储桶的当前配置，但已注释掉剪切复制粘贴(和编辑的)Terraform Cloud后端。

Switching remote state storage locations is straight forward. The process the following:

切换远程状态存储位置非常简单。 该过程如下：

1. Stop all mutation of the state (ensure no running changes and the state is unlocked).
   停止状态的所有变异(确保没有运行更改并且状态被解锁)。

2. Pull remote state to local file system: terraform state pull > terraform.tfstate

   将远程状态terraform state pull > terraform.tfstate本地文件系统： terraform state pull > terraform.tfstate

3. Comment out (or remove) current backend configuration (lines 18–23)

   注释掉(或删除)当前的backend配置(第18-23行)

4. Uncomment the Terraform Cloud backend configuration (lines 11–16)

   取消注释Terraform Cloud backend配置(第11–16行)

5. Authenticate the CLI to Terraform Cloud. The simplest method is to create a ~/.terraformrc (example next)

   验证CLI到Terraform Cloud。 最简单的方法是创建~/.terraformrc (下一个示例)

6. Initialize Terraform with terraform init . The tool will ask to push the existing terraform.tfstate file to the Terraform Cloud workspace (type “yes”)

   用terraform init初始化Terraform。 该工具将要求将现有的terraform.tfstate文件推送到Terraform Cloud工作区(键入“是”)

Sample configuration to authenticate with Terraform Cloud 示例配置以通过Terraform Cloud进行身份验证

For more advanced and detailed credential authentication documentation, read through the official documentation.

有关更高级和详细的凭据身份验证文档，请阅读官方文档。

查询状态的输出值 (Query State for Output Values)

With the remote state uploaded, we can now use the API to query for values of the outputs variables. Any non-sensitive output variables can be queried via the API. For simplicity, let’s use the sample repository. We will be looking for full-bucket-name defined in the outputs.tf file.

上载远程状态后，我们现在可以使用API​​查询输出变量的值。 可以通过API查询任何非敏感输出变量。 为简单起见，让我们使用示例存储库。 我们将寻找在outputs.tf文件中定义的full-bucket-name 。

The API query is pretty straight forward, however, it is important to note that the query parameter include=outputs is required to list the values of outputs in the query response.

API查询非常简单，但是，请务必注意，需要查询参数include=outputs才能列出查询响应中的输出值。

The below code block will query the API using the user-token and workspace-id obtained above in this blog post. The output will be JSON and piped through jq to make it pretty (and to setup for jsonpath).

下面的代码块将使用此博客文章中上面获得的user-token和workspace-id查询API。 输出将是JSON，并通过jq传递给它，使其漂亮(并设置jsonpath)。

export BASE_URL="https://app.terraform.io/api/v2"
export TOKEN="xxxx--replace-with-user-token--xxxx"
export WORKSPACE_ID="xxxx-replace-with-workspace-id--xxxx"curl -s \
    --header "Authorization: Bearer ${TOKEN}" \
    --header "Content-Type: application/vnd.api+json" \
    --request GET \
    "${BASE_URL}/workspaces/${WORKSPACE_ID}/current-state-version?include=outputs | jq

Note: A respectful practice is to capture the contents to a temp file, then pipe the contents of the file into jq to avoid over calling the API service. API services are free to use, and while not expensive to maintain, being respectful of free services is always a polite thing to do!

注意：谨慎的做法是将内容捕获到临时文件中，然后将文件内容通过管道传输到jq以避免过度调用API服务。 API服务是免费使用的，并且维护成本并不昂贵，但是尊重免费服务始终是一件有礼貌的事情！

Validate that the output of the command produces a JSON response and not a 404. If a 404 or 401 are returned, authentication and/or authorization on the API need to be fixed.

验证命令的输出产生的是JSON响应，而不是404。如果返回404或401，则需要修复API上的身份验证和/或授权。

With the output, extracting the attribute.value can be performed by using this crafty jsonpath string:

通过输出，可以使用以下巧妙的jsonpath字符串来提取attribute.value ：

# shortening of the above curl callcurl -s https://...omitted...version?include=outputs > output.json
export FIELD=full-bucket-namecat output.json | jq -r ".included[].attributes | select(.name | contains(\"${FIELD}\")).value"# Output is the value of the attribute field. Empty responses are "not found"

结论 (Conclusion)

The purpose for the solution is that I was storing dynamic values in a key-value storage because I needed to use them in a CICD pipeline. While this approach works, if the infrastructure changed any, the copied value stored within the key-value storage would become stale and inaccurate, so I needed a way to query the canonical value and avoid breaking pipelines due to incorrect values.

该解决方案的目的是因为我需要在CICD管道中使用它们，所以我将动态值存储在键值存储中。 尽管此方法有效，但是如果基础架构发生任何更改，则存储在键值存储中的复制值将变得陈旧且不准确，因此我需要一种查询规范值并避免由于值不正确而中断管道的方法。

Exposing state variables derived from dynamic infrastructure can be implemented by using the Terraform Cloud API. This blog post detailed some high-level reasons why you might want to pull variables out of state files when not in a Terraform context. Additionally, (and admittedly) the most complicated process learned during this post was the jsonpath configuration. I am now a Terraform Cloud convert and while I haven’t explored much of the TFC features, I am happy that it only took me a few hours, some intuition and lots of documentation reading to hack this solution together.

可以使用Terraform Cloud API来实现从动态基础结构派生的状态变量的公开。 这篇博客文章详细介绍了一些高级原因，为什么您可能想在不在Terraform上下文中时从状态文件中拉出变量。 另外，(当然，)本文中学习到的最复杂的过程是jsonpath配置。 我现在是Terraform Cloud转换者，虽然我没有探索许多TFC功能，但我很高兴只花了几个小时，一些直觉和大量文档阅读就可以共同破解该解决方案。

The purpose for the solution is that I was storing dynamic values in a key-value storage because I needed to use them in a CICD pipeline. While this approach works, if the infrastructure changed any, the copied value stored within the key-value storage would become stale and inaccurate, so I needed a way to query the canonical value and avoid breaking pipelines due to incorrect values.

该解决方案的目的是因为我需要在CICD管道中使用它们，所以我将动态值存储在键值存储中。 尽管此方法有效，但是如果基础架构发生任何更改，则存储在键值存储中的复制值将变得陈旧且不准确，因此我需要一种查询规范值并避免由于值不正确而中断管道的方法。

翻译自: https://medium.com/@mike.ensor/query-outputs-from-terraform-cloud-state-de0cef973aba

terraform