AWS学习笔记(六)--利用CloudFormation管理AWS资源

CloudFormation是管理AWS资源的服务，利用CloudFormation可以自动化创建和配置AWS资源。

CloudFormation

CloudFormation的两个基本概念：Template和Stack。Template为文本文件，用来定义AWS资源，比如VPC、EC2、RDS等。Template支持参数化，可根据条件创建资源，以适应不同的环境。Stack是CloudFormation的基本管理单元，通过创建、更新或删除Stack来创建、更新或删除一系列资源。每个Stack包含一个Template，可以设定Template中定义的参数值。

Template

Template支持两种格式：JSON和YAML，YAML语法更简捷易读，并且支持注释。可将Template文件保存为任何扩展名，比如：.json、.yaml、.template、.txt。Template只是文本文件，可使用任何文本编辑器编辑，可使用图形工具CloudFormation Designer编辑，可使用SCM进行管理。
JSON示例

{  
  "AWSTemplateFormatVersion" : "2010-09-09",  
  "Description" : "A sample template",  
  "Resources" : {  
    "MyEC2Instance" : {  
      "Type" : "AWS::EC2::Instance",  
      "Properties" : {  
        "ImageId" : "ami-2f726546",  
        "InstanceType" : "t1.micro",  
        "KeyName" : "testkey",  
        "BlockDeviceMappings" : [  
          {  
            "DeviceName" : "/dev/sdm",  
            "Ebs" : {  
              "VolumeType" : "io1",  
              "Iops" : "200",  
              "DeleteOnTermination" : "false",  
              "VolumeSize" : "20"  
            }  
          }  
        ]  
      }  
    }  
  }  
}

YAML示例

AWSTemplateFormatVersion: "2010-09-09"  
Description: A sample template  
Resources:  
  MyEC2Instance:  
    Type: "AWS::EC2::Instance"  
    Properties:   
      ImageId: "ami-2f726546"  
      InstanceType: t1.micro  
      KeyName: testkey  
      BlockDeviceMappings:  
        -  
          DeviceName: /dev/sdm  
          Ebs:  
            VolumeType: io1  
            Iops: 200  
            DeleteOnTermination: false  
            VolumeSize: 20

模板基本知识请参考：
Template Anatomy
Learn Template Basics

Parameters
参数类型除String、Number、List、CommaDelimitedList外，还支持特定的类型，比如AWS::EC2::KeyPair::KeyName、AWS::EC2::AvailabilityZone::Name、AWS::EC2::SecurityGroup::GroupName等，用户必须指定其账户中的现有值。

在CloudFormation 控制台中，参数默认是按字母排序的，如果参数多看起来会很乱，不易查找，这就需要借助Metadata和AWS::CloudFormation::Interface来定义参数的分组和排序方式：

"Metadata" : {  
  "AWS::CloudFormation::Interface" : {  
    "ParameterGroups" : [  
      {  
        "Label" : { "default" : "Network Configuration" },  
        "Parameters" : [ "VPCID", "SubnetId", "SecurityGroupID" ]  
      },  
      {  
        "Label" : { "default":"Amazon EC2 Configuration" },  
        "Parameters" : [ "InstanceType", "KeyName" ]  
      }  
    ]  
  }  
}

cfn-init和shell
下例演示了CloudFormation中cfn-init和shell的使用。AMI使用了RHEL 7，其中包括安装cfn-init、awscli、awslogs，配置DNS、awscli、awslog等：

"AWSTemplateFormatVersion": "2010-09-09",  
  "Resources": {  
    "InstanceTest": {  
      "Type": "AWS::EC2::Instance",  
      "Metadata": {  
        "AWS::CloudFormation::Init": {  
          "configSets": {  
            "Install": [  
              "Install"  
            ]  
          },  
          "Install": {  
            "packages": {  
              "yum": {  
                "ntp": [],  
                "nfs-utils": [],  
                "rpcbind": []  
              }  
            },  
            "files": {  
              "/etc/sysconfig/network-scripts/ifcfg-eth0": {  
                "content": { "Fn::Join": ["", [  
                  "DEVICE=\"eth0\"\n",  
                  "BOOTPROTO=\"dhcp\"\n",  
                  "ONBOOT=\"yes\"\n",  
                  "TYPE=\"Ethernet\"\n",  
                  "USERCTL=\"yes\"\n",  
                  "PEERDNS=\"no\"\n",  
                  "IPV6INIT=\"no\"\n",  
                  "DNS1=\"10.184.13.14\""  
                ]]},  
                "owner": "root",  
                "group": "root"  
              },  
              "/root/.aws/config": {  
                "content": { "Fn::Join": ["", [
                  "[default]\n",  
                  "output = json\n",  
                  "region = cn-north-1"  
                ]]},  
                "mode": "000600",  
                "owner": "root",  
                "group": "root"  
              },  
              "/root/.aws/credentials": {  
                "content": { "Fn::Join": ["", [  
                  "[default]\n",  
                  "aws_access_key_id = AKIAPM6UDU6DGZPYXXXX\n",   
                  "aws_secret_access_key = CrYrR/LeiDXJWbAX3KQwFnaScDU0zGE+XXXXXXXX"  
                ]]},  
                "mode": "000600",  
                "owner": "root",  
                "group": "root"  
              },  
              "/root/.aws/awslogs.conf": {  
                "content": { "Fn::Join": ["", [  
                  "[general]\n",  
                  "state_file = /var/awslogs/state/agent-state\n",  
                  "[/var/log/messages]\n",  
                  "datetime_format = %Y-%m-%d %H:%M:%S\n",  
                  "file = /var/log/messages\n",  
                  "buffer_duration = 5000\n",  
                  "log_stream_name = server-log\n",  
                  "initial_position = start_of_file\n",  
                  "log_group_name = /var/log/messages"  
                ]]},  
                "mode": "000600",  
                "owner": "root",  
                "group": "root"  
              }  
            },
            "commands": {  
              "1-timezone": {  
                "command": "timedatectl set-timezone UTC",  
                "ignoreErrors": "true"  
              },  
              "2-groups": {  
                "command": "groupadd -g 1001 monet",  
                "ignoreErrors": "true"  
              },  
              "3-users": {  
                "command": "useradd -d /monet -g monet -u 1001 monetmanager\nchmod 755 /monet ",  
                "ignoreErrors": "true"  
              }  
            },  
            "services": {  
              "sysvinit": {  
                "ntpd": { "enabled" : "true", "ensureRunning" : "true" },  
                "rpcbind": { "enabled" : "true", "ensureRunning" : "true" },  
                "nfslock": { "enabled" : "true", "ensureRunning" : "true" }  
              }  
            }  
          }  
        }  
      },  
      "Properties": {  
        "AvailabilityZone": "cn-north-1a",  
        "DisableApiTermination": "false",  
        "EbsOptimized": "false",  
        "ImageId": "ami-3ce23651",  
        "InstanceInitiatedShutdownBehavior": "stop",  
        "InstanceType": "t2.small",  
        "KeyName": "Prod Key Pair",  
        "Monitoring": "false",  
        "PrivateIpAddress": "10.184.12.247",
        "SecurityGroupIds": [  
          {  
            "Fn::ImportValue": "prod-network-sg-private-id"  
          }  
        ],  
        "SubnetId": {  
          "Fn::ImportValue": "prod-network-subnet-private-id"  
        },  
        "BlockDeviceMappings": [  
          {  
            "DeviceName": "/dev/sda1",  
            "Ebs": {  
              "VolumeSize": "30"  
            }  
          }  
        ],  
        "Tags": [  
          {  
            "Key": "Name",  
            "Value": "test-ec2"  
          }  
        ],  
        "UserData": {  
          "Fn::Base64": {  
            "Fn::Join" : ["", [  
                "#!/bin/bash\n",  

                "# Install cfn-init\n",  
                "curl -O https://bootstrap.pypa.io/ez_setup.py\n",  
                "python ez_setup.py\n",  
                "easy_install --script-dir /opt/aws/bin https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",  

                "# Install the files and packages from the metadata\n",  
                "/opt/aws/bin/cfn-init -v ",  
                "         --stack ", { "Ref" : "AWS::StackName" },  
                "         --resource InstanceTest ",  
                "         --configsets Install ",  
                "         --region ", { "Ref" : "AWS::Region" }, "\n",  

                "# Install awscli\n",  
                "curl -O https://bootstrap.pypa.io/get-pip.py\n",  
                "python get-pip.py\n",  
                "pip install awscli\n",

                "# Install awslogs\n",  
                "curl -O https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py\n",  
                "python ./awslogs-agent-setup.py --region cn-north-1 --non-interactive --configfile=/root/.aws/awslogs.conf\n",  

                "# Delete installation files\n",  
                "rm -f get-pip.py ez_setup.py setuptools*.zip awslogs-agent-setup.py\n"  
              ]  
            ]  
          }  
        }  
      }  
    }  
  }  
}

powershell
下例演示了在Windows 2016上通过powershell安装DNS:

{  
  "AWSTemplateFormatVersion": "2010-09-09",  
  "Resources": {  
    "InstanceDNS": {  
      "Type": "AWS::EC2::Instance",  
      "Properties": {  
        "AvailabilityZone": "cn-north-1a",  
        "DisableApiTermination": "false",  
        "EbsOptimized": "false",  
        "ImageId": "ami-2797404a",  
        "InstanceInitiatedShutdownBehavior": "stop",  
        "InstanceType": "t2.small",  
        "KeyName": "Prod Key Pair",  
        "Monitoring": "false",  
        "PrivateIpAddress": "10.184.13.15",  
        "SecurityGroupIds": [  
          "sg-b0525cd5"  
        ],  
        "SubnetId": "subnet-a65208c3",  
        "BlockDeviceMappings": [  
          {  
            "DeviceName": "/dev/sda1",  
            "Ebs": {  
              "VolumeSize": "30"  
            }  
          }  
        ],  
        "UserData": { "Fn::Base64" : { "Fn::Join" : ["", [  
          "\n",  
          "Install-WindowsFeature -Name DNS -IncludeManagementTools\n",  
          "Add-DnsServerPrimaryZone -Name \"iata.org\" -ZoneFile \"iata.org.dns\"\n",  
          "Add-DnsServerResourceRecordA -Name \"test\" -ZoneName \"iata.org\" -AllowUpdateAny -IPv4Address \"10.184.12.111\"\n",  
          ""  
        ]]}},  
        "Tags": [  
          {  
            "Key": "Name",  
            "Value": "test-dns"  
          }  
        ]  
      }  
    }  
  }  
}

CloudFormation设计器

AWS CloudFormation Console提供图形化的Template设计器，支持拖拽操作，支持自动完成功能。
使用CloudFormation Designer的拖拽功能，或用其打开已有的Template会增加组件位置信息，这些信息在实际应用中是没有必要的，也增加了Template的复杂度，常只利用其自动完成功能，当编辑已有的Template时，使用Copy/Paste的方式。

快捷键
【Ctrl+Shift+Space】 显示提示列表
【Ctrl+F】 搜索
【Ctrl+\】 格式化Template(会删除所有空行)
【Ctrl+Shift+\】 删除所有空格

Template样例

CloudFormation提供了大量的示例模板Sample Templates可供学习，其中China Region的示例较少，推荐使用US West (Northern California) Region。比如Services中下面的示例，全面演示了从创建VPC到EC2。
Creates a VPC and adds an Amazon EC2 instance with an Elastic IP address and a security group

CloudFormer

CloudFormer是模板创建工具，可从现有AWS资源创建CloudFormation Template，用来作为初始Template，是学习Template语法结构的好工具。

CloudFormer使用很简单，它是一个CloudFormation Stack，可从CloudFormation Console创建，创建后在一个EC2实例上运行，不需其他资源。

创建CloudFormer
登录CloudFormation Console > Create New Stack > Select a sample template > CloudFormer > Next > 输入Stack name、Username、Password > 根据提示信息完成后续步骤

启动CloudFormer
点击CloudFormer Stack选项卡Outputs中的链接，输入用户名，密码即可启动CloudFormer。然后Select the AWS Region > Create Template > 一步步的选择要创建模板的资源。

注：如出现错误NoMethodError in TemplatesController，可能是资源配置有问题，试着减少一些资源。

最佳实践

• 按生命周期和所有权组织堆栈，而不是将所有资源放在一个Stack内，避免资源变化时互相影响。另外，可以根据层次结构和面向的服务（SOA）划分资源。
• 使用跨堆栈引用来导出共享资源,其他堆栈使用 Fn::ImportValue函数调用导出的资源。引用另一个 AWS CloudFormation 堆栈中的资源输出
• 重复使用模板以在多个环境中复制堆栈，要使模板可重用，可使用参数、映射和条件，以便能在创建堆栈时对其进行自定义。
• 使用嵌套堆栈来重复使用常见模板，要创建嵌套堆栈，可使用模板中的AWS::CloudFormation::Stack资源来引用其他模板。

CloudFormation CLI

Creating a Stack

You must provide the stack name, the location of a valid template, and any input parameters. If you specify a local template file, AWS CloudFormation uploads it to an Amazon S3 bucket in your AWS account.

$ aws cloudformation create-stack --stack-name myteststack --template-body file:///home/testuser/mytemplate.json --parameters ParameterKey=Parm1,ParameterValue=test1 ParameterKey=Parm2,ParameterValue=test2

Listing Your Stacks

Note The aws cloudformation list-stacks command returns information on deleted stacks for 90 days after they have been deleted.

$ aws cloudformation list-stacks --stack-status-filter CREATE_COMPLETE

Describing Your Stacks

$ aws cloudformation describe-stacks --stack-name myteststack

By default, aws cloudformation describe-stacks returns parameter values. To prevent sensitive parameter values such as passwords from being returned, include a NoEcho property set to TRUE in your AWS CloudFormation template.

Viewing Stack Event History

You can track the status of the resources AWS CloudFormation is creating and deleting with the aws cloudformation describe-stack-events command. The amount of time to create or delete a stack depends on the complexity of your stack.

$ aws cloudformation describe-stack-events --stack-name myteststack

Listing Stack Resources

$ aws cloudformation list-stack-resources --stack-name myteststack

Retrieving a Template

AWS CloudFormation stores the template you use to create your stack as part of the stack.

$ aws cloudformation get-template --stack-name myteststack

Validating a Template

You can validate templates locally by using the --template-body parameter, or remotely with the --template-url parameter.

$ aws cloudformation validate-template --template-url https://s3.amazonaws.com/cloudformation-templates-us-east-1/S3_Bucket.template
$ aws cloudformation validate-template --template-body file:///home/local/test/sampletemplate.json

Deleting a Stack

$ aws cloudformation delete-stack --stack-name myteststack

参考文档

Getting Started with AWS CloudFormation
Template Anatomy
Learn Template Basics
AWS CloudFormation Templates
AWS Resource Types Reference
Using CloudFormer to Create AWS CloudFormation Templates from Existing AWS Resources
Using the AWS Command Line Interface
AWS CloudFormation Best Practices
Install roles, role services, and features by using Windows PowerShell cmdlets
Domain Name System (DNS) Server Cmdlets