AWS实战:ECS Fargate部署nodejs docker image

项目地址

https://github.com/JessicaWin/aws-fargate-example

项目架构图

AWS实战:ECS Fargate部署nodejs docker image_第1张图片

1.构建nodejs项目

$ sudo npm i -g @nestjs/cli
$ nest new aws-fargate-example
$ cd aws-fargate-example
$ npm install
$ npm run start

项目启动成功后,浏览器输入http://localhost:3000/,可以看到Hello World!

2.构建docker image并启动

创建Dockerfile:

#use alpine version to decrease docker image size, https://hub.docker.com/_/node?tab=description&page=1&ordering=last_updated
FROM node:18-alpine3.14 As development

WORKDIR /app

COPY package*.json ./

RUN npm install

#copies files from a local source location to a destination in the Docker container
COPY . .

RUN npm run build

CMD ["npm", "run", "start"]

# Multi Staging Build
FROM node:18-alpine3.14 As production

ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}

WORKDIR /app

COPY package*.json ./

# Create node_modules that is necessary only for production
RUN npm install production

#copies files from a local source location to a destination in the Docker container
COPY . .

# Copy dist generated in development stage
COPY --from=development /app/dist ./dist

EXPOSE 3000

CMD ["npm", "run", "start:prod"]

指令含义:

  • FROM :指定 基础镜像,因此一个 DockerfileFROM 是必备的指令,并且必须是第一条指令
  • WORKDIR:制定后续RUNCMDENTRYPOINTCOPY 和 ADD指令的工作目录
  • COPY :将从构建上下文目录中 <源路径> 的文件/目录复制到新的一层的镜像内的 <目标路径> 位置。
  • RUN :执行命令行命令
  • CMD:容器启动命令
  • ARG:构建参数
  • ENV:设置环境变量
  • EXPOSE:声明容器运行时提供服务的端口,这只是一个声明,在容器运行时并不会因为这个声明应用就会开启这个端口的服务

构建和启动docker

$ docker build -t aws-fargate-example:latest --target=development ./
$ docker run -it -d -p 3001:3000 aws-fargate-example:latest

docker启动成功后,浏览器输入http://localhost:3001/,可以看到Hello World!

3.使用docker compose管理容器

Compose 是用于定义和运行多容器 Docker 应用程序的工具。通过 Compose,可以使用 YML 文件来配置应用程序需要的所有服务。然后,使用一个命令,就可以从 YML 文件配置中创建并启动所有服务。

Compose 使用的三个步骤:

  • 使用 Dockerfile 定义应用程序的环境。

  • 使用 docker-compose.yml 定义构成应用程序的服务,这样它们可以在隔离环境中一起运行。

  • 最后,执行 docker-compose up 命令来启动并运行整个应用程序。

# set based on docker version, refer to https://docs.docker.com/compose/compose-file/compose-file-v3/
version: '3.6'

services:
    dev:
        container_name: aws-fargate-example-dev
        image: aws-fargate-example-dev:${IMAGE_TAG:-latest}
        build:
            context: .
            target: development
            dockerfile: ./Dockerfile
        command: npm run start
        ports:
            - 3000:3000
        networks:
            - aws-fargate-example-network
        restart: unless-stopped
    prod:
        container_name: aws-fargate-example-prod
        image: aws-fargate-example-prod:${IMAGE_TAG:-latest}
        build:
            context: .
            target: production
            dockerfile: ./Dockerfile
        command: npm run start:prod
        ports:
            - 3000:3000
        networks:
            - aws-fargate-example-network
        restart: unless-stopped

networks:
    aws-fargate-example-network:

构建和启动docker

# use ocker-compose to build image for all services/stages
$ docker-compose build
# use ocker-compose to start specific service
$ docker-compose up dev
$ docker-compose up prod

4.部署deploy bucket

在部署ecr之前,我们首先部署一个bucket用作以后各个aws资源的deploy bucket

deploy bucket cloudformation template

AWSTemplateFormatVersion: '2010-09-09'
Description: Cloudformation template for creating deployment related S3 buckets
Parameters:
  Stage:
    Type: String
    Default: develop
    Description: the stage of the environment like develop, production

Resources:
  DeploymentBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub com.jessica.${Stage}-deploy-bucket
      CorsConfiguration:
        CorsRules:
          - AllowedHeaders: ['*']
            AllowedMethods: [GET, PUT, HEAD, POST, DELETE]
            AllowedOrigins: ['*']
Outputs:
  DeploymentBucket:
    Value: !Ref DeploymentBucket
    Export:
      Name: !Sub ${Stage}-deploy-bucket

deploy bucket 部署脚本

#!/bin/bash
echo "Deploying deploy bucket ..."
DELETE=$1

STAGE_LIST=( develop production )

for i in "${STAGE_LIST[@]}"
do
	STAGE=$i
	REGION="ap-northeast-1"
	if [ "$STAGE" = "develop" ];then
		REGION="ap-southeast-1"
	fi
	echo "Deploying deploy-bucket.yml to $STAGE in $REGION ..."
	if [ "$DELETE" = "remove" ];then
		aws cloudformation delete-stack --stack-name $STAGE-deploy-bucket
	else
		stackOutput=`aws cloudformation create-stack --stack-name $STAGE-deploy-bucket --template-body file://./deploy-bucket.yml --parameters ParameterKey=Stage,ParameterValue=$STAGE --region $REGION 2>&1`
        if [[ "$stackOutput" =~ "AlreadyExistsException" ]]; then
			noUpdate=`aws cloudformation update-stack --stack-name $STAGE-deploy-bucket --template-body file://./deploy-bucket.yml --parameters ParameterKey=Stage,ParameterValue=$STAGE --region $REGION 2>&1`
			if [[ "$noUpdate" =~ "No updates are to be performed" ]]; then
				echo "No updates are to be performed"
			else
				echo $noUpdate
			fi
		else
			echo $stackOutput
		fi
	fi
done

5.部署ECR repository

部署ecr repository用来存储docker image

ecr cloudformation template

service: aws-fargate-example-ecr

provider:
  name: aws
  region: ${opt:region, 'ap-southeast-1'}
  stage: ${opt:stage, 'develop'}
  stackName: ${self:provider.stage}-${self:service}
  deploymentBucket:
    name: com.jessica.${self:provider.stage}-deploy-bucket
    serverSideEncryption: AES256
resources:
  Resources:
    NestjsstarterEcr:
      Type: AWS::ECR::Repository
      Properties:
        RepositoryName: aws-fargate-example
        RepositoryPolicyText:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Principal:
                AWS:
                  - !Sub arn:aws:iam::${AWS::AccountId}:root
              Action:
                - 'ecr:ListImages'
                - 'ecr:BatchGetImage'
                - 'ecr:GetDownloadUrlForLayer'
                - 'ecr:PutImage'

ecr部署命令:

# deploy to develop stage
$ sls deploy --stage develop --region ap-southeast-1

# deploy to production stage
$ sls deploy --stage production --region ap-norheast-1

6.部署IAM Role

部署ECS Fargate Service至少需要两个role:

  • Task Role:ECS task执行时使用的role,需要的权限根据task
  • ExecutionRole:ECS service启动task使用的role,需要的权限

此外,因为本项目中的fargate service用到了auto scale,所以额外需要一个auto scale role

service: aws-fargate-example-iam

provider:
  name: aws
  region: ${opt:region, 'ap-southeast-1'}
  stage: ${opt:stage, 'develop'}
  stackName: ${self:provider.stage}-${self:service}
  deploymentBucket:
    name: name: com.jessica.${self:provider.stage}-deploy-bucket
    serverSideEncryption: AES256
resources:
  Resources:
    ECSPolicy:
      Type: AWS::IAM::ManagedPolicy
      Properties:
        ManagedPolicyName: !Sub '${self:provider.stage}_ECSPolicy'
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Action:
                - ecr:BatchGetImage
                - ecr:BatchCheckLayerAvailability
                - ecr:CompleteLayerUpload
                - ecr:GetDownloadUrlForLayer
                - ecr:InitiateLayerUpload
                - ecr:PutImage
                - ecr:UploadLayerPart
                - ecr:GetAuthorizationToken
              Resource:
                - '*'
              Effect: Allow
            - Action:
                - ecs:*
                - elasticloadbalancing:DescribeTargetGroups
                - elasticloadbalancing:DescribeListeners
                - elasticloadbalancing:ModifyListener
                - elasticloadbalancing:DescribeRules
                - elasticloadbalancing:ModifyRule
                - lambda:InvokeFunction
                - cloudwatch:DescribeAlarms
                - sns:Publish
                - s3:GetObject
                - s3:GetObjectVersion
              Resource:
                - '*'
              Effect: Allow
            - Action:
                - iam:PassRole
              Resource:
                - !Sub 'arn:aws:iam::*:role/${self:provider.stage}_ECSTaskRole'
              Effect: Allow

    ECSTaskPolicy:
      Type: AWS::IAM::ManagedPolicy
      Properties:
        ManagedPolicyName: !Sub '${self:provider.stage}_ECSTaskPolicy'
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Action:
                - kinesis:*
              Resource:
                - '*'
              Effect: Allow

    ECSTaskRole:
      Type: AWS::IAM::Role
      Properties:
        RoleName: !Sub '${self:provider.stage}_ECSTaskRole'
        AssumeRolePolicyDocument:
          Statement:
            - Effect: Allow
              Action: sts:AssumeRole
              Principal:
                Service:
                  - ecs-tasks.amazonaws.com
        ManagedPolicyArns:
          - !Ref ECSTaskPolicy
          - !Ref ECSPolicy

    ECSExecuteRole:
      Type: AWS::IAM::Role
      Properties:
        RoleName: !Sub '${self:provider.stage}_ECSExecuteRole'
        AssumeRolePolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Principal:
                Service: ecs-tasks.amazonaws.com
              Action: 'sts:AssumeRole'
        ManagedPolicyArns:
          - 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'

    ECSAutoScalingRole:
      Type: AWS::IAM::Role
      Properties:
        RoleName: !Sub '${self:provider.stage}_ECSAutoScalingRole'
        AssumeRolePolicyDocument:
          Statement:
            - Effect: Allow
              Principal:
                Service: ecs-tasks.amazonaws.com
              Action: 'sts:AssumeRole'
        ManagedPolicyArns:
          - 'arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceAutoscaleRole'
  Outputs:
    ECSExecuteRole:
      Value: !GetAtt ECSExecuteRole.Arn
      Export:
        Name: ${self:provider.stage}-ECSExecuteRole
    ECSTaskRole:
      Value: !GetAtt ECSTaskRole.Arn
      Export:
        Name: ${self:provider.stage}-ECSTaskRole
    ECSAutoScalingRole:
      Value: !GetAtt ECSAutoScalingRole.Arn
      Export:
        Name: ${self:provider.stage}-ECSAutoScalingRole

 iam部署命令:

# deploy to develop stage
$ sls deploy --stage develop --region ap-southeast-1

# deploy to production stage
$ sls deploy --stage production --region ap-norheast-1

7.部署VPC

本项目中fargate task的NetworkMode使用awsvpc,因此我们需要先部署一个vpc,vpc中包含的资源如下:

  • 三个public subnet
  • 三个private subnet
  • 一个Internet gateway
  • 一个EIP
  • 一个Nat gateway
  • 一个public security group
  • 一个private security group
  • 一个public network access control list
  • 一个private network access control list
  • 一个public route table
  • 一个private route table

vpc cloudformation template

service: aws-fargate-example-vpc

custom:
  classB: ${opt:classB, '10'}

provider:
  name: aws
  region: ${opt:region, 'ap-southeast-1'}
  stage: ${opt:stage, 'develop'}
  stackName: ${self:provider.stage}-${self:service}
  deploymentBucket:
    name: com.jessica.${self:provider.stage}-deploy-bucket
    serverSideEncryption: AES256

resources:
  Resources:
    VPC:
      Type: 'AWS::EC2::VPC'
      Properties:
        CidrBlock: !Sub '10.${self:custom.classB}.0.0/16'
        EnableDnsSupport: true
        EnableDnsHostnames: true
        InstanceTenancy: default
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}'
    # Internet Gateway
    InternetGateway:
      Type: 'AWS::EC2::InternetGateway'
      Properties:
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_InternetGateway'
    VPCGatewayAttachment:
      Type: 'AWS::EC2::VPCGatewayAttachment'
      Properties:
        VpcId: !Ref VPC
        InternetGatewayId: !Ref InternetGateway

    # public security group
    PublicSecurityGroup:
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: Allow http to client host
        VpcId: !Ref VPC
        SecurityGroupIngress:
          - IpProtocol: tcp
            FromPort: 80
            ToPort: 80
            CidrIp: 0.0.0.0/0
          - IpProtocol: tcp
            FromPort: 443
            ToPort: 443
            CidrIp: 0.0.0.0/0
          - IpProtocol: tcp
            FromPort: 3000
            ToPort: 3000
            CidrIp: 0.0.0.0/0
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_PublicSecurityGroup'
    # private security group
    PrivateSecurityGroup:
      Type: AWS::EC2::SecurityGroup
      Properties:
        GroupDescription: Allow access frm public securty group
        VpcId: !Ref VPC
        SecurityGroupIngress:
          - IpProtocol: tcp
            FromPort: 3000
            ToPort: 3000
            SourceSecurityGroupId: !Ref PublicSecurityGroup
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_PrivateSecurityGroup'
    # public route table
    RouteTablePublic:
      Type: 'AWS::EC2::RouteTable'
      Properties:
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_RouteTablePublic'
    RouteTablePublicInternetRoute:
      Type: 'AWS::EC2::Route'
      DependsOn: VPCGatewayAttachment
      Properties:
        RouteTableId: !Ref RouteTablePublic
        DestinationCidrBlock: '0.0.0.0/0'
        GatewayId: !Ref InternetGateway

    # public subnet
    SubnetAPublic:
      Type: 'AWS::EC2::Subnet'
      Properties:
        AvailabilityZone: !Select [0, !GetAZs '']
        CidrBlock: !Sub '10.${self:custom.classB}.0.0/20'
        MapPublicIpOnLaunch: true
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_SubnetAPublic'
    RouteTableAssociationAPublic:
      Type: 'AWS::EC2::SubnetRouteTableAssociation'
      Properties:
        SubnetId: !Ref SubnetAPublic
        RouteTableId: !Ref RouteTablePublic

    SubnetBPublic:
      Type: 'AWS::EC2::Subnet'
      Properties:
        AvailabilityZone: !Select [1, !GetAZs '']
        CidrBlock: !Sub '10.${self:custom.classB}.32.0/20'
        MapPublicIpOnLaunch: true
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_SubnetBPublic'
    RouteTableAssociationBPublic:
      Type: 'AWS::EC2::SubnetRouteTableAssociation'
      Properties:
        SubnetId: !Ref SubnetBPublic
        RouteTableId: !Ref RouteTablePublic

    SubnetCPublic:
      Type: 'AWS::EC2::Subnet'
      Properties:
        AvailabilityZone: !Select [2, !GetAZs '']
        CidrBlock: !Sub '10.${self:custom.classB}.64.0/20'
        MapPublicIpOnLaunch: true
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_SubnetCPublic'
    RouteTableAssociationCPublic:
      Type: 'AWS::EC2::SubnetRouteTableAssociation'
      Properties:
        SubnetId: !Ref SubnetCPublic
        RouteTableId: !Ref RouteTablePublic

    # nat gateway
    EIP:
      Type: 'AWS::EC2::EIP'
      Properties:
        Domain: vpc
    NatGateway:
      Type: 'AWS::EC2::NatGateway'
      Properties:
        AllocationId: !GetAtt 'EIP.AllocationId'
        SubnetId: !Ref SubnetAPublic

    # private route table
    RouteTablePrivate:
      Type: 'AWS::EC2::RouteTable'
      Properties:
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_RouteTablePrivate'
    RouteTablePrivateRoute:
      Type: 'AWS::EC2::Route'
      Properties:
        RouteTableId: !Ref RouteTablePrivate
        DestinationCidrBlock: '0.0.0.0/0'
        NatGatewayId: !Ref NatGateway

    # private subnet
    SubnetAPrivate:
      Type: 'AWS::EC2::Subnet'
      Properties:
        AvailabilityZone: !Select [0, !GetAZs '']
        CidrBlock: !Sub '10.${self:custom.classB}.16.0/20'
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_SubnetAPrivate'
    RouteTableAssociationAPrivate:
      Type: 'AWS::EC2::SubnetRouteTableAssociation'
      Properties:
        SubnetId: !Ref SubnetAPrivate
        RouteTableId: !Ref RouteTablePrivate

    SubnetBPrivate:
      Type: 'AWS::EC2::Subnet'
      Properties:
        AvailabilityZone: !Select [1, !GetAZs '']
        CidrBlock: !Sub '10.${self:custom.classB}.48.0/20'
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_SubnetBPrivate'
    RouteTableAssociationBPrivate:
      Type: 'AWS::EC2::SubnetRouteTableAssociation'
      Properties:
        SubnetId: !Ref SubnetBPrivate
        RouteTableId: !Ref RouteTablePrivate

    SubnetCPrivate:
      Type: 'AWS::EC2::Subnet'
      Properties:
        AvailabilityZone: !Select [2, !GetAZs '']
        CidrBlock: !Sub '10.${self:custom.classB}.80.0/20'
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_SubnetCPrivate'
    RouteTableAssociationCPrivate:
      Type: 'AWS::EC2::SubnetRouteTableAssociation'
      Properties:
        SubnetId: !Ref SubnetCPrivate
        RouteTableId: !Ref RouteTablePrivate

    # public subnet network access control list
    NetworkAclPublic:
      Type: 'AWS::EC2::NetworkAcl'
      Properties:
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_NetworkAclPublic'
    NetworkAclEntryInPublicAllowAll:
      Type: 'AWS::EC2::NetworkAclEntry'
      Properties:
        NetworkAclId: !Ref NetworkAclPublic
        RuleNumber: 99
        Protocol: -1
        RuleAction: allow
        Egress: false
        CidrBlock: '0.0.0.0/0'
    NetworkAclEntryOutPublicAllowAll:
      Type: 'AWS::EC2::NetworkAclEntry'
      Properties:
        NetworkAclId: !Ref NetworkAclPublic
        RuleNumber: 99
        Protocol: -1
        RuleAction: allow
        Egress: true
        CidrBlock: '0.0.0.0/0'
    SubnetNetworkAclAssociationAPublic:
      Type: 'AWS::EC2::SubnetNetworkAclAssociation'
      Properties:
        SubnetId: !Ref SubnetAPublic
        NetworkAclId: !Ref NetworkAclPublic
    SubnetNetworkAclAssociationBPublic:
      Type: 'AWS::EC2::SubnetNetworkAclAssociation'
      Properties:
        SubnetId: !Ref SubnetBPublic
        NetworkAclId: !Ref NetworkAclPublic
    SubnetNetworkAclAssociationCPublic:
      Type: 'AWS::EC2::SubnetNetworkAclAssociation'
      Properties:
        SubnetId: !Ref SubnetCPublic
        NetworkAclId: !Ref NetworkAclPublic

    # private subnet network access control list
    NetworkAclPrivate:
      Type: 'AWS::EC2::NetworkAcl'
      Properties:
        VpcId: !Ref VPC
        Tags:
          - Key: Name
            Value: !Sub 'VPC_${self:custom.classB}_NetworkAclPrivate'
    NetworkAclEntryInPrivateAllowVPC:
      Type: 'AWS::EC2::NetworkAclEntry'
      Properties:
        NetworkAclId: !Ref NetworkAclPrivate
        RuleNumber: 99
        Protocol: -1
        RuleAction: allow
        Egress: false
        CidrBlock: '0.0.0.0/0'
    NetworkAclEntryOutPrivateAllowAll:
      Type: 'AWS::EC2::NetworkAclEntry'
      Properties:
        NetworkAclId: !Ref NetworkAclPrivate
        RuleNumber: 99
        Protocol: -1
        RuleAction: allow
        Egress: true
        CidrBlock: '0.0.0.0/0'
    SubnetNetworkAclAssociationBPrivate:
      Type: 'AWS::EC2::SubnetNetworkAclAssociation'
      Properties:
        SubnetId: !Ref SubnetBPrivate
        NetworkAclId: !Ref NetworkAclPrivate
    SubnetNetworkAclAssociationAPrivate:
      Type: 'AWS::EC2::SubnetNetworkAclAssociation'
      Properties:
        SubnetId: !Ref SubnetAPrivate
        NetworkAclId: !Ref NetworkAclPrivate
    SubnetNetworkAclAssociationCPrivate:
      Type: 'AWS::EC2::SubnetNetworkAclAssociation'
      Properties:
        SubnetId: !Ref SubnetCPrivate
        NetworkAclId: !Ref NetworkAclPrivate

  Outputs:
    VPC:
      Description: 'VPC.'
      Value: !Ref VPC
      Export:
        Name: !Sub '${self:provider.stackName}'
    SubnetsPublic:
      Description: 'Subnets public.'
      Value:
        !Join [
          ',',
          [!Ref SubnetAPublic, !Ref SubnetBPublic, !Ref SubnetCPublic],
        ]
      Export:
        Name: !Sub '${self:provider.stackName}-PublicSubnets'
    SubnetsPrivate:
      Description: 'Subnets private.'
      Value:
        !Join [
          ',',
          [!Ref SubnetAPrivate, !Ref SubnetBPrivate, !Ref SubnetCPrivate],
        ]
      Export:
        Name: !Sub '${self:provider.stackName}-PrivateSubnets'
    DefaultSecurityGroup:
      Description: 'VPC Default Security Group'
      Value: !GetAtt VPC.DefaultSecurityGroup
      Export:
        Name: !Sub '${self:provider.stackName}-DefaultSecurityGroup'
    PublicSecurityGroup:
      Description: 'The id of the PublicSecurityGroup'
      Value: !Ref PublicSecurityGroup
      Export:
        Name: !Sub '${self:provider.stackName}-PublicSecurityGroup'
    PrivateSecurityGroup:
      Description: 'The id of the PrivateSecurityGroup'
      Value: !Ref PrivateSecurityGroup
      Export:
        Name: !Sub '${self:provider.stackName}-PrivateSecurityGroup'

 vpc部署命令:

# deploy to develop stage
$ sls deploy --stage develop --region ap-southeast-1

# deploy to production stage
$ sls deploy --stage production --region ap-norheast-1

8.项目添加health check代码

本项目中会使用application load balancer对服务进行负载均衡,需要一个path来进行服务健康检测。

health.controller.ts

import { Controller, Get, HttpCode } from '@nestjs/common';
import {
  DiskHealthIndicator,
  HealthCheck,
  HealthCheckService,
  HttpHealthIndicator,
  MemoryHealthIndicator,
} from '@nestjs/terminus';

@Controller('health')
export class HealthController {
  constructor(
    private health: HealthCheckService,
    private http: HttpHealthIndicator,
    private memory: MemoryHealthIndicator,
    private disk: DiskHealthIndicator,
  ) {}

  @Get('success')
  @HttpCode(200)
  @HealthCheck()
  ping() {
    return 'ok!';
  }

  @Get('error')
  @HttpCode(500)
  @HealthCheck()
  pingError() {
    return 'error';
  }

  @Get('check')
  @HealthCheck()
  checkHttp() {
    return this.health.check([
      async () =>
        this.http.pingCheck('sample', 'http://localhost:3000/health/success'),
      async () => this.memory.checkHeap('memory_heap', 500 * 1024 * 1024),
      async () => this.memory.checkRSS('memory_rss', 3000 * 1024 * 1024),
      async () =>
        this.disk.checkStorage('storage', { thresholdPercent: 0.9, path: '/' }),
    ]);
  }
}

9.构建并上传docker image到ECR

# replace ${AWS_AccountId} with your own aws accountId 
# Retrieve an authentication token and authenticate your Docker client to your registry.
$ aws ecr get-login-password --region ap-southeast-1 | docker login --username AWS --password-stdin ${AWS_AccountId}.dkr.ecr.ap-southeast-1.amazonaws.com

# Build your Docker image using the following command. For information on building a Docker file from scratch see the instructions here . You can skip this step if your image is already built:
$ docker-compose build dev

#After the build completes, tag your image so you can push the image to this repository:
$ docker tag aws-fargate-example:latest ${AWS_AccountId}.dkr.ecr.ap-southeast-1.amazonaws.com/aws-fargate-example:latest 

#Run the following command to push this image to your newly created AWS repository:
$ docker push ${AWS_AccountId}.dkr.ecr.ap-southeast-1.amazonaws.com/aws-fargate-example:latest

10.部署ECS Fargate Service

本项目中ECS Fargate Service包含的资源如下:

  • Application Load Balancer
    • Target Group
    • Http Listerner
    • Load Balance
  • Auto Scaling
    • Scaling Target
    • Scaling Policy
  • ECS Cluster
  • ECS Service
  • ECS Task Definition
  • ECS Task Log Group

ecs cloudformation template

service: aws-fargate-example-ecs

provider:
  name: aws
  region: ${opt:region, 'ap-southeast-1'}
  stage: ${opt:stage, 'develop'}
  stackName: ${self:provider.stage}-${self:service}
  deploymentBucket:
    name: com.jessica.${self:provider.stage}-deploy-bucket
    serverSideEncryption: AES256
resources:
  Parameters:
    ServiceName:
      Type: String
      Default: aws-fargate-example
    ContainerPort:
      Type: Number
      Default: 3000
    LoadBalancerPort:
      Type: Number
      Default: 80
    HealthCheckPath:
      Type: String
      Default: /health/check
    # for autoscaling
    MinCapacity:
      Type: Number
      Default: 3
    # for autoscaling
    MaxCapacity:
      Type: Number
      Default: 6
    DesiredCount:
      Type: Number
      Default: 3
    # target CPU utilization (%)
    AutoScalingTargetValue:
      Type: Number
      Default: 50
    VPCName:
      Type: String
      Default: aws-fargate-example-vpc

  Resources:
    LogGroup:
      Type: AWS::Logs::LogGroup
      Properties:
        LogGroupName: !Join ['', [/ecs/, !Ref ServiceName, task]]
    NestjsFargateCluster:
      Type: 'AWS::ECS::Cluster'
      Properties:
        ClusterName: !Join ['-', [!Ref ServiceName, cluster]]
        CapacityProviders:
          - FARGATE
          - FARGATE_SPOT
        DefaultCapacityProviderStrategy:
          - CapacityProvider: FARGATE
            Weight: 1
          - CapacityProvider: FARGATE_SPOT
            Weight: 1
        Configuration:
          ExecuteCommandConfiguration:
            Logging: DEFAULT
        ClusterSettings:
          - Name: containerInsights
            Value: enabled
    NestjsFargateTaskDefinition:
      Type: AWS::ECS::TaskDefinition
      # Makes sure the log group is created before it is used.
      DependsOn: LogGroup
      Properties:
        # Name of the task definition. Subsequent versions of the task definition are grouped together under this name.
        Family: !Join ['-', [!Ref ServiceName, task-definition]]
        # awsvpc is required for Fargate
        NetworkMode: awsvpc
        RequiresCompatibilities:
          - FARGATE
        Cpu: 256
        # Between 8GB and 30GB in 1GB increments - Available cpu values: 4096 (4 vCPU)
        Memory: 512
        # A role needed by ECS.
        # "The ARN of the task execution role that containers in this task can assume. All containers in this task are granted the permissions that are specified in this role."
        # "There is an optional task execution IAM role that you can specify with Fargate to allow your Fargate tasks to make API calls to Amazon ECR."
        ExecutionRoleArn:
          Fn::ImportValue: ${self:provider.stage}-ECSExecuteRole
        # "The Amazon Resource Name (ARN) of an AWS Identity and Access Management (IAM) role that grants containers in the task permission to call AWS APIs on your behalf."
        TaskRoleArn:
          Fn::ImportValue: ${self:provider.stage}-ECSTaskRole
        ContainerDefinitions:
          - Name: !Ref ServiceName
            Image: !Sub ${AWS::AccountId}.dkr.ecr.${self:provider.region}.amazonaws.com/aws-fargate-example:latest
            PortMappings:
              - ContainerPort: !Ref ContainerPort
            # Send logs to CloudWatch Logs
            LogConfiguration:
              LogDriver: awslogs
              Options:
                awslogs-region: ${self:provider.region}
                awslogs-group: !Ref LogGroup
                awslogs-stream-prefix: ecs
    NestjsFargateService:
      Type: AWS::ECS::Service
      DependsOn:
        - ListenerHTTP
      Properties:
        ServiceName: !Ref ServiceName
        Cluster: !Ref NestjsFargateCluster
        TaskDefinition: !Ref NestjsFargateTaskDefinition
        DeploymentConfiguration:
          MinimumHealthyPercent: 100
          MaximumPercent: 200
        DesiredCount: !Ref DesiredCount
        # This may need to be adjusted if the container takes a while to start up
        HealthCheckGracePeriodSeconds: 30
        LaunchType: FARGATE
        NetworkConfiguration:
          AwsvpcConfiguration:
            # change to DISABLED if you're using private subnets that have access to a NAT gateway
            AssignPublicIp: DISABLED
            Subnets:
              Fn::Split:
                - ','
                - Fn::ImportValue: !Sub ${self:provider.stage}-${VPCName}-PrivateSubnets
            SecurityGroups:
              - Fn::ImportValue: !Sub ${self:provider.stage}-${VPCName}-PrivateSecurityGroup
        LoadBalancers:
          - ContainerName: !Ref ServiceName
            ContainerPort: !Ref ContainerPort
            TargetGroupArn: !Ref TargetGroup
    TargetGroup:
      Type: AWS::ElasticLoadBalancingV2::TargetGroup
      Properties:
        HealthCheckIntervalSeconds: 10
        # will look for a 200 status code by default unless specified otherwise
        HealthCheckPath: !Ref HealthCheckPath
        HealthCheckTimeoutSeconds: 5
        UnhealthyThresholdCount: 2
        HealthyThresholdCount: 2
        Name: !Join ['-', [!Ref ServiceName, target-group]]
        Port: !Ref ContainerPort
        Protocol: HTTP
        TargetGroupAttributes:
          - Key: deregistration_delay.timeout_seconds
            Value: 60 # default is 300
        TargetType: ip
        VpcId:
          Fn::ImportValue: !Sub ${self:provider.stage}-${VPCName}
    ListenerHTTP:
      Type: AWS::ElasticLoadBalancingV2::Listener
      Properties:
        DefaultActions:
          - TargetGroupArn: !Ref TargetGroup
            Type: forward
        LoadBalancerArn: !Ref LoadBalancer
        Port: !Ref LoadBalancerPort
        Protocol: HTTP
        # Certificates:
        #   - CertificateArn: !Ref Certificate
    LoadBalancer:
      Type: AWS::ElasticLoadBalancingV2::LoadBalancer
      Properties:
        LoadBalancerAttributes:
          # this is the default, but is specified here in case it needs to be changed
          - Key: idle_timeout.timeout_seconds
            Value: 60
        Name: !Join ['-', [!Ref ServiceName, balancer]]
        # "internal" is also an option
        Scheme: internet-facing
        SecurityGroups:
          - Fn::ImportValue: !Sub ${self:provider.stage}-${VPCName}-PublicSecurityGroup
        Subnets:
          Fn::Split:
            - ','
            - Fn::ImportValue: !Sub ${self:provider.stage}-${VPCName}-PublicSubnets

    NestjsFargateAutoScalingTarget:
      Type: AWS::ApplicationAutoScaling::ScalableTarget
      Properties:
        MinCapacity: !Ref MinCapacity
        MaxCapacity: !Ref MaxCapacity
        ResourceId:
          !Join [
            '/',
            [
              service,
              !Ref NestjsFargateCluster,
              !GetAtt NestjsFargateService.Name,
            ],
          ]
        ScalableDimension: ecs:service:DesiredCount
        ServiceNamespace: ecs
        # "The Amazon Resource Name (ARN) of an AWS Identity and Access Management (IAM) role that allows Application Auto Scaling to modify your scalable target."
        RoleARN:
          Fn::ImportValue: ${self:provider.stage}-ECSAutoScalingRole
    NestjsFargateAutoScalingPolicy:
      Type: AWS::ApplicationAutoScaling::ScalingPolicy
      Properties:
        PolicyName: ${self:provider.stage}-NestjsFargateAutoScalingPolicy
        PolicyType: TargetTrackingScaling
        ScalingTargetId: !Ref NestjsFargateAutoScalingTarget
        TargetTrackingScalingPolicyConfiguration:
          PredefinedMetricSpecification:
            PredefinedMetricType: ECSServiceAverageCPUUtilization
          ScaleInCooldown: 10
          ScaleOutCooldown: 10
          # Keep things at or lower than 50% CPU utilization, for example
          TargetValue: !Ref AutoScalingTargetValue

 部署命令:

# deploy to develop stage
$ sls deploy --stage develop --region ap-southeast-1

# deploy to production stage
$ sls deploy --stage production --region ap-norheast-1

11.配置github OpenID Connect (OIDC) identity providers

service: aws-fargate-example-github-iam

provider:
  name: aws
  region: ${opt:region, 'ap-southeast-1'}
  stage: ${opt:stage, 'develop'}
  stackName: ${self:service}
  deploymentBucket:
    name: com.jessica.${self:provider.stage}-deploy-bucket
    serverSideEncryption: AES256
resources:
  Resources:
    GithubECSPolicy:
      Type: AWS::IAM::ManagedPolicy
      Properties:
        ManagedPolicyName: GithubECSPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Action:
                - ecr:BatchGetImage
                - ecr:BatchCheckLayerAvailability
                - ecr:CompleteLayerUpload
                - ecr:GetDownloadUrlForLayer
                - ecr:InitiateLayerUpload
                - ecr:PutImage
                - ecr:UploadLayerPart
                - ecr:GetAuthorizationToken
              Resource:
                - '*'
              Effect: Allow
            - Action:
                - ecs:*
                - elasticloadbalancing:DescribeTargetGroups
                - elasticloadbalancing:DescribeListeners
                - elasticloadbalancing:ModifyListener
                - elasticloadbalancing:DescribeRules
                - elasticloadbalancing:ModifyRule
                - lambda:InvokeFunction
                - cloudwatch:DescribeAlarms
                - sns:Publish
                - s3:GetObject
                - s3:GetObjectVersion
              Resource:
                - '*'
              Effect: Allow
            - Action:
                - iam:PassRole
              Resource:
                - !Sub 'arn:aws:iam::${AWS::AccountId}:role/develop_ECSTaskRole'
                - !Sub 'arn:aws:iam::${AWS::AccountId}:role/production_ECSTaskRole'
              Effect: Allow
    GitHubRole:
      Type: AWS::IAM::Role
      Properties:
        RoleName: GitHubRole
        AssumeRolePolicyDocument:
          Statement:
            - Effect: Allow
              Action: sts:AssumeRoleWithWebIdentity
              Principal:
                Federated:
                  - !Ref GithubOidc
              Condition:
                StringLike:
                  token.actions.githubusercontent.com:sub: repo:JessicaWin/aws-fargate-example:*
        ManagedPolicyArns:
          - !Ref GithubECSPolicy
    GithubOidc:
      Type: AWS::IAM::OIDCProvider
      Properties:
        Url: https://token.actions.githubusercontent.com
        ClientIdList:
          - sts.amazonaws.com
        ThumbprintList:
          - 6938fd4d98bab03faadb97b34396831e3780aea1

  Outputs:
    GitHubRole:
      Value: !GetAtt GitHubRole.Arn
      Export:
        Name: GitHubRole

 部署命令:

# deploy to develop stage
$ sls deploy --stage develop --region ap-southeast-1

# deploy to production stage
$ sls deploy --stage production --region ap-norheast-1

12.配置GitHub repository secrets

打开自己项目的setting页面,左边栏选择Secrets->Actions,打开页面后,点击右上角的New repository secret 按钮,然后添加Secrets。

具体步骤参考:actions-workflow-samples/create-secrets-for-GitHub-workflows.md at master · Azure/actions-workflow-samples · GitHub

本项目中用到了三个Secrets,需要添加以下三个Secrets:

  • AWS_FEDARATED_ROLE:arn:aws:iam::${AWS_AccountId}:role/GitHubRole
  • DEVELOP_ECS_TASK_ROLE: arn:aws:iam::${AWS_AccountId}:role/develop_ECSTaskRole
  • PRODUCTION_ECS_TASK_ROLE:  arn:aws:iam::${AWS_AccountId}:role/production_ECSTaskRole

13.使用GitHub workflow实现CI/CD

代码检测

一般代码检测包括formate和执行测试用例,检测时机为提交pull request

pre-merge.yml

name: Pre Merge Formate Check

on:
  # Triggers the workflow on pull request events but only for the master branch
  pull_request:
    branches: [develop, master]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-20.04
    strategy:
      matrix:
        node-version: [14.x]

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - name: Checkout repository
        uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v2
        with:
          node-version: ${{ matrix.node-version }}

      - name: Install
        run: npm ci

      - name: Format code
        run: npm run format

      - name: Check diff exists
        run: |
          git add -N .
          git diff
          line=`git diff | wc -l`
          if [ $line -gt 0 ]; then
            echo "■■■■■ You need to format before commit ■■■■"
            git diff
            exit -1
          fi
      - name: Lint code
        run: npm run lint

      - name: Run tests
        run: npm run test

Develop环境自动部署ECS Fargate Task Definition

develop环境实现代码merge时进行自动部署。部署流程如下:

  • checkout 当前分支
  • 使用${{ secrets.DEVELOP_ECS_TASK_ROLE }} 替换aws-resources/ecs/aws-fargate-example-task-def.json中的executionRoleArnValue
  • 生成aws credentials
  • 登录ecr
  • 生成版本号
  • 使用生成的版本号给项目打tag
  • 构建docker image并上传到ecr
  • 替换task definition中的image为ecr中image的路径
  • 更新task definition并部署

aws-fargate-example-task-def.json

{
    "family": "aws-fargate-example-task-definition",
    "executionRoleArn": "executionRoleArnValue",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [ "FARGATE" ],
    "cpu": "256",
    "memory": "512",
    "containerDefinitions": [
      {
        "name": "aws-fargate-example",
        "image": "TO_BE_REPLACED",
        "essential": true,
        "portMappings": [
          {
            "hostPort": 3000,
            "protocol": "tcp",
            "containerPort": 3000
          }
        ]
      }
    ]
  }

develop-release-and-deploy.yml

name: Develop Release & Deploy

on:
  push:
    branches: [master]

permissions:
  id-token: write
  contents: write

jobs:
  # For Staging. Build this project into docker image and push it in ECR.
  build-release-deploy:
    runs-on: ubuntu-20.04
    timeout-minutes: 300

    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
      
      - name: Replace executionRoleArn in task definition
        env:
          TASK_ROLE: ${{ secrets.DEVELOP_ECS_TASK_ROLE }}
        run: |
          sed -i "s@executionRoleArnValue@$TASK_ROLE@g" aws-resources/ecs/aws-fargate-example-task-def.json

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-region: ap-southeast-1
          role-to-assume: ${{ secrets.AWS_FEDARATED_ROLE }}
          role-session-name: aws-fargate-example

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      - name: Generate version
        id: generate-version
        run: |
          VERSION=`date "+%Y%m%d%H%M%S"`
          echo "::set-output name=version::$VERSION"

      - name: Build, tag, and push image to Amazon ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: aws-fargate-example
          IMAGE_TAG: ${{ steps.generate-version.outputs.version }}
        run: |
          git tag $IMAGE_TAG
          git push origin $IMAGE_TAG
          docker-compose build dev
          docker tag aws-fargate-example-dev:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

      - name: logout from Amazon ECR
        if: always()
        run: docker logout $ECR_REGISTRY

      # Insert Image URI in Task Definition file
      - name: render new task definition
        id: render-container
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: aws-resources/ecs/aws-fargate-example-task-def.json
          container-name: aws-fargate-example
          image: ${{ steps.build-image.outputs.image }}

      # Update ECS task version and Deploy
      - name: Deploy task definition to Amazon ECS
        uses: aws-actions/amazon-ecs-deploy-task-definition@v1
        with:
          task-definition: ${{ steps.render-container.outputs.task-definition }}
          service: aws-fargate-example
          cluster: aws-fargate-example-cluster

Production环境release时部署ECS Fargate Task Definition

production环境的部署以release作为触发条件,可以用develop部署过程中生成的tag进行release,也可以自己重新打tag进行release。

production-release-and-deploy.yml

name: Production Release & Deploy

on:
  release:
    types: [released]

permissions:
  id-token: write
  contents: read

jobs:
  # For Staging. Build this project into docker image and push it in ECR.
  build-release-deploy:
    runs-on: ubuntu-20.04
    timeout-minutes: 300

    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
      
      - name: Replace executionRoleArn in task definition
        env:
          TASK_ROLE: ${{ secrets.PRODUCTION_ECS_TASK_ROLE }}
        run: |
          sed -i "s@executionRoleArnValue@$TASK_ROLE@g" aws-resources/ecs/aws-fargate-example-task-def.json

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-region: ap-northeast-1
          role-to-assume: ${{ secrets.AWS_FEDARATED_ROLE }}
          role-session-name: aws-fargate-example

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1

      - name: Build, tag, and push image to Amazon ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: aws-fargate-example
        run: |
          IMAGE_TAG=$(echo ${{ github.ref }} | sed -e "s#refs/tags/##g")
          export IMAGE_TAG=$IMAGE_TAG
          docker-compose build prod
          docker tag aws-fargate-example-prod:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

      - name: logout from Amazon ECR
        if: always()
        run: docker logout $ECR_REGISTRY

      # Insert Image URI in Task Definition file
      - name: render new task definition
        id: render-container
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: aws-resources/ecs/aws-fargate-example-task-def.json
          container-name: aws-fargate-example
          image: ${{ steps.build-image.outputs.image }}

      # Update ECS task version and Deploy
      - name: Deploy task definition to Amazon ECS
        uses: aws-actions/amazon-ecs-deploy-task-definition@v1
        with:
          task-definition: ${{ steps.render-container.outputs.task-definition }}
          service: aws-fargate-example
          cluster: aws-fargate-example-cluster

项目地址

https://github.com/JessicaWin/aws-fargate-example

参考

Webhook events and payloads - GitHub Docs

About workflows - GitHub Docs

actions-workflow-samples/create-secrets-for-GitHub-workflows.md at master · Azure/actions-workflow-samples · GitHub

https://github.blog/changelog/2021-04-20-github-actions-control-permissions-for-github_token/

你可能感兴趣的:(AWS实战,aws,docker,云计算)