AWS 官网 | https://aws.amazon.com/ |
AWS Management Console | https://aws.amazon.com/console/ |
AWS Doc |
https://docs.aws.amazon.com/ |
AWS CLI (Lastest 2.13.20 Command Reference) | https://awscli.amazonaws.com/v2/documentation/api/latest/reference/index.html |
AWS SDK | https://aws.amazon.com/developer/tools/ Python: https://aws.amazon.com/sdk-for-python/ Python API 1.28.53: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/index.html Java: https://docs.aws.amazon.com/sdk-for-java/ Java API 2.20.153: https://sdk.amazonaws.com/java/api/latest/ |
AWS Github | https://github.com/aws |
AWS Book | 《Amazon Web Services in Action, Third Edition》By Michael Wittig and Andreas Wittig (中文版叫《AWS云计算实战》) |
AWS Video | https://www.udemy.com/course/aws-solutions-architect-professional/ https://www.udemy.com/course/aws-certified-solutions-architect-professional-training/ https://learning.oreilly.com/videos/amazon-web-services/9780137928521/ |
AWS Glossary | https://docs.aws.amazon.com/glossary/latest/reference/glos-chap.html |
AWS Training | https://aws.amazon.com/training/learn-about/architect/ |
AWS Certification 认证 |
https://aws.amazon.com/certification |
AWS Certification Practice Test | Examtopic:(注意甄别答案) https://www.examtopics.com/exams/amazon/ https://www.examtopics.com/exams/amazon/aws-certified-solutions-architect-professional-sap-c02/
https://www.udemy.com/course/practice-exam-aws-certified-solutions-architect-professional/ |
附:如有需要,以上链接的网页右上角都可以切换成中文版。
Management Console (管理控制台)
CLI (Command Line Interface)
SDK (Software Development Kit)
Please note that everything in AWS is 100% API driven.
其他推荐阅读:
谷歌云GCP | https://blog.csdn.net/Beth_Chan/article/details/113461721 |
阿里云(包含阿里云计算、存储、数据处理、Java 微服务等案例) | https://blog.csdn.net/Beth_Chan/article/details/111176779 |
MOOC网站访问日志分析(阿里云案例) | https://blog.csdn.net/Beth_Chan/article/details/113727493 |
Infrastructure as Code - Terraform | https://blog.csdn.net/Beth_Chan/article/details/133276479(待整理) |
Python | https://blog.csdn.net/Beth_Chan/article/details/133421056(待整理) |
IAM 身份和访问管理
Identity & Federation: MFA 多重身份认证
Account 账号
Key Concepts:
IAM Users 用户
IAM Roles 角色
IAM Permission 权限
IAM Groups 组
IAM Policies: Service Control Policies (SCP)
STS: Security Token Service
允许这两者共存
(Organizational Units, OU; Management Account, Member Account)
Create Customer Master Key (CMK) for S3 Encryption
Parameter Store
Create TLS certificate
Amazon Virtual Private Cloud is a commercial cloud computing service that provides users a virtual private cloud, by "provision[ing] a logically isolated section of Amazon Web Services Cloud". Enterprise customers are able to access the Amazon Elastic Compute Cloud over an IPsec based virtual private network.
You need to provide IPv4 or IPv6 CIDR range while creating the VPC.
AWS Direct Connect and Site to Site VPN are the services which provide the connectivity between AWS and on-premises networks. AWS Direct connect provides the private connectivity via the dedicated network while Site to Site VPN provides the secure (IPSec) connectivity over the internet.
AWS Direct Connect and VPN both provides the private connectivity between AWS and your corporate network. However VPN traffic flows over the internet and hence can not be considered as consistent whereas Direct Connect connection is over the dedicated physical connection and is more consistent and stable.
AWS Lambda requires NAT to connect to the Internet. Public IP addresses cannot be assigned to an AWS Lambda function.
Amazon CloudFront is a content delivery network operated by Amazon Web Services. Content delivery networks provide a globally-distributed network of proxy servers that cache content, such as web videos or other bulky media, more locally to consumers, thus improving access speed for downloading the content.
AWS Shield is a managed Distributed Denial of Service (DDoS) protection service that safeguards applications running on AWS. AWS Shield provides always-on detection and automatic inline mitigations that minimize application downtime and latency, so there is no need to engage AWS Support to benefit from DDoS protection.
虚拟私有云
基本概念:
Private, Public vs Elastic IP
Internetwork traffic privacy in Amazon VPC - Amazon Virtual Private Cloud
如果Web架构是单台EC2的话还算好处理,把EC2设定好固定IP即可;但如果是比较大型或是高流量的架构通常都会用 Auto Scaling,这时候EC2的数量跟IP变成不固定,随时都可能变动,因此我们需要把所有EC2内对外的请求连线,做出一些处理,让这些请求到达第三方服务的时候,IP永远是固定的。
NAT (Network Address Translation, 网络地址转换),字面上的意思就是它可以转换IP。
由AWS官方提供的架构图可知,DB Server是设定在Private Subnet里面,并不能直接连到外网,但透过Route table的设定可以连到Public Subnet的NAT Gateway,再通过NAT Gateway转换IP,并通过Public Subnet的Internet Gateway连到Internet。
DNS: Domain Name System
Route 53 Scenairos: EC2 instance; EC2 DNS name; ALB; CloudFront distribution; API Gateway; RDS DB instance; S3 bucket; VPC interface endpoint
VPC Flow Logs Monitoring
Network monitoring
What is PrivateLink?
https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html
Target Group & Application Load Balancer & Network Load Balancer & VPC Endpoint Service & VPC Endpoint:
https://aws.amazon.com/blogs/networking-and-content-delivery/application-load-balancer-type-target-group-for-network-load-balancer/
直接将ALB注册为NLB目标,无需主动管理不断变化的ALB IP地址。这是通过使用引入的应用程序负载均衡器类型的目标组来实现的。由此可以将NLB的优势(包括PrivateLink和区域静态IP地址)与ALB提供的高级路由结合起来,对应用程序的流量进行负载平衡。
Transit Gateway Route Table
Attachment
Transit Gateway Peering
Hybrid
What is Direct Connect?
Direct Connect uses private, public, and transit virtual interfaces (VIF).
Direct Connect Gateway
Border Gateway Protocol (BGP)
Bidirectional Forwarding Detection (BFD)
Link Aggregation Group (LAG)
Direct Connect Monitoring
Metrics
Elastic Load Balancers (ELB)
2. Network Load Balancer
3. Gateway Load Balancer
Auto Scaling Groups(ASGs,自动扩展组/自动伸缩组)AZ: Availability Zone 可用区
AMI (Amazon Machine Image, Amazon系统映像):操作系统和预安装软件的组合。不包括操作系统内核。操作系统内核从Amazon Kernel Image (AKI) 加载。
HVM (Hardware Virtual Machine):最新也是最快的虚拟化类型。
Container Service
Amazon Elastic Container Service (Amazon ECS)
Amazon Elastic Kubernetes Service (Amazon EKS)
Kubernetes Architecture
Pod to Pod communication
Security Groups in EKS
Exposing services
EKS Summary
• EKS control plane is launched in AWS managed VPC and EKS data plane (worker nodes) is launched in customer VPC.
• EKS provisions ENIs into customer VPC to enable communication between EKS control plane and data plane
• EKS cluster API endpoint is publicly accessible by default but can be configured as a private in which case it can be accessed from customer VPC via the EKS owned ENI
• EKS uses Amazon VPC Container Network Interface (CNI) plugin for Pod networking.
• CNI allocates IPs to each Pod from available Secondary IPs
• Maximum number of Pods per node depends on number of ENIs and number of IP addresses per ENI
• For supported Nitro based instance types, Pod per node limit can be increased using Prefix delegation (/28 for IPv4 and /80 for IPv6)
• Custom Networking enables associating secondary VPC CIDR (100.64.0.0/16) and when combined with SNAT enables much larger IPv4 private IPs for Pods.
• CNI allows Nodes to enable/disable SNAT to allow outbound internet access to Pods through the Internet gateway or NAT gateway respectively.
• By default, ENI security group is assigned to all the Pods which have been allocated secondary IPs for that ENI
• Pods specific security group can be assigned using Trunk & Branch ENI feature for selected Nitro system based instances.
• Pod services can be configured using ClusterIP, NodePort, LoadBalancer and Ingress resources.
• ClusterIP allows accessing services from within the cluster only.• NodePort allows accessing services externally using Node IP and static port
• LoadBalancer service can be configured to use CLB or NLB in instance mode.
• Ingress service can be configured to use ALB in instance or IP mode.
• AWS Load Balancer Controller can be used for LoadBalancer (with NLB IP mode) and Ingress service (with ALB) configurations.
• externalTrafficPolicy=Local allows NLB in instance mode to preserve client IP address by disabling kube-proxy to send traffic to other nodes.
Amazon Elastic Container Registry (Amazon ECR)
ECS Fargate (Serverless Docker)
EKS Fargate (Serverless Kubernetes)
No need to provison and manage server.
Implement a function in Python, Java, JavaScript(Node.js), Go, C# or Ruby, etc.
在调用AWS lambda函数时,输入可提供一个事件(event)和一个上下文(context)对象。event是函数获得输入参数的一种方法,通常采用JSON格式。
Python的print和JavaScript的console.log都默认会被重定向到CloudWatch Logs。
Python JSON dumps & load
json.dumps(): from JSON object to string
json.load(): from string to JSON object
JavaScript(Node.js) JSON stringify & parse
JSON.stringify(): from JSON object to string
JSON.parse(): parse string to JSON object
用code inline,zip或者contain image部署都可以。
例子:
Python:
def handler_name(event, context):
//...
return some_value
Context:
import time
def lambda_handler(event, context):
print("Lambda function ARN:", context.invoked_function_arn)
print("CloudWatch log stream name:", context.log_stream_name)
print("CloudWatch log group name:", context.log_group_name)
print("Lambda Request ID:", context.aws_request_id)
print("Lambda function memory limits in MB:", context.memory_limit_in_mb)
# We have added a 1 second delay so you can see the time remaining in get_remaining_time_in_millis.
time.sleep(1)
print("Lambda time remaining in MS:", context.get_remaining_time_in_millis())
Node.js:
Context:
exports.handler = async function(event, context) {
console.log('Remaining time: ', context.getRemainingTimeInMillis())
console.log('Function name: ', context.functionName)
return context.logStreamName
}
对象存储服务 Amazon S3
存储桶 bucket:可提供访问控制,不同存储桶可以有不同的可访问性
数据对象 data object:由内容和元数据组成。元数据:最后修改日期、内容类型、用户自定义。每个对象由键来确定。存储桶位于一个区域内。上传静态文件后得到的是一个URL (https://bucket-name.s3.amazonaws.com/sample+key/name.jpg)
单个文件最大5T。
设计为99.999999999%的可靠性。
EBS (Elastic Block Store)
备份和归档的存储服务
NAS (Network Attached Storage,网络附加存储)
NFS (Network File System, 解决多个EC2实例之间共享块存储的问题)
Elastic File System, 基于NFSv4协议
PostgreSQL/MySQL/Oracle/Microsoft Server SQL/MariaDB
Amazon Aurora PostgreSQL
Amazon Aurora MySQL DB
Amazon Aurora Serverless
Managed ActiveMQ, RabbitMQ
Detect and translate text
Perform speech-to-text and vice versa
Extract information from text
build voice and text chatbox
What's Monitoring & 360 Degree View:
1. Monitoring End-to-End
2. External Monitoring (End User Experience)
External monitoring is anything that happens out HERE, from the USER perspective. Not from AWS centric perspective, like EC2, ALB, CloudFront, etc.
Passive Response
Active Response
Concept Overview
Differenct type of Monitoring
AWS Services
Monitoring Permissions
IAM Role allowing CloudTrail to write to CloudWatch
Logs (when services access other service APIs)
Bucket Policy to allow ELB to write access logs to S3
Bucket
Primarily used for S3 bucket access
Primitive - Pre-dates IAM
Permission Combination with Multiple Policy
CloudWatch Agent
Events
Alarms
Logs
Metrics
Dashboards
Performance Monitoring
Logging Monitoring
VPC Flow Logs
Record is great for monitoring, troubleshooting and root cause analysis.
重要字段有:srcaddr, dstport, bytes, action等。
VPC Traffic Mirroring
Create stack from 蓝图 (blueprint),配置管理服务 AWS CloudFormation 使用的蓝图被称为模版 (template)。
Parameter Store
Run Command
Session Manager
Change Manager
Inventory
Schema Tool Service
Amazon Simple Notification Service
Amazon Simple Queue Service
Amazon MQ
Simple Email Service
distribute tracing
Automate operations with Chef and Puppet
Fault Injection Simulator
JSON基础架构标记语言(JSON Infrastructure Markup Language,JIML)。
JMESPath (JSON Matching Expression paths)
ICMP ( Internet Control Message Protocol, 因特网控制报文协议)
RTO (Recovery Time Objective 恢复时间目标)
RPO (Recovery Point Objective 恢复点目标)
TTL (Time to Live 生存时间)
DSL (Domain-Specific Language)
(可用Jenkins)
AWS自带的服务有:
CodeCommit / Github / Bitbucket
CodeBuild / Jenkins
CodePipeline
CodeDeploy / Jenkins
AWS Credentials 一个小时自动过期!
Using S3 Lifecycle Policies to Reduce Storage Costs
$ ls
book_cover.png lifecycle-rule.json README.md
$ RANDOM_STRING=$(aws secretsmanager get-random-password \
> --exclude-punctuation --exclude-uppercase \
> --password-length 6 --require-each-included-type \
> --output text \
> --query RandomPassword)
$ aws s3api create-bucket --bucket awscookbook301-$RANDOM_STRING
{
"Location": "/awscookbook301-4e5oom"
}
$ cat lifecycle-rule.json
{
"Rules": [
{
"ID": "Move all objects to Standard Infrequently Access",
"Prefix": "",
"Status": "Enabled",
"Transitions": [
{
"Days": 30,
"StorageClass": "STANDARD_IA"
}
]
}
]
}
$ aws s3api put-bucket-lifecycle-configuration \
> --bucket awscookbook301-$RANDOM_STRING \
> --lifecycle-configuration file://lifecycle-rule.json
$ aws s3api get-bucket-lifecycle-configuration \
> --bucket awscookbook301-$RANDOM_STRING
{
"Rules": [
{
"ID": "Move all objects to Standard Infrequently Access",
"Prefix": "",
"Status": "Enabled",
"Transitions": [
{
"Days": 30,
"StorageClass": "STANDARD_IA"
}
]
}
]
}
$ aws s3 cp book_cover.png s3://awscookbook301-$RANDOM_STRING
upload: ./book_cover.png to s3://awscookbook301-4e5oom/book_cover.png
$ aws s3api list-objects-v2 --bucket awscookbook301-$RANDOM_STRING
{
"Contents": [
{
"Key": "book_cover.png",
"LastModified": "2023-11-18T07:37:43+00:00",
"ETag": "\"d38461283ddc63b80044e2af6a7afd0d\"",
"Size": 255549,
"StorageClass": "STANDARD"
}
],
"RequestCharged": null
}