aws(学习笔记第三十一课)
-
aws cdk深入学习
学习内容：
- 深入练习
aws cdk下部署batch-arm64-instance-type
1. 深入练习aws cdk下部署batch-arm64-instance-type
代码链接
- 代码链接
代码链接 -> batch-arm64-instance-type - 之前代码学习
之前学习代码链接 -> aws(学习笔记第十八课) 使用aws cdk(python)进行部署
- 代码链接
batch-arm64-instance-type代码测试setup环境setup virtual environmentpython -m venv .venv- 激活
python virtual environmentsource .venv/Scripts/activate #这里使用的是git bash - 安装
python的依赖包pip install -r requirements.txt - 代码中
cdk.json的python3进行替换
这里的phthon3没有配置，所以不能执行python3，讲这里改成了python。
代码执行
预先执行
cdk bootstrap
cdk提供了bootstrap命令来创建cdk执行需要的前提条件。CDKToolkit: creating CloudFormation changeset... CDKToolkit | 0/12 | 21:22:46 | REVIEW_IN_PROGRESS | AWS::CloudFormation::Stack | CDKToolkit User Initiated CDKToolkit | 0/12 | 21:22:52 | CREATE_IN_PROGRESS | AWS::CloudFormation::Stack | CDKToolkit User Initiated CDKToolkit | 0/12 | 21:22:56 | CREATE_IN_PROGRESS | AWS::IAM::Role | FilePublishingRole CDKToolkit | 0/12 | 21:22:56 | CREATE_IN_PROGRESS | AWS::IAM::Role | CloudFormationExecutionRole CDKToolkit | 0/12 | 21:22:56 | CREATE_IN_PROGRESS | AWS::ECR::Repository | ContainerAssetsRepository CDKToolkit | 0/12 | 21:22:56 | CREATE_IN_PROGRESS | AWS::S3::Bucket | StagingBucket CDKToolkit | 0/12 | 21:22:56 | CREATE_IN_PROGRESS | AWS::IAM::Role | ImagePublishingRole CDKToolkit | 0/12 | 21:22:56 | CREATE_IN_PROGRESS | AWS::IAM::Role | LookupRole CDKToolkit | 0/12 | 21:22:56 | CREATE_IN_PROGRESS | AWS::SSM::Parameter | CdkBootstrapVersion CDKToolkit | 0/12 | 21:22:57 | CREATE_IN_PROGRESS | AWS::SSM::Parameter | CdkBootstrapVersion Resource creation Initiated CDKToolkit | 0/12 | 21:22:57 | CREATE_IN_PROGRESS | AWS::IAM::Role | FilePublishingRole Resource creation Initiated CDKToolkit | 0/12 | 21:22:57 | CREATE_IN_PROGRESS | AWS::ECR::Repository | ContainerAssetsRepository Resource creation Initiated CDKToolkit | 0/12 | 21:22:57 | CREATE_IN_PROGRESS | AWS::IAM::Role | CloudFormationExecutionRole Resource creation Initiated CDKToolkit | 0/12 | 21:22:57 | CREATE_IN_PROGRESS | AWS::IAM::Role | ImagePublishingRole Resource creation Initiated CDKToolkit | 0/12 | 21:22:57 | CREATE_IN_PROGRESS | AWS::S3::Bucket | StagingBucket Resource creation Initiated CDKToolkit | 0/12 | 21:22:57 | CREATE_IN_PROGRESS | AWS::IAM::Role | LookupRole Resource creation Initiated CDKToolkit | 1/12 | 21:22:57 | CREATE_COMPLETE | AWS::SSM::Parameter | CdkBootstrapVersion CDKToolkit | 2/12 | 21:22:58 | CREATE_COMPLETE | AWS::ECR::Repository | ContainerAssetsRepository CDKToolkit | 3/12 | 21:23:12 | CREATE_COMPLETE | AWS::S3::Bucket | StagingBucket CDKToolkit | 3/12 | 21:23:14 | CREATE_IN_PROGRESS | AWS::S3::BucketPolicy | StagingBucketPolicy CDKToolkit | 3/12 | 21:23:15 | CREATE_IN_PROGRESS | AWS::S3::BucketPolicy | StagingBucketPolicy Resource creation Initiated CDKToolkit | 4/12 | 21:23:15 | CREATE_COMPLETE | AWS::S3::BucketPolicy | StagingBucketPolicy CDKToolkit | 5/12 | 21:23:16 | CREATE_COMPLETE | AWS::IAM::Role | ImagePublishingRole CDKToolkit | 6/12 | 21:23:16 | CREATE_COMPLETE | AWS::IAM::Role | FilePublishingRole CDKToolkit | 7/12 | 21:23:16 | CREATE_COMPLETE | AWS::IAM::Role | CloudFormationExecutionRole CDKToolkit | 7/12 | 21:23:16 | CREATE_IN_PROGRESS | AWS::IAM::Policy | FilePublishingRoleDefaultPolicy CDKToolkit | 8/12 | 21:23:16 | CREATE_COMPLETE | AWS::IAM::Role | LookupRole CDKToolkit | 8/12 | 21:23:17 | CREATE_IN_PROGRESS | AWS::IAM::Policy | ImagePublishingRoleDefaultPolicy CDKToolkit | 8/12 | 21:23:17 | CREATE_IN_PROGRESS | AWS::IAM::Role | DeploymentActionRole CDKToolkit | 8/12 | 21:23:18 | CREATE_IN_PROGRESS | AWS::IAM::Policy | FilePublishingRoleDefaultPolicy Resource creation Initiated CDKToolkit | 8/12 | 21:23:18 | CREATE_IN_PROGRESS | AWS::IAM::Policy | ImagePublishingRoleDefaultPolicy Resource creation Initiated CDKToolkit | 8/12 | 21:23:19 | CREATE_IN_PROGRESS | AWS::IAM::Role | DeploymentActionRole Resource creation Initiated CDKToolkit | 9/12 | 21:23:34 | CREATE_COMPLETE | AWS::IAM::Policy | FilePublishingRoleDefaultPolicy CDKToolkit | 10/12 | 21:23:34 | CREATE_COMPLETE | AWS::IAM::Policy | ImagePublishingRoleDefaultPolicy CDKToolkit | 11/12 | 21:23:38 | CREATE_COMPLETE | AWS::IAM::Role | DeploymentActionRole CDKToolkit | 12/12 | 21:23:39 | CREATE_COMPLETE | AWS::CloudFormation::Stack | CDKToolkit ✅ Environment aws://081353481087/ap-northeast-1 bootstrapped.这里，
cdk bootstrap命令执行完之后，会生成一个CDKToolkit名字的cloudformation。
如果由于平时的误删除之类的操作，会使CDKToolkit的创建的cdk deploy需要的前提条件遭到破坏。这时如果重新执行cdk bootstrap会导致没有差分执行出来。这时候需要将CDKToolkit这个cloudformation删除掉，之后执行cdk bootstrap命令，这样就会重新生成cdk deploy执行的所有条件。执行
cdk --require-approval never deploy
如果执行不加上--require-approval never，会导致错误，所以这里加上该选项。Trouble Shouting
如果存在默认的vpc，这里会出现如下错误。
原因在于:
Fn:GetAZs这个函数调用的话，会只返回default VPC的Available Zone的数量，从而影响cdk的执行。
对策：
所以要删除default VPC，不过不用担心，default VPC可以重新创建，大可放心。
代码解析
全体架构
作成
VPC# This resource alone will create a private/public subnet in each AZ as well as nat/internet gateway(s) vpc = ec2.Vpc(self, "VPC")创建一下
service组件：- 一个
VPC - 一个
public subnet - 一个
private subnet - 一个
Internet Gateway - 一个
NAT GatewayCompute Environment通过这个NAT Gateway访问internet
- 一个
创建
Job Queue# Create AWS Batch Job Queue self.batch_queue = batch.JobQueue(self, "JobQueueArm64")
创建
Compute Environment，并将每个Compute Environment分配给JobQueue# For loop to create Batch Compute Environments for i in range(count): name = "MyBatchARM64Env" + str(i) batch_environment = batch.ManagedEc2EcsComputeEnvironment(self, name, spot=True, spot_bid_percentage=75, instance_types=[ec2.InstanceType("c7g.medium"),ec2.InstanceType("c7g.large")], use_optimal_instance_classes=False, vpc_subnets=ec2.SubnetSelection(subnet_type=ec2.SubnetType.PRIVATE_WITH_NAT), vpc=vpc ) self.batch_queue.add_compute_environment(batch_environment, i)
创建
Job Defintion# Create ECS Job Definition to submit job in batch job queue. batch_jobDef = batch.EcsJobDefinition(self, "MyJobDefArm64", container=batch.EcsEc2ContainerDefinition(self, "CDKJobDefArm64", image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"), command=["sleep", "60"], memory=Size.mebibytes(512), cpu=1 ) )
执行
job- 投入新的
Job
Job执行成功
- 投入新的