AWS SDK for Ruby で EC2 ImageBuilder のパイプラインを作成する

まとめ

• Aws::Imagebuilder::Client クラスを使う

事の起こり

• EC2 ImageBuilder でゴールデンAMIを作成している
• EC2 ImageBuilder のコンポーネントを変更する際、それを使うようにパイプラインを更新するには、コンポーネント、レシピ、パイプラインとそれぞれ作り直す必要があり、面倒
• AWSマネジメントコンソール上からだと設定できないオプション（たとえばイメージ作成に使用するEC2インスタンスのEBSボリュームサイズ）がある
• CloudFormationでは作成できない

AWS SDKを用いてコード化し、管理可能かつ実行可能にすることで、作業を効率化したくなりました。

サンプルコード

CloudFormation

S3バケットやEC2インスタンスプロファイルなどはCloudFormationで作成していたのでそれを使いました。 スタックのOutputsでRubyスクリプトに渡します。

ポイントはEC2インスタンスに割り当てるロールです。

• 管理ポリシーを割り当てる
  • AmazonSSMManagedInstanceCore
  • EC2InstanceProfileForImageBuilder
• S3へログをアップロードするため、S3バケットへのアクセス権を付けておく（関連 EC2 ImageBuilder コンポーネントのデバッグ）
• その他ユースケースに応じて必要な権限をつけておく
  • 私の場合はコンポーネント中でCodeCommitからソースコードをチェックアウト、設定値をParameterStoreから取得していたので、それぞれの権限をつけました。
    • AWSCodeCommitReadOnly
    • AmazonSSMReadOnlyAccess

Mappings:
  SubnetConfig:
    VPC:
      CIDR: "10.0.0.0/16"
    Private:
      CIDR: "10.0.1.0/24"
    Public:
      CIDR: "10.0.2.0/24"
Resources:
  ImageBuilderRole:
    Type: "AWS::IAM::Role"
    DeletionPolicy: Retain
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ec2.amazonaws.com
            Action:
              - "sts:AssumeRole"
      Path: /
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
        - "arn:aws:iam::aws:policy/EC2InstanceProfileForImageBuilder"
      Policies:
        - PolicyName: allow-put-log
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                  - "s3:*"
                Resource:
                  - !GetAtt BuildLogBucket.Arn
                  - !Join ["/", [!GetAtt BuildLogBucket.Arn, "*"]]

  EIP:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc

  VPC:
    Type: AWS::EC2::VPC
    Properties:
      EnableDnsSupport: true
      EnableDnsHostnames: true
      CidrBlock: !FindInMap [SubnetConfig, VPC, CIDR]
  PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - Fn::GetAZs: !Ref "AWS::Region"
      VpcId: !Ref "VPC"
      CidrBlock: !FindInMap ["SubnetConfig", "Private", "CIDR"]
  PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - Fn::GetAZs: !Ref "AWS::Region"
      VpcId: !Ref "VPC"
      CidrBlock: !FindInMap ["SubnetConfig", "Public", "CIDR"]
  InternetGateway:
    Type: AWS::EC2::InternetGateway
  GatewayAttachement:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway
  NatGateway:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt EIP.AllocationId
      SubnetId: !Ref PublicSubnet
  PrivateRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref "VPC"
  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref "VPC"
  PrivateRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref "PrivateRouteTable"
      DestinationCidrBlock: "0.0.0.0/0"
      NatGatewayId: !Ref NatGateway
  PublicRoute:
    Type: AWS::EC2::Route
    Properties:
      RouteTableId: !Ref "PublicRouteTable"
      DestinationCidrBlock: "0.0.0.0/0"
      GatewayId: !Ref "InternetGateway"
  PrivateSubnetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet
      RouteTableId: !Ref PrivateRouteTable
  PublicSubnetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PublicSubnet
      RouteTableId: !Ref PublicRouteTable

  EC2SecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Access to EC2 Instances
      VpcId: !Ref "VPC"

  ImageBuilderInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: "/"
      Roles:
        - !Ref ImageBuilderRole

  BuildLogBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: buildlog
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      LifecycleConfiguration:
        Rules:
          - Status: Enabled
            ExpirationInDays: 30
            AbortIncompleteMultipartUpload:
              DaysAfterInitiation: 7

Outputs:
  PrivateSubnetId:
    Value: !Ref PrivateSubnet
  EC2SecurityGroupId:
    Value: !Ref EC2SecurityGroup
  BuildLogBucket:
    Value: !Ref BuildLogBucket
  ImageBuilderInstanceProfile:
    Value: !Ref ImageBuilderInstanceProfile

Ruby

CloudFormationのスタックから必要な設定値を取得するようにしました。 ここはコマンド引数などで与えてもよいですし、アプリ等に組み込むなら関数にして引数として与えてもよいでしょう。

require "aws-sdk"
require "securerandom"

semantic_version = Time.now.strftime('0.%Y%m%d.%H%M%S')
imagebuilder = Aws::Imagebuilder::Client.new(region: AWS_REGION)

def get_stack_output(name)
  stack_outputs[name]
end

def stack_outputs
  return @stack_outputs if defined?(@stack_outputs)

  cloud_formation = Aws::CloudFormation::Client.new(region: AWS_REGION)
  stack = cloud_formation.describe_stacks(stack_name: STACK_NAME).stacks[0]
  @stack_outputs = stack.outputs.inject({}) { |memo, output| memo.tap { |memo| memo[output.output_key] = output.output_value } }
end

component_build_version_arn = imagebuilder.create_component(
  name: COMPONENT_NAME, # required
  semantic_version: semantic_version, # required
  platform: "Linux", # required, accepts Windows, Linux
  data: COMPONENT_YAML,
  client_token: SecureRandom.uuid, # required
).component_build_version_arn

image_recipe_arn = imagebuilder.create_image_recipe(
  name: RECIPE_NAME,
  semantic_version: semantic_version, # required
  components: [ # required
    {
      component_arn: 'arn:aws:imagebuilder:ap-northeast-1:aws:component/update-linux/1.0.0',
    },
    {
      component_arn: component_build_version_arn,
    },
    {
      component_arn: 'arn:aws:imagebuilder:ap-northeast-1:aws:component/simple-boot-test-linux/1.0.0',
    },
    {
      component_arn: 'arn:aws:imagebuilder:ap-northeast-1:aws:component/reboot-test-linux/1.0.0',
    }
  ],
  parent_image: "arn:aws:imagebuilder:ap-northeast-1:aws:image/amazon-linux-2-x86/x.x.x", # required
  block_device_mappings: [
    {
      device_name: "/dev/xvda",
      ebs: {
        encrypted: false,
        delete_on_termination: true,
        volume_size: 16,
        volume_type: "gp2", # accepts standard, io1, gp2, sc1, st1
      }
    },
  ],
  client_token: SecureRandom.uuid, # required
).image_recipe_arn

infrastructure_configuration_arn = imagebuilder.create_infrastructure_configuration({
  name: INFRASTRUCTURE_CONFIGURATION_NAME, # required
  instance_types: ["m5a.large"],
  instance_profile_name: get_stack_output('ImageBuilderInstanceProfile'), # required
  security_group_ids: [get_stack_output('EC2SecurityGroupId')],
  subnet_id: get_stack_output('PrivateSubnetId'),
  logging: {
    s3_logs: {
      s3_bucket_name: get_stack_output('BuildLogBucket'),
      s3_key_prefix: "path/to/logs/",
    },
  },
  terminate_instance_on_failure: true,
  client_token: SecureRandom.uuid, # required
}).infrastructure_configuration_arn

distribution_configuration_arn = imagebuilder.create_distribution_configuration({
  name: DISTRIBUTION_CONFIGURATION_NAME, # required
  distributions: [ # required
    {
      region: "ap-northeast-1", # required
      ami_distribution_configuration: {
        name: IMAGE_NAME, # e.g. "my-app {{imagebuilder:buildDate}}"
        ami_tags: {
          "TagName" => "TagValue"
        },
      }
    },
    {
      region: "us-west-2", # required
      ami_distribution_configuration: {
        name: IMAGE_NAME,
        ami_tags: {
          "TagName" => "TagValue"
        },
      }
    },
  ],
  client_token: SecureRandom.uuid, # required
}).distribution_configuration_arn


image_pipeline_arn = imagebuilder.list_image_pipelines(
  filters: [
    {
      name: "name",
      values: [IMAGE_PIPELINE_NAME],
    },
  ],
).image_pipeline_list[0]&.arn
if image_pipeline_arn
  imagebuilder.delete_image_pipeline(image_pipeline_arn: image_pipeline_arn)
end
imagebuilder.create_image_pipeline({
  name: IMAGE_PIPELINE_NAME, # required
  image_recipe_arn: image_recipe_arn, # required
  infrastructure_configuration_arn: infrastructure_configuration_arn, # required
  distribution_configuration_arn: distribution_configuration_arn,
  image_tests_configuration: {
    image_tests_enabled: true,
    timeout_minutes: 60,
  },
  schedule: {
    schedule_expression: "cron(0 0 * * tue)",
    pipeline_execution_start_condition: "EXPRESSION_MATCH_AND_DEPENDENCY_UPDATES_AVAILABLE", # accepts EXPRESSION_MATCH_ONLY, EXPRESSION_MATCH_AND_DEPENDENCY_UPDATES_AVAILABLE
  },
  status: "DISABLED", # accepts DISABLED, ENABLED
  client_token: SecureRandom.uuid, # required
})