今回はGitLab CI/CDでコンテナイメージを作成し、作成したイメージを使ってAmazon ECSのサービスを更新する例を紹介します。
背景
GitLab CI/CDからAmazon ECSサービスを更新するには、大きく2つの方法があります。
.gitlab-ci.ymlにスクリプトを記載するinclude:templateを利用する
GitLabドキュメントでは include:template の利用例が紹介されていますが、今回は1つ目のほうで実施しました。具体的には以下の処理をGitLab CI/CDで実施します。
build-image- コンテナイメージのビルド
- Amazon ECRへのイメージ配置
deploy-task-definition- task definition定義ファイルの書き換え
- タスク定義の更新
- サービスの更新
検証
ここから検証です。今回も使用したのはGitLab SaaS版 (Freeプラン) です。
AWSリソースの作成
まずはCI/CDで利用するリソースを作成します。リソースの作成はAWS CloudFormationで行いました。
ecr.yaml
AWSTemplateFormatVersion: "2010-09-09" Description: ECR for GitLab CI/CD example Parameters: PJPrefix: Type: String Default: "gitlab-cicd-example" Resources: ECR: Type: AWS::ECR::Repository Properties: RepositoryName: !Sub "${PJPrefix}-ecr" EncryptionConfiguration: EncryptionType: "KMS" ImageScanningConfiguration: ScanOnPush: true ImageTagMutability: IMMUTABLE Tags: - Key: Name Value: !Sub "${PJPrefix}-ecr" Outputs: ECR: Value: !Ref ECR Export: Name: !Sub "${PJPrefix}-ecr"
vpc.yaml
AWSTemplateFormatVersion: "2010-09-09" Description: VPC and Subnet for GitLab CI/CD example Parameters: PJPrefix: Type: String Default: "gitlab-cicd-example" VPCCIDR: Type: String Default: "10.1.0.0/16" PublicSubnetACIDR: Type: String Default: "10.1.10.0/24" PublicSubnetCCIDR: Type: String Default: "10.1.20.0/24" PrivateSubnetACIDR: Type: String Default: "10.1.100.0/24" PrivateSubnetCCIDR: Type: String Default: "10.1.200.0/24" Resources: # VPC VPC: Type: "AWS::EC2::VPC" Properties: CidrBlock: !Ref VPCCIDR EnableDnsSupport: true EnableDnsHostnames: true Tags: - Key: Name Value: !Sub "${PJPrefix}-vpc" # Internet Gateway InternetGateway: Type: "AWS::EC2::InternetGateway" Properties: Tags: - Key: Name Value: !Sub "${PJPrefix}-igw" InternetGatewayAttachment: Type: "AWS::EC2::VPCGatewayAttachment" Properties: InternetGatewayId: !Ref InternetGateway VpcId: !Ref VPC # NAT Gateway NATGatewayA: Type: "AWS::EC2::NatGateway" Properties: AllocationId: !GetAtt NATGatewayAEIP.AllocationId SubnetId: !Ref PublicSubnetA Tags: - Key: Name Value: !Sub "${PJPrefix}-natgw-a" NATGatewayAEIP: Type: "AWS::EC2::EIP" Properties: Domain: vpc Tags: - Key: Name Value: !Sub "${PJPrefix}-natgw-a" NATGatewayC: Type: "AWS::EC2::NatGateway" Properties: AllocationId: !GetAtt NATGatewayCEIP.AllocationId SubnetId: !Ref PublicSubnetC Tags: - Key: Name Value: !Sub "${PJPrefix}-natgw-c" NATGatewayCEIP: Type: "AWS::EC2::EIP" Properties: Domain: vpc Tags: - Key: Name Value: !Sub "${PJPrefix}-natgw-c" # Subnet PublicSubnetA: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: "ap-northeast-1a" CidrBlock: !Ref PublicSubnetACIDR VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-public-subnet-a" PublicSubnetC: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: "ap-northeast-1c" CidrBlock: !Ref PublicSubnetCCIDR VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-public-subnet-c" PrivateSubnetA: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: "ap-northeast-1a" CidrBlock: !Ref PrivateSubnetACIDR VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-private-subnet-a" PrivateSubnetC: Type: "AWS::EC2::Subnet" Properties: AvailabilityZone: "ap-northeast-1c" CidrBlock: !Ref PrivateSubnetCCIDR VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-private-subnet-c" # RouteTable PublicRouteTableA: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-public-route-a" PublicRouteTableC: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-public-route-c" PrivateRouteTableA: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-private-route-a" PrivateRouteTableC: Type: "AWS::EC2::RouteTable" Properties: VpcId: !Ref VPC Tags: - Key: Name Value: !Sub "${PJPrefix}-private-route-c" PublicRouteA: Type: "AWS::EC2::Route" Properties: RouteTableId: !Ref PublicRouteTableA DestinationCidrBlock: "0.0.0.0/0" GatewayId: !Ref InternetGateway PublicRouteC: Type: "AWS::EC2::Route" Properties: RouteTableId: !Ref PublicRouteTableC DestinationCidrBlock: "0.0.0.0/0" GatewayId: !Ref InternetGateway PrivateRouteA: Type: "AWS::EC2::Route" Properties: RouteTableId: !Ref PrivateRouteTableA DestinationCidrBlock: "0.0.0.0/0" NatGatewayId: !Ref NATGatewayA PrivateRouteC: Type: "AWS::EC2::Route" Properties: RouteTableId: !Ref PrivateRouteTableC DestinationCidrBlock: "0.0.0.0/0" NatGatewayId: !Ref NATGatewayC PublicSubnetARouteTableAssociation: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: SubnetId: !Ref PublicSubnetA RouteTableId: !Ref PublicRouteTableA PublicSubnetCRouteTableAssociation: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: SubnetId: !Ref PublicSubnetC RouteTableId: !Ref PublicRouteTableC PrivateSubnetARouteTableAssociation: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: SubnetId: !Ref PrivateSubnetA RouteTableId: !Ref PrivateRouteTableA PrivateSubnetCRouteTableAssociation: Type: "AWS::EC2::SubnetRouteTableAssociation" Properties: SubnetId: !Ref PrivateSubnetC RouteTableId: !Ref PrivateRouteTableC Outputs: VPC: Value: !Ref VPC Export: Name: !Sub "${PJPrefix}-vpc" PublicSubnetA: Value: !Ref PublicSubnetA Export: Name: !Sub "${PJPrefix}-public-subnet-a" PublicSubnetC: Value: !Ref PublicSubnetC Export: Name: !Sub "${PJPrefix}-public-subnet-c" PrivateSubnetA: Value: !Ref PrivateSubnetA Export: Name: !Sub "${PJPrefix}-private-subnet-a" PrivateSubnetC: Value: !Ref PrivateSubnetC Export: Name: !Sub "${PJPrefix}-private-subnet-c"
iam.yaml
AWSTemplateFormatVersion: '2010-09-09' Description: OIDC for GitLab CI/CD example Parameters: RepositoryPath: Type: String PJPrefix: Type: String Default: "gitlab-cicd-example" Resources: OIDCProviderForGitLab: Type: AWS::IAM::OIDCProvider Properties: Url: https://gitlab.com ClientIdList: - https://gitlab.com ThumbprintList: - ffffffffffffffffffffffffffffffffffffffff RoleForGitLab: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PJPrefix}-role" MaxSessionDuration: 3600 AssumeRolePolicyDocument: Statement: - Action: sts:AssumeRoleWithWebIdentity Principal: Federated: !Sub 'arn:aws:iam::${AWS::AccountId}:oidc-provider/gitlab.com' Effect: Allow Condition: ForAnyValue:StringLike: "gitlab.com:sub": - !Sub "project_path:${RepositoryPath}:ref_type:branch:ref:*" Policies: - PolicyName: !Sub "${PJPrefix}-policy" PolicyDocument: Statement: - Effect: Allow Action: - sts:GetCallerIdentity Resource: - '*' - Effect: Allow Action: - ecr:GetAuthorizationToken Resource: '*' - Effect: Allow Action: - ecr:UploadLayerPart - ecr:PutImage - ecr:InitiateLayerUpload - ecr:CompleteLayerUpload - ecr:BatchCheckLayerAvailability Resource: !Sub arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/${PJPrefix}-ecr - Effect: Allow Action: - ecs:RegisterTaskDefinition Resource: '*' - Effect: Allow Action: - ecs:UpdateServicePrimaryTaskSet - ecs:DescribeServices - ecs:UpdateService Resource: !Sub arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:service/${PJPrefix}-cluster/${PJPrefix}-service - Effect: Allow Action: - iam:PassRole Resource: !Sub arn:aws:iam::${AWS::AccountId}:role/${PJPrefix}-task-execution-role Condition: StringLike: iam:PassedToService: ecs-tasks.amazonaws.com
ecs.yaml
AWSTemplateFormatVersion: "2010-09-09" Description: ECS and ALB resources for GitLab CI/CD example Parameters: PJPrefix: Type: String Default: "gitlab-cicd-example" ALBSecurityGroupIngressIPAddress: Type: String InternetALBName: Type: String Default: "alb" TargetGroupName: Type: String Default: "tg" ECSClusterName: Type: String Default: "cluster" ECSTaskName: Type: String Default: "task" ECSTaskCPUUnit: AllowedValues: [ 256, 512, 1024, 2048, 4096 ] Type: String Default: "256" ECSTaskMemory: AllowedValues: [ 256, 512, 1024, 2048, 4096 ] Type: String Default: "512" ECSContainerName: Type: String Default: "container" ECSImageName: Type: String Default: "<AWS Account ID>.dkr.ecr.ap-northeast-1.amazonaws.com/gitlab-cicd-example-ecr:0.1" ECSServiceName: Type: String Default: "service" ECSTaskDesiredCount: Type: Number Default: 1 Resources: #Security Group ALBSecurityGroup: Type: "AWS::EC2::SecurityGroup" Properties: VpcId: { "Fn::ImportValue": !Sub "${PJPrefix}-vpc" } GroupName: !Sub "${PJPrefix}-alb-sg" GroupDescription: "-" Tags: - Key: "Name" Value: !Sub "${PJPrefix}-alb-sg" SecurityGroupIngress: - IpProtocol: tcp FromPort: 80 ToPort: 80 CidrIp: !Ref ALBSecurityGroupIngressIPAddress - IpProtocol: tcp FromPort: 443 ToPort: 443 CidrIp: !Ref ALBSecurityGroupIngressIPAddress ECSSecurityGroup: Type: "AWS::EC2::SecurityGroup" Properties: VpcId: { "Fn::ImportValue": !Sub "${PJPrefix}-vpc" } GroupName: !Sub "${PJPrefix}-ecs-sg" GroupDescription: "-" Tags: - Key: "Name" Value: !Sub "${PJPrefix}-ecs-sg" # Security Group Rule ECSSecurityGroupIngress: Type: "AWS::EC2::SecurityGroupIngress" Properties: IpProtocol: tcp FromPort: 80 ToPort: 80 SourceSecurityGroupId: !GetAtt [ ALBSecurityGroup, GroupId ] GroupId: !GetAtt [ ECSSecurityGroup, GroupId ] # Target Group TargetGroup: Type: "AWS::ElasticLoadBalancingV2::TargetGroup" Properties: VpcId: { "Fn::ImportValue": !Sub "${PJPrefix}-vpc" } Name: !Sub "${PJPrefix}-${TargetGroupName}" Protocol: HTTP Port: 80 TargetType: ip # ALB InternetALB: Type: "AWS::ElasticLoadBalancingV2::LoadBalancer" Properties: Name: !Sub "${PJPrefix}-${InternetALBName}" Tags: - Key: Name Value: !Sub "${PJPrefix}-${InternetALBName}" Scheme: "internet-facing" LoadBalancerAttributes: - Key: "deletion_protection.enabled" Value: false - Key: "idle_timeout.timeout_seconds" Value: 60 SecurityGroups: - !Ref ALBSecurityGroup Subnets: - { "Fn::ImportValue": !Sub "${PJPrefix}-public-subnet-a" } - { "Fn::ImportValue": !Sub "${PJPrefix}-public-subnet-c" } ALBListener: Type: "AWS::ElasticLoadBalancingV2::Listener" Properties: DefaultActions: - TargetGroupArn: !Ref TargetGroup Type: forward LoadBalancerArn: !Ref InternetALB Port: 80 Protocol: HTTP # ECS Cluster ECSCluster: Type: "AWS::ECS::Cluster" Properties: ClusterName: !Sub "${PJPrefix}-${ECSClusterName}" # Loggroup ECSLogGroup: Type: "AWS::Logs::LogGroup" Properties: LogGroupName: !Sub "/ecs/logs/${PJPrefix}-ecs-group" # ECS Task Definition ECSTaskDefinition: Type: "AWS::ECS::TaskDefinition" Properties: Cpu: !Ref ECSTaskCPUUnit ExecutionRoleArn: !Ref ECSTaskExecutionRole Family: !Sub "${PJPrefix}-${ECSTaskName}" Memory: !Ref ECSTaskMemory NetworkMode: awsvpc RequiresCompatibilities: - FARGATE ContainerDefinitions: - Name: !Sub "${PJPrefix}-${ECSContainerName}" Image: !Ref ECSImageName LogConfiguration: LogDriver: awslogs Options: awslogs-group: !Ref ECSLogGroup awslogs-region: !Ref "AWS::Region" awslogs-stream-prefix: !Ref PJPrefix MemoryReservation: 128 PortMappings: - HostPort: 80 Protocol: tcp ContainerPort: 80 # ECS Service ECSService: Type: AWS::ECS::Service DependsOn: ALBListener Properties: Cluster: !Ref ECSCluster DesiredCount: !Ref ECSTaskDesiredCount LaunchType: FARGATE LoadBalancers: - TargetGroupArn: !Ref TargetGroup ContainerPort: 80 ContainerName: !Sub "${PJPrefix}-${ECSContainerName}" NetworkConfiguration: AwsvpcConfiguration: AssignPublicIp: DISABLED SecurityGroups: - !Ref ECSSecurityGroup Subnets: - { "Fn::ImportValue": !Sub "${PJPrefix}-private-subnet-a" } - { "Fn::ImportValue": !Sub "${PJPrefix}-private-subnet-c" } ServiceName: !Sub "${PJPrefix}-${ECSServiceName}" TaskDefinition: !Ref ECSTaskDefinition ECSTaskExecutionRole: Type: AWS::IAM::Role Properties: RoleName: !Sub "${PJPrefix}-task-execution-role" Path: / AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: Allow Principal: Service: ecs-tasks.amazonaws.com Action: sts:AssumeRole ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy Outputs: ALBDNSName: Value: !GetAtt InternetALB.DNSName Export: Name: !Sub "${PJPrefix}-${InternetALBName}-dnsname"
VPC/ECR/IAMの作成
上記ファイルを使用してVPC/ECR/IAMを作成します。
IAMについてだけ補足すると、ここではOIDCを使ってGitLabからAWSへのアクセス権限を付与しています。
Dockerイメージの作成
続いてDockerイメージを作成します。作成には以下のファイルを使用しました。
FROM nginx:latest COPY ./src/index.html /usr/share/nginx/html/index.html
<html><body>Hello GitLab CI/CD</body></html>
ここではAWS CloudShell上でDockerイメージをビルドし、作成したAmazon ECRにプッシュしました。
ECSの作成
次に上記ファイルを使用してECSを作成します。
リソース作成後、ALB URLからアクセスすると、以下のようにテキストが表示されます。
GitLabの設定
続いてGitLab側を設定します。今回は ecs-example というGitLab Projectを使用します。
CI/CD環境変数の設定
まずは .gitlab-ci.yml上で呼び出す環境変数を設定します。ここでは以下の変数を設定します。なお設定時は Masked Expanded をチェックし、Jobログ中には変数情報をマスクし、 $ で呼び出せるようにしておきます。
- AWS_DEFAULT_REGION:
ap-northeast-1 - AWS_ECR_LOGIN_URL:
<AWS Account ID>.dkr.ecr.ap-northeast-1.amazonaws.com - AWS_ECR_REPOSITORY:
<AWS Account ID>.dkr.ecr.ap-northeast-1.amazonaws.com/gitlab-cicd-example-ecr - AWS_ECS_CLUSTER_NAME:
gitlab-cicd-example-cluster - AWS_ECS_SERVICE_NAME:
gitlab-cicd-example-service - AWS_IAM_ROLE:
arn:aws:iam::<AWS Account ID>:role/gitlab-cicd-example-role
ファイルの配置
続いてGitLab Projectにファイルを配置します。ファイルは以下のように配置します。
. ├── .gitlab-ci.yml ├── Dockerfile ├── README.md ├── cfn │ ├── ecr.yaml │ ├── ecs.yaml │ ├── iam.yaml │ └── vpc.yaml ├── src │ └── index.html └── task-definition.json
.gitlab-ci.yml task-definition.json は以下のような内容です。なお今回は .gitlab-ci.yml にruleを設定していないので、ファイルを配置した時からCI/CDパイプラインが実行されます。
.gitlab-ci.yml
variables: AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION AWS_ECR_LOGIN_URL: $AWS_ECR_LOGIN_URL AWS_ECR_REPOSITORY: $AWS_ECR_REPOSITORY AWS_ECS_CLUSTER_NAME: $AWS_ECS_CLUSTER_NAME AWS_ECS_SERVICE_NAME: $AWS_ECS_SERVICE_NAME AWS_IAM_ROLE: $AWS_IAM_ROLE DOCKER_TLS_CERTDIR: "/certs" stages: - build - deploy default: image: docker:24.0.5 services: - docker:24.0.5-dind id_tokens: GITLAB_OIDC_TOKEN: aud: https://gitlab.com before_script: - apk add --no-cache aws-cli - > export $(printf "AWS_ACCESS_KEY_ID=%s AWS_SECRET_ACCESS_KEY=%s AWS_SESSION_TOKEN=%s" $(aws sts assume-role-with-web-identity --role-arn ${AWS_IAM_ROLE} --role-session-name "GitLabRunner-${CI_PROJECT_ID}-${CI_PIPELINE_ID}" --web-identity-token ${GITLAB_OIDC_TOKEN} --duration-seconds 3600 --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' --output text)) - IMAGE_TAG=$CI_COMMIT_SHORT_SHA build-image: stage: build script: - docker build -t $AWS_ECR_REPOSITORY:$IMAGE_TAG . - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ECR_LOGIN_URL - docker push $AWS_ECR_REPOSITORY:$IMAGE_TAG deploy-task-definition: stage: deploy script: - sed -i -e s/SED_TARGET_IMAGE_TAG/$IMAGE_TAG/g task-definition.json - TASK_DEF_ARN=$(aws ecs register-task-definition --cli-input-json file://task-definition.json --query 'taskDefinition.taskDefinitionArn' --output text) - aws ecs update-service --cluster $AWS_ECS_CLUSTER_NAME --service $AWS_ECS_SERVICE_NAME --task-definition $TASK_DEF_ARN
task-definition.json
{ "containerDefinitions": [ { "name": "gitlab-cicd-example-container", "image": "<AWS Account ID>.dkr.ecr.ap-northeast-1.amazonaws.com/gitlab-cicd-example-ecr:SED_TARGET_IMAGE_TAG", "cpu": 0, "memoryReservation": 128, "portMappings": [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ], "essential": true, "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "/ecs/logs/gitlab-cicd-example-ecs-group", "awslogs-region": "ap-northeast-1", "awslogs-stream-prefix": "gitlab-cicd-example" } } } ], "family": "gitlab-cicd-example-task", "executionRoleArn": "arn:aws:iam::<AWS Account ID>:role/gitlab-cicd-example-task-execution-role", "networkMode": "awsvpc", "requiresCompatibilities": [ "FARGATE" ], "cpu": "256", "memory": "512" }
少しだけ補足します。
.gitlab-ci.yml:
※参考:
ここまででCI/CDを動かす準備は完了です。
GitLab CI/CDの検証
ここからGitLab CI/CDの検証をします。
まずはProject中のファイルを修正します。ここではindex.htmlを修正しました。
修正後、CI/CDパイプラインの実行を確認し、しばらくすると完了します。実行されたパイプラインを確認すると、 build-image deploy-task-definition のJobが実行されているのを確認できます。
Amazon ECSの画面を見ると、タスク・サービスが更新される様子も確認できます。しばらくたつと、問題なければ起動していることを確認できます。
最後にAmazon ELBのURLにアクセスすると、修正した内容のテキストが表示されているのを確認できます。
