AWS Platform Guide

Building CI/CD Pipelines

There are many continuous integration and deployment tools available. The process for using these tools is fairly similar:

• When a change to the application source code is merged, a new container image is built for the application and pushed to an image repository.
• When changes are pushed to a deploy branch, the latest container image is substituted into the manifests for the application and the manifests are applied to the appropriate cluster.

We prefer to use GitHub Actions to deploy our applications to AWS.

Some of our clients also use AWS Code Pipeline to deploy.

GitHub Actions

Some of our clients deploy applications using GitHub Actions.

Continuous Integration

For each application repository, resources related to the CI Job should be provisioned in the Operations Account.

infra/
  applications/
    APPLICATION/
      operations/

In order to build Docker images for an application, you’ll need:

• A Dockerfile. This can be kept in the application repository.

• An ECR repository. You can use the ECR repository Terraform module to set this up.

  module "ecr_repository" {
    source = "github.com/thoughtbot/terraform-eks-cicd//modules/ecr-repository?ref=v0.1.0"
  
    name = "example-org/example-app"
  
    # AWS accounts allowed to pull this image
    workload_account_ids = [
      "123456789010", # Sandbox account ID
      "123456789010", # Production account ID
    ]
  }

• An IAM OIDC provider which trusts your GitHub Actions workflow. If you used the landing zone template, one will already be created in the Operations account and you can locate its ARN using the SSM parameter /GitHubActions/OIDCProviderArn.

• An IAM role that can be assumed by GitHub using OIDC and access ECR in order to push your built Docker image.

  module "ecr_role" {
    source = "github.com/thoughtbot/terraform-eks-cicd//modules/github-actions-ecr-role?ref=v0.1.1"
  
    allow_github_pull_requests = true
    ecr_repositories           = [module.ecr_repository.name]
    github_branches            = ["main", "production"]
    github_organization        = "example-org"
    github_repository          = "example-app"
    iam_oidc_provider_arn      = data.aws_ssm_parameter.iam_oidc_provider_arn.value
    name                       = "github-actions-ecr-example"
  
    depends_on = [module.ecr_repository]
  }
  
  data "aws_ssm_parameter" "iam_oidc_provider_arn" {
    name = "/GitHubActions/OIDCProviderArn"
  }

• Once this root module is applied, create GitHub Actions job(s) to build Docker images and generate manifests. These Actions can be triggered when pull requests are opened by defining triggers in the Job itself.

GitHub Actions will start building and pushing Docker images for your application to ECR whenever developers open pull requests.

Continuous Deployment

One of the application roles for each stage of the software development lifecycle is a deploy role, which can be used by CI/CD pipelines to deploy new versions of the application. You can assume this role from your deployment GitHub workflow using the configure-aws-credentials action.

You can use the k8s-bake action to generate Kubernetes manifests for your application. There is a helm-rails Helm chart you can use with this action to deploy a Rails application.

Finally, you can use the k8s-deploy action to apply the updated manifests to your cluster.

Once configured, deployments will kick off in GitHub Actions whenever you push code to the appropriate branch in your source repository.

Accessing another GitHub repository

If for any reason, your workflow needs to access another GitHub repository different than its source, for example:

1. Pulling application code from CodeBuild, or
2. Pulling manifests from a separate repo than the application code.

In these cases, the CI/CD workflow will require a Personal Access Token with the correct permissions set as a secret in the workflow source.

To generate a personal access token (PAT)

1. Sign in as the CI GitHub user (stored in GitHub)
2. Access Settings via the user picture dropdown in the upper right corner
3. From the left navigation menu, access <> Developer settings(all the way at the bottom at time of writing)
4. From the left navigation menu in Developer settings, access Personal Access Token
   1. If presented with a dropdown, select Tokens (classic)
5. You may now generate a new PAT with the Generate new token button above the list of PATs already in existence
6. In the popup:
   1. Notate the purpose of the token
   2. Choose its expiration timeframe
   3. Ensure that the token has the correct permissions by selecting the checkbox next to repo.

   

7. Click the Generate token green button at the bottom

Setting the token secret

1. In the repo that is running the CI/CD workflow, navigate to Settings with the horizontal navigation menu
2. From the left navigation menu, go to Security > Secrets > Actions
3. Create a New repository secret with the green button at the top
4. Fill in the name and value of your token
5. Once saved, you’ll be able to retrieve the value with secrets.NAME_OF_SECRET

AWS Code Pipeline

Some of our clients deploy applications using CodePipeline and CodeBuild.

Common Resources

In the infrastructure repository, create a root module for common CI/CD resources:

infra/
  cicd-common/

In order to deploy CI/CD pipelines on AWS, you will need the following:

• An artifact bucket for your organization’s build artifacts. You can provision one using the artifact bucket Terraform module.
• A CodeBuild credential to trigger builds when pull requests are opened. You can set this up using the CodeBuild credential Terraform module.
• A CodeStar connection to trigger pipeline runs when pull requests are merged. You can set this up using the CodeStar connection Terraform module.

These resources should be provisioned in the Operations account.

After adding these modules to your root module, apply the module. Once the module has completed successfully, you will need to sign into the AWS Management Console. Find the pending GitHub connection and follow the instructions in the AWS Developer Tools Console User Guide to complete the connection.

Continuous Integration

For each application repository, create CodeBuild projects to build Docker images and generate manifests. These projects can be triggered when pull requests are opened using webhooks. Build projects and related resources should be provisioned in the Operations account.

infra/
  ci/
    APPLICATION/

In order to build Docker images for an application, you’ll need:

• A Dockerfile. This can be kept in the application repository.
• A buildspec for building and pushing Docker images. This can also be kept in the application repository.
• An ECR repository. You can use the ECR repository Terraform module to set this up.
• A CodeBuild project for building Docker images. You can create one using the ECR project Terraform module.
• A manifests repository to store configuration and manifests for your application.
• A buildspec for generating application manifests. This can be stored in the manifests repository and should contain Kustomize or Helm commands for generating Kubernetes manifests. This buildspec must produce YAML files as artifacts that can later be applied to the cluster to deploy the application.
• A CodeBuild project for generating application manifests. You can create one using the manifests project Terraform module.

Once this root module is applied, CodeBuild will start building and storing Docker images for your application whenever developers open pull requests.

Continuous Deployment

For each stage of the software development lifecycle, create a CodePipeline pipeline to deploy the latest images and manifests to the cluster. This pipeline can be triggered when pull requests are merged using webhooks. Pipelines should be provisioned in the Operations account.

infra/
  cd/
    APPLICATION/

• An IAM role for deploying to each cluster. You can create one using the deploy role Terraform module. This role must be provisioned in the same Workload account as the cluster.
• A buildspec for applying manifests to the cluster. You can use kubectl and other commands to apply the manifests generated by your manifests project.
• A CodeBuild project for applying manifests to the cluster. You can set one up using the deploy-project Terraform module.
• A CodePipeline pipeline for deploying the latest Docker images and manifests to the cluster. You can create pipelines using the cicd-pipeline Terraform module.

Once a pipeline is provisioned, the CodeBuild projects for building Docker images and manifests will be triggered whenever a pull request is merged. The artifacts from the manifest project will be provided to the deploy project, which can apply the manifests to the cluster.