AWS Platform Guide

Managing Secrets

On AWS, thoughtbot uses AWS Secrets Manager whenever possible to store the secret data. We use the Kubernetes secret storage provider to expose secrets to applications.

AWS Secrets Manager

AWS Secrets Manager is a service for storing, securing, updating, and accessing secrets like passwords, API keys, and cryptographic certificates. A secret can consist of either a single blob of data, or a JSON object consisting of key value string pairs.

You can use Terraform to provision a Secrets Manager secret to store your secret data.

• For things like a database URL, we use a Lambda rotation function which creates a new password in the database each month and then updates the secret. (For example, see thoughtbot/terraform-aws-databases/rds-postgres/admin-login)

• For things like the Rails secret base, we generate a random one in Terraform and populate it there.

  resource "random_password" "secret_key_base" {
    length  = 32
    special = false
  }
  
  module "rails_secret" {
    source = "github.com/thoughtbot/terraform-aws-secrets//secret?ref=v0.4.0"
  
    description = "Secrets for the Rails application"
    name        = "example-app-secret"
  
    initial_value = jsonencode({
      SECRET_KEY_BASE = random_password.secret_key_base.result
    })
  }

• For external tokens that we can’t control, we create an empty secret in Terraform (using thoughtbot/terraform-aws-secrets/secret as source) and populate it by hand in AWS Management Console.

  module "prismic_secret" {
    source = "github.com/thoughtbot/terraform-aws-secrets//secret?ref=v0.4.0"
  
    description = "Secrets for accessing the Prismic API"
    name        = "example-prismic"
  
    initial_value = jsonencode({
      # Fill this in using the SecretsManager UI or CLI
      PRISMIC_ACCESS_TOKEN = ""
    })
  }

Once you’ve safely stored your secret using Secrets Manager, you can add secrets to pods as environment variables or files.

Mounting Secrets

Secrets are functionally identical to config maps, but they can be configured with stricter permissions due to their sensitive nature. Secret manifests are not committed to Git.

The best way to manage secrets on AWS is to store the secret value using AWS Secrets Manager and synchronize the secret to your cluster using the Kubernetes Secret Storage provider.

On AWS, you can synchronize a secret to your cluster by creating a secret provider class:

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
   name: example
spec:
  provider: aws
  secretObjects:
  - secretName: example
    type: Opaque
    data:
    - key: SECRET_KEY_BASE
      objectName: SECRET_KEY_BASE
  parameters:
    objects: |
      - objectName: my-secrets-manager-secret
        objectType: secretsmanager
        jmesPath:
        - path: SECRET_KEY_BASE
          objectAlias: SECRET_KEY_BASE

Once a secret provider class is created, you can mount them similarly to config maps:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-web
  namespace: default
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: example
  template:
    metadata:
      labels:
        app.kubernetes.io/name: example
    spec:

      # Define your secret as a volume using the secrets storage provider
      volumes:
      - name: example
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: example

      containers:
      - name: main

        # Mount a secret as environment variables
        envFrom:
        - secretRef:
            name: example

        # Or mount the volume in your container
        volumeMounts:
        - name: application
          mountPath: /app/config/application.yml
          subPath: application.yml