Learning Terraform - A hands-on, open source contribution

Senior Developer Clarissa Borges joined Development Team Lead Fernando Perales for a live-streamed coding session working on an open source Terraform module. Clarissa and Fernando worked on moving an OpenSearch database module to thoughtbot’s Terraform module for databases, and they showed live how we maintain our Terraform modules. They discussed some Terraform basics, which you can learn more about in this post. You can also watch the full replay on YouTube.

Terraform 101

Terraform is a tool for infrastructure-as-code that allows you to define and provision resources. You can define a database, a cluster, almost anything in your infrastructure can be defined and provisioned using Terraform.

Because Terraform uses reusable modules, it allows you to be consistent in how you provision your resources, making for faster deployments and easier maintenance.

It’s a worthwhile time investment: once you have it stood up, then it’s a great way to track and document your resources, so you can easily check what you have or onboard new team members.

Getting started

Each configuration for your infrastructure could follow this file structure:

infrastructure/           # Your infrastructure repository
├── cluster/              # Terraform configuration for your infrastructure cluster
│   ├── production/       # Production environment configuration
│   │   ├── README.md     # Auto-generated Terraform docs for this module
│   │   ├── main.tf       # Core resource definitions - infrastructure, modules, data sources
│   │   ├── outputs.tf    # Declares values to output after 'terraform apply'
│   │   ├── variables.tf  # Input variable definitions and defaults
│   │   └── versions.tf   # Define Terraform and provider versions for compatibility
│   ├── sandbox/          # Non-production environment configuration
│   │   ├── README.md
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   ├── variables.tf
│   │   └── versions.tf
...

The main.tf file is the one we’ll most often add things to - it’s the heart of our Terraform configuration. In there, you’ll describe the actual infrastructure you’re managing, which includes things like compute instances, VPCs, load balancers, and more. It can also include module calls if you’re reusing shared logic.

If you’re following a modular approach (which we highly recommend!), main.tf often becomes a lightweight orchestrator that wires reusable modules with specific inputs required by your environment or application.

Here’s an example of what our main.tf file would look like for defining our cluster resources:

module "cluster" {
  source = "github.com/thoughtbot/flightdeck//aws/cluster?ref=v0.12.1"

  # Unique name for this EKS cluster.
  name = "example-production-v1"

  # Version of Kubernetes for this cluster. Must be supported by EKS.
  k8s_version = "1.31"

  node_groups = {
    # You need at least one node group definition.
    default = {
      instance_types = ["t3.large"]
      max_size       = 30
      min_size       = 2
    }
  }
}

The Terraform cycle

Once you are in your Terraform configuration directory and have described a module in your main.tf file, you can begin the three-step Terraform cycle.

1. terraform init to install the providers, initialize modules, and configure the backend.
2. terraform plan to review what changes Terraform will perform against your infrastructure.
3. terraform apply to execute the changes and provision the resources.

We recommend avoiding terraform apply from your local machine until changes have been pushed to the main branch of your infrastructure repository. It’s surprisingly easy to forget to push your changes, which can lead to someone else accidentally overriding your work. If you use remote version control in your workflow for managing your infrastructure, you can also get feedback from your peers and facilitate collaboration.

In addition, we strongly recommend setting up a CI/CD workflow to automate formatting, validation, planning, and even applying your infrastructure changes. This reduces the risk of human error, enforces consistency, and makes your infrastructure workflow just as reliable as your application code.

And it’s as simple as that! Once your terraform apply completes successfully, your resources will be live and available in your infrastructure.

In the live-coding example, we used a real thoughtbot infrastructure that involved the OpenSearch module. To ensure there wouldn’t be any differences in the resulting infrastructure as a result of our moving the module, we used terraform init and terraform plan.

How thoughtbot uses Terraform

Most of our platform engineering projects at thoughtbot use Terraform modules, which we have developed over the years, so we can very easily and quickly deploy complete infrastructures. We’ve made them open source so anyone can reuse or adapt them to hit the ground running with their own infrastructure project.

Flightdeck is thoughtbot’s open source platform engineering set of modules that allow you to create a complete infrastructure from scratch. It contains everything we’ve learnt from years of experience building and deploying apps for our clients.

If you use Flightdeck, you can reduce to a week or two what would previously have been months of work. Not to mention the time you will save going forwards when you have to upgrade resources.

You can check our AWS Platform Guide to learn more about how to deploy infrastructures to AWS using our modules.

If this is code, then what about tests, docs and linters?

Terraform has you covered.

You can run tf-docs to auto-generate and auto-update documentation for your module, terraform fmt to lint and format your infrastructure-as-code, and terraform validate to ensure your changes are valid.

We added a formatting check to our CI/CD pipeline using GitHub actions, and a tf-docs step to all our pull requests so changes are documented before being merged.

We don’t typically add unit tests to our infrastructure code. Because Terraform is declarative, it wouldn’t be like other traditional test approaches. Instead, we rely on acceptance tests in Flightdeck to verify that our changes to the modules work as expected.

By the end of the livestream, Clarissa and Fer had created the pull request to the repository, which included moving the module to its new home. That’s now merged in, and you can check it out here Create OpenSearch module.

Let us help scale your infrastructure

If you like what you’ve seen here in terms of how quick and easy infrastructure can be to deploy and maintain, then we’d love to share our expertise with you. Get in touch today to discover how thoughtbot can leverage our past experience to quickly get your team up and running in the cloud.