Testing and Environment Variables

As developers have moved hosting applications to cloud-based solutions like Heroku, a number of patterns for developing and deploying applications emerged. Perhaps the most well-known set of methodologies is the Twelve-Factor App, which outlines areas in which applications become difficult to maintain and what to do to improve them.

One area which impacts developers directly in the code we write and test is application configuration. Because we deploy across multiple environments (e.g. staging and production) with different sets of configurations for services like Stripe and Segment, and because often these are critical aspects of our application, we want to ensure things work correctly.

There are a few routes we can take to test the code using environment variables, but none are ideal.

Rely on the values of the environment variables defined by our application

Most often, these environment variables are defined either in the .env or config/environments/test.rb files. Testing against these values introduces mystery guests, as any values we’re testing against are defined outside of the specs themselves.

# config/environments/test.rb

ENV["TWILIO_CALLER_ID"] = "+15555551212"

# spec/models/calls/call_initiator_spec.rb
require "spec_helper"

describe Calls::CallInitiator do
  it "creates a new call with the appropriate data" do
    call_creator = double("calls", create: nil)
    initiator = Calls::CallInitiator.new("555-555-1234", call_creator)

    initiator.run

    call_data = {
      from: "+15555551212",
      to: "555-555-1234",
    }
    expect(call_creator).to have_received(:create).with(call_data)
  end
end

Override environment variables on a per-test basis

Another common way to test environment variables is by overriding them on a per-test basis. This introduces additional complexity by adding additional setup and teardown steps.

# spec/models/calls/call_initiator_spec.rb
require "spec_helper"

describe Calls::CallInitiator do
  it "creates a new call with the appropriate data" do
    cached_twilio_caller_id = ENV["TWILIO_CALLER_ID"]
    ENV["TWILIO_CALLER_ID"] = "+15555551212"

    call_creator = double("calls", create: nil)
    initiator = Calls::CallInitiator.new("555-555-1234", call_creator)

    initiator.run

    call_data = {
      from: "+15555551212",
      to: "555-555-1234",
    }
    expect(call_creator).to have_received(:create).with(call_data)

    ENV["TWILIO_CALLER_ID"] = cached_twilio_caller_id
  end
end

Because ENV contains global state, and because there are no expectations about which other tests are relying on this state, we must always cache and reassign state every time we modify ENV for a test.

Stub ENV

Stubbing ENV is another option (at least at the unit level) which allows us to control the values. One added benefit is mocking and stubbing libraries traditionally handle cleaning up stubs during the teardown phase.

# spec/models/calls/call_initiator_spec.rb
require "spec_helper"

describe Calls::CallInitiator do
  it "creates a new call with the appropriate data" do
    allow(ENV).to receive(:[]).with("TWILIO_CALLER_ID").and_return("+15555551212")

    call_creator = double("calls", create: nil)
    initiator = Calls::CallInitiator.new("555-555-1234", call_creator)

    initiator.run

    call_data = {
      from: "+15555551212",
      to: "555-555-1234",
    }
    expect(call_creator).to have_received(:create).with(call_data)
  end
end

I’ve always been a fan of following “Don’t mock what you don’t own”, and in the case of ENV (part of Ruby’s core library), we don’t own it (even though I’d consider its interface to be fairly stable).

Explicitly access keys with ENV#fetch

By using ENV#fetch instead of ENV#[] to retrieve values in the code we’d be testing, we reduce likelihood of misspellings or mis-configurations. This doesn’t guarantee variables are used correctly; one example I’ve seen firsthand was the same value (admin and support email addresses) assigned to two environment variables, and misused in the mailer. When the environment variable was updated on production (after finding the bug), one email was emailed to the wrong group of people.

Use Climate Control

Climate Control is a gem which handles the above case of modifying environment variables on a per-test basis. It avoids mystery guests, doesn’t stub ENV, and (with arbitrarily strange strings!) provides a high level of confidence that the appropriate environment variables are being used correctly. It’s likely most applicable in unit and integration level tests, since we’ll likely be using fakes at the acceptance level.

Let’s see it in action:

# spec/models/calls/call_initiator_spec.rb
require "spec_helper"

describe Calls::CallInitiator do
  it "creates a new call with the appropriate data" do
    ClimateControl.modify TWILIO_CALLER_ID: "awesome Twilio caller ID" do
      call_creator = double("calls", create: nil)
      initiator = Calls::CallInitiator.new("555-555-1234", call_creator)

      initiator.run

      call_data = {
        from: "awesome Twilio caller ID",
        to: "555-555-1234",
      }
      expect(call_creator).to have_received(:create).with(call_data)
    end
  end
end

Overriding environment variables only within the block ensures state is reset accordingly and environment variable values are immediately obvious to developers. Because of this, Climate Control fosters moving more configuration into ENV by making it easier to test, resulting in more adherence to the twelve-factor app methodology.