---
title: Faking APIs in Development and Staging
teaser: Use fake API servers everywhere, not just in tests.
tags: web,testing,rails,ruby
author: Joël Quenneville
published_on: 2015-07-06
---

You are a developer for startup called _Movie Social Network++_, building a
social network for movie aficionados. Several features are dependent on data
about various movies, actors, or directors. You do a quick Google search to
figure out where to get this sort of data and come across `movie-facts.com`. They
have already gathered all this data and have a team constantly keeping it up to
date. They offer to make all this data available to you via their API for a
moderate fee.

This seems like a great solution. The team agrees and one of the founders
reaches out to `movie-facts.com` to negotiate a contract.

## Test-driving the first feature

In the meantime, you start working on a feature that depends on this API:

> When I reference a movie title in a post, I want to see some quick facts
> (director, actors).

In test-driven fashion, you start by writing a feature test:

```ruby
visit root_path

click_on "New Post"
fill_in :post, with "Jurassic World was soooo good!!!"
click_on "Post"

info_box = find(".info-box")
expect(info_box).to have_content("Chris Pratt")
```

As you try to make this test pass, you eventually need to actually fetch the
data from the API. You decide to isolate all of the API-related code in an
adapter class:

```ruby
# app/models/movie_database.rb

class MovieDatabase
  BASE_URL = "http://api.movie-facts.com".freeze

  def actors_for(movie:)
    HTTParty.get(BASE_URL + "/movies/#{movie.id}/actors", format: :json)
  end
end
```

Now you get an error from Webmock saying that external requests are blocked in
tests. You decide to build a test fake for the `movie-facts.com` API using
Sinatra.

```ruby
# spec/support/fake_movie_facts.rb

module FakeMovieFacts
  class Application < Sinatra::Base
    get "/movies/:movie_name/actors" do
      {
        actors: [
          {
            name: "Actor 1",
            character_played: "Character 1"
          },
          {
            name: "Actor 2",
            character_played: "Character 2"
          }
        ]
      }.to_json
    end
  end
end
```

There are a few ways to load up a fake in a test.

* [Using
  Webmock](https://thoughtbot.com/blog/how-to-stub-external-services-in-tests)
* [Via
  middleware](https://thoughtbot.com/blog/faking-remote-services-with-rack-test)
* With [capybara-discoball](https://github.com/thoughtbot/capybara_discoball)

Capybara Discoball is a gem that allows you to boot other Rack apps in the
background of a feature test run. Once you have an app running, the adapter can
be pointed to that app.

You decide to use Capybara Discoball because we don't need to stub anything. The
application is literally making HTTP requests to APIs that are running on your
localhost.

You add configuration to allow Capybara Discoball to run your fake
`movie-facts.com` API:

```ruby
# spec/support/fake_movie_facts_runner.rb

FakeMovieFactsRunner =
  Capbybara::Discoball::Runner.new(FakeMovieFacts::Application) do |server|
    MovieDatabase.base_url = "#{server.host}:#{server.port}"
  end
```

Then in the spec helper:

```ruby
# spec/spec_helper.rb

FakeMovieFactsRunner.boot
```

The `MovieDatabase` adapter doesn't currently allow its base url to be changed
so you change it to use a class accessor rather than a hard coded constant, and
default it to the `movie-facts.com` API.

```ruby
# app/models/movie_database.rb

class MovieDatabase
  cattr_accessor :base_url
  base_url = "http://api.movie-facts.com"

  def actors_for(movie:)
    HTTParty.get(self.class.base_url + "/movies/#{movie.id}/actors", format: json)
  end
end
```

Now your tests pass. Mission accomplished!

## Fakes in development

Excited by this new feature, you grab a colleague to show off the new
functionality your local machine. To your chagrin, it doesn't work at all.
Instead, you get unauthorized errors from the `movie-facts.com` API. You explain
to your colleague that the feature _does_ work in the tests because you are
using a fake.

She nods and makes a surprising suggestion: Why not also use the fake in
development? You think about it briefly and agree that this is probably the
easiest way to get this feature running in development.

You open a new terminal and run the following:

<kbd>ruby spec/support/fake_movie_facts.rb</kbd>

This boots up a Sinatra server on `localhost:4567`.

Now you just need to point the adapter to this url. In addition to allowing the
base url to be changed at runtime, you decide to default the value to an
environment variable instead of the hardcoded url you are currently using.

```ruby
# app/models/movie_database.rb

class MovieDatabase
  cattr_accessor :base_url
  self.base_url = ENV.fetch("MOVIE_FACTS_API_BASE_URL")

  def actors_for(movie:)
    HTTParty.get(self.base_url + "/movies/#{movie.id}/actors", format: :json)
  end
end
```

You try booting up a Rails server again:

<kbd>MOVIE_FACTS_API_BASE_URL=localhost:4567 rails server</kbd>

You open the app in your browser and create a post that references a movie.
"HEY, IT WORKS!"

## Fakes on staging

Hearing your shout of jubilation, your boss comes over to admire your
achievement. He mentions that it would be great if you could deploy this feature
to staging so that he can demo it at an investor meeting tomorrow.

You scratch your head. The real `movie-facts.com` API won't be available in time
you explain to your boss. The demo he's just seen used a test fake to simulate
the real API. "Can you do the same thing on the staging server?", he asks. You
agree that this should be possible and timebox an hour to explore the options.

You need to run both the main application and the fake and have them be able to
communicate with each other via HTTP. The simplest way to accomplish this is to
have each be its own Heroku application. However, extracting the fake into its
own app leads to a dilemma. The tests still need the fake in order to pass. How
do you share the code with the main application while still making it easy to
deploy as its own application?

After discussing with your colleague, you decide on the following architecture:

* Extract to a separate git repository
* Wrap it as a gem so it can be used by the tests
* Provide a
  [`config.ru`](https://github.com/rack/rack/wiki/(tutorial)-rackup-howto) file
  in the root of the gem so it can be deployed on Heroku.

Initializing a new gem with <kbd>bundle gem fake_movie_facts</kbd> will generate
the following structure (via <kbd>tree fake_movie_facts</kbd>):

```tree
fake_movie_facts
├── Gemfile
├── README.md
├── Rakefile
├── bin
│   ├── console
│   └── setup
├── fake_movie_facts.gemspec
├── lib
│   ├── fake_movie_facts
│   │   └── version.rb
│   └── fake_movie_facts.rb
└── spec
    ├── fake_movie_facts_spec.rb
    └── spec_helper.rb
```

You extract `FakeMovieFacts::Application` from the main application into
`fake_movie_facts/lib/fake_movie_facts/application.rb` and add a `config.ru`
file to the root of the repo:

```ruby
# config.ru

$LOAD_PATH << File.expand_path("../lib", __FILE__)
require "fake_movie_facts/application"

run FakeMovieFacts::Application
```

Heroku will automatically pickup on the `config.ru` file when you deploy the
gem. You can also run it locally via the `rackup` command.

You deploy both apps Heroku (staging) and edit the main application's
environment variables to point to the gem:

<kbd>heroku config:set MOVIE_FACTS_API_BASE_URL=http://fake-movie-facts.herokuapp.com --remote staging</kbd>

A test drive confirms that the various components are correctly working with
each other. Investor demo, here we come!

## Advanced Fakes

With your feature complete, you pull the next one from the top of the queue:

> As a user, I want to be able to subscribe to news for an upcoming movie and
> have it piped into my news feed.

The task mentions that movie-news.com has a free API that allows you to
subscribe to events for a given movie via a webhook. As previously, you
test-drive via a feature spec and run into Webmock's "external URL" error, so
you write a fake:

```ruby
module FakeUpcomingMovieEvents
  class Application < Sinatra::Base
    post "subscriptions/:movie_name" do
      successful_subscription.to_json
    end

    private

    def successful_subscription
      {
        subscription_id: "123",
        movie_subscribed_to: params[:movie_name]
      }
    end
  end
end
```

Great! This fixes your test failure. You are now faced with another problem
though. You need some way to trigger an event in your tests. Thinking ahead, you
realize that you will probably also need a way to do this in development and
staging for demo purposes. Since you have no control over `movie-events.com`'s
event API, you decide to have the fake automatically trigger the webhook right
after each subscription.

```ruby
module FakeUpcomingMovieEvents
  class Application < Sinatra::Base
    post "subscriptions/:movie_name" do
      trigger_webhook

      successful_subscription.to_json
    end

    private

    def trigger_webhook(callback_url)
      HTTParty.post(params[:callback_url], event_payload_json)
    end

    def event_payload_json
      {
        event_type: "New Trailer",
        url: "http://video-sharing-platform.com/123"
      }.to_json
    end

    # ...
  end
end
```

This results in an unexpected bug. The subscription deadlocks because the
subscription endpoint can't return until the webhook request succeeds but the
webhook can't be processed by the main app until the subscription request
succeeds. This catch-22 situation is caused by having the two tasks be
synchronous (blocking).

After giving it some thought you decide to try and background the webhook task.
Adding a queueing system such as DelayedJob to a fake seems a bit heavy handed
so you try to build something using threads:

```ruby
module FakeUpcomingMovieEvents
  class Application < Sinatra::Base
    post "subscriptions/:movie_name" do
      async do
        trigger_webhook(params[:callback_url])
      end

      successful_subscription.to_json
    end

    private

    def async
      Thread.new do
        sleep ENV.fetch("WEBHOOK_DELAY").to_i
        yield
      end
    end

    # other methods
  end
end
```

This fixes your tests. You extract a gem + `config.ru` as with the previous fake
and deploy to heroku/staging. You configure the fake to trigger a "New Trailer"
event 15 seconds after subscribing to a movie's events. Tomorrow's demo should
be a good one.

## Conclusion

Fakes are great for testing an application that interacts with 3rd party APIs.
Their usefulness extends beyond just testing however. In situations where the
API is not currently available, doesn't have a sandbox mode, or you need more
control over events that it emits, fakes can be a great solution in development
and staging as well.

Packaging them as Rack-compatible apps plus a config.ru file wrapped in a gem
makes it easy to share across environments and servers. It also makes it easy to
open source and develop fakes for popular APIs with the community, even though
your main app remains closed source.