---
title: Caching API Requests
teaser:
tags: web,ruby
author: Dan Croak
published_on: 2013-11-29
---

When making requests to an external service's API, some requests will frequently occur with the same parameters and return the same result. If we cache our request or response, we can reduce HTTP requests, which can improve performance and avoid hitting rate limits.

The [APICache][apicache] Ruby gem is a good choice for caching the <abbr title="Application Programming Interface">API</abbr> responses (typically JSON) in any [Moneta][moneta] store, such as Memcache or Redis.

[apicache]: https://github.com/mloughran/api_cache
[moneta]: https://github.com/minad/moneta

However, we don't always need to cache the entire <abbr title="Application Programming Interface">API</abbr> response. We can save space, avoid adding the operational overhead of Memcache or Redis, and avoid repeating the <abbr title="JavaScript Object Notation">JSON</abbr> parsing step if we cache only the URL requested.

## Foursquare venue search

In the following example, our app only needs a venue's name, latitude, longitude, and street address. We'll get the data from Foursquare's venue search <abbr title="Application Programming Interface">API</abbr> by category ("restaurant") and neighborhood ("The Mission").

    url = Foursquare.new(category, neighborhood).venue_search_url

    ApiRequest.cache(url, Foursquare::CACHE_POLICY) do
      # make a GET to the URL
      # parse JSON
      # create or update venue name, lat, lon, street address
    end

The first time this code runs for a venue search for restaurants in the Mission,
`ApiRequest` will save the URL to the database and the block will be executed.

Whenever this runs again for a venue search for restaurants in the Mission, as
long as it is within Foursquare's 30 day cache policy, the block won't be
executed and expensive work will be avoided.

## The internals

It's a pretty simple pattern and the code to make it happen is also straightforward.

Here's the database migration:

    class CreateApiRequests < ActiveRecord::Migration
      def change
        create_table :api_requests do |t|
          t.timestamps null: false
          t.text :url, null: false
        end

        add_index :api_requests, :url, unique: true
      end
    end

The index improves performance of future lookups and enforces uniqueness of the
URL.

Here's the `ApiRequest` model:

    class ApiRequest < ActiveRecord::Base
      validates :url, presence: true, uniqueness: true

      def self.cache(url, cache_policy)
        find_or_initialize_by(url: url).cache(cache_policy) do
          if block_given?
            yield
          end
        end
      end

      def cache(cache_policy)
        if new_record? || updated_at < cache_policy.call
          update_attributes(updated_at: Time.zone.now)
          yield
        end
      end
    end

We've kept this model generic enough that it can be used with other <abbr title="Application Programming Interface">API</abbr>s, not just Foursquare. We [inject a cache policy dependency][dependency] that must respond to `call`. This allows us to pass in [`ActiveSupport`'s nice `Numeric`][numeric] methods like `30.days.ago` and have them execute at runtime.

[dependency]: https://thoughtbot.com/blog/ruby-science-dependency-injection-inlining-classes/
[numeric]: http://api.rubyonrails.org/classes/Numeric.html

Here's the `Foursquare` model:

    class Foursquare
      BASE_URL = 'https://api.foursquare.com/v2/'
      CACHE_POLICY = lambda { 30.days.ago }

      attr_reader :category, :neighborhood

      def initialize(category, neighborhood)
        @category = category
        @neighborhood = neighborhood
      end

      def venue_search_url
        BASE_URL + 'venues/search?' + {
          categoryId: category_id,
          client_id: ENV['FOURSQUARE_CLIENT_ID'],
          client_secret: ENV['FOURSQUARE_CLIENT_SECRET'],
          limit: 50,
          ll: lat_lon,
          radius: 800,
          v: '20130118'
        }.to_query
      end

      private

      def category_id
        category.foursquare_id
      end

      def lat_lon
        "#{neighborhood.lat},#{neighborhood.lon}"
      end
    end

In this example, we chose to build the URL ourselves, using [`ActiveSupport`'s
`Hash#to_query`][to_query] and pulling our client ID and secret in from
[environment variables][env].

[to_query]: http://api.rubyonrails.org/classes/Hash.html#method-i-to_param
[env]: http://12factor.net/config

## What's next

If you found this useful, you might also enjoy:

* [Handling API Rate Limits by Retrying Requests in Background Jobs][background]
* [Geocoding on Rails][geocoding]
* [How to Efficiently Handle Large Amounts of Data on iOS Maps][ios]

[background]: https://thoughtbot.com/blog/handling-api-rate-limits/
[geocoding]: http://geocodingonrails.com
[ios]: https://thoughtbot.com/blog/how-to-handle-large-amounts-of-data-on-maps/