---
title: 'Appraisal: find out what your gems are worth'
teaser:
tags: news,web,testing,open source,appraisal
author: Joe Ferris
published_on: 2011-10-29
---

Since the introduction of bundler to the Ruby community, dealing with
dependencies has gotten much easier. Almost every library now has a Gemfile that
looks like this:

    source "http://rubygems.org"
    gemspec

This pulls runtime and development dependencies from your project's gemspec and
finds versions that can all agree with each other. Requiring any of your
dependencies will always get the expected version regardless of order, and
updating dependencies can be performed with a single command.

However, there's still one piece missing from this puzzle: how do you make sure
that your library works with all supported versions of your dependencies? A
common example is Rails: now that Rails 3.1 is out, how can you make sure that
your library works with both Rails 3.0.x and Rails 3.1.x?

One side effect of using bundler is that all dependencies are "locked." This
essentially means that, whether you're running your tests on your home computer,
work computer, or continuous integration server, your dependencies are always
the same exact versions. This obviously means that you'll only ever test against
one version of Rails; Yehuda Katz, one of the authors of bundler, has a [partial
answer](http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/)
to this problem:

> "When developing a gem, use the gemspec method in your Gemfile to avoid
> duplication. In general, a gem’s Gemfile should contain the Rubygems source
> and a single gemspec line. Do not check your Gemfile.lock into version
> control, since it enforces precision that does not exist in the gem command,
> which is used to install gems in practice. Even if the precision could be
> enforced, you wouldn’t want it, since it would prevent people from using your
> library with versions of its dependencies that are different from the ones you
> used to develop the gem."

Unfortunately, there are some holes in this practice.

For one, this means that tests that pass on one machine may not pass on another,
even if the library code hasn't changed at all. This can be frustrating, as it
breaks the general expectation that a fresh checkout of any master branch should
have passing tests.

For another, this doesn't reliably exercise your library across versions of your
dependencies; rather, it hopes that you happen to run the tests on enough varied
environments between each release that you happen to catch any regressions.

If locking your dependencies hides errors, and not locking your dependencies
creates them, how can you reliably run your tests on different versions of your
dependencies without creating an unstable development environment? The answer is
[appraisal](https://github.com/thoughtbot/appraisal).

Appraisal runs your tests across configurable, reproducible scenarios that
describe variations in dependencies. As an example, if you need to test
compatibility with several versions of Rails, you can use an Appraisals file
like this:

    appraise "3.0" do
      gem "rails", "~> 3.0.11"
    end

    appraise "3.1" do
      gem "rails", "~> 3.1.1"
    end

You can include as many appraise blocks as you want, and you can be as specific
or loose with requirements as you like. Running `rake appraisal:install` will
generate separate bundler-compatible Gemfiles for each scenario, combining the
base requirements from your main Gemfile with the changes in each scenario. You
can run any rake task across every scenario using the "appraisal" task:

    # Run your specs on Rails 3.0 and 3.1
    rake appraisal spec

When using Appraisal, we recommend that you check your Gemfile.lock into version
control, as well as the gemfiles directory generated by Appraisal. This will
ensure that your tests are always green when you check out a fresh copy. It also
allows you to create reproducible regression tests for version-specific bugs.

We've successfully used Appraisal on numerous projects this year. If you'd like
to see real world examples, read the Appraisals files for
[Clearance](https://github.com/thoughtbot/clearance/blob/master/Appraisals),
[Factory
Bot](https://github.com/thoughtbot/factory_bot/blob/master/Appraisals),
[Capybara
Webkit](https://github.com/thoughtbot/capybara-webkit/blob/master/Appraisals),
[Copycopter
Client](https://github.com/copycopter/copycopter-ruby-client/blob/master/Appraisals),
and [Paperclip](https://github.com/thoughtbot/paperclip/blob/master/Appraisals).

Do you have a gem that supports multiple version of Rails or any other library?
Find out what your gem is worth: [install
appraisal](https://rubygems.org/gems/appraisal).