Since the introduction of bundler to the Ruby community, dealing with dependencies has gotten much easier. Almost every library now has a Gemfile that looks like this:
source "http://rubygems.org"
gemspec
This pulls runtime and development dependencies from your project’s gemspec and finds versions that can all agree with each other. Requiring any of your dependencies will always get the expected version regardless of order, and updating dependencies can be performed with a single command.
However, there’s still one piece missing from this puzzle: how do you make sure that your library works with all supported versions of your dependencies? A common example is Rails: now that Rails 3.1 is out, how can you make sure that your library works with both Rails 3.0.x and Rails 3.1.x?
One side effect of using bundler is that all dependencies are “locked.” This essentially means that, whether you’re running your tests on your home computer, work computer, or continuous integration server, your dependencies are always the same exact versions. This obviously means that you’ll only ever test against one version of Rails; Yehuda Katz, one of the authors of bundler, has a partial answer to this problem:
“When developing a gem, use the gemspec method in your Gemfile to avoid duplication. In general, a gem’s Gemfile should contain the Rubygems source and a single gemspec line. Do not check your Gemfile.lock into version control, since it enforces precision that does not exist in the gem command, which is used to install gems in practice. Even if the precision could be enforced, you wouldn’t want it, since it would prevent people from using your library with versions of its dependencies that are different from the ones you used to develop the gem.”
Unfortunately, there are some holes in this practice.
For one, this means that tests that pass on one machine may not pass on another, even if the library code hasn’t changed at all. This can be frustrating, as it breaks the general expectation that a fresh checkout of any master branch should have passing tests.
For another, this doesn’t reliably exercise your library across versions of your dependencies; rather, it hopes that you happen to run the tests on enough varied environments between each release that you happen to catch any regressions.
If locking your dependencies hides errors, and not locking your dependencies creates them, how can you reliably run your tests on different versions of your dependencies without creating an unstable development environment? The answer is appraisal.
Appraisal runs your tests across configurable, reproducible scenarios that describe variations in dependencies. As an example, if you need to test compatibility with several versions of Rails, you can use an Appraisals file like this:
appraise "3.0" do
gem "rails", "~> 3.0.11"
end
appraise "3.1" do
gem "rails", "~> 3.1.1"
end
You can include as many appraise blocks as you want, and you can be as specific
or loose with requirements as you like. Running rake appraisal:install will
generate separate bundler-compatible Gemfiles for each scenario, combining the
base requirements from your main Gemfile with the changes in each scenario. You
can run any rake task across every scenario using the “appraisal” task:
# Run your specs on Rails 3.0 and 3.1
rake appraisal spec
When using Appraisal, we recommend that you check your Gemfile.lock into version control, as well as the gemfiles directory generated by Appraisal. This will ensure that your tests are always green when you check out a fresh copy. It also allows you to create reproducible regression tests for version-specific bugs.
We’ve successfully used Appraisal on numerous projects this year. If you’d like to see real world examples, read the Appraisals files for Clearance, Factory Bot, Capybara Webkit, Copycopter Client, and Paperclip.
Do you have a gem that supports multiple version of Rails or any other library? Find out what your gem is worth: install appraisal.
