Shoulda has long been one of our most useful and popular open source projects, and it continues to serve us well as we use it daily. However, there have been some changes over the past year in the way that we use Shoulda, and these changes have led to some decisions about its future. In our post about the Rails 3 roadmap, we briefly covered the changes we made in Shoulda 2.11 as well as our intentions for Shoulda 3. However, we wanted to go over these changes in depth and explain some of our motivation.
Putting things in context
Shoulda has two main components: a test context framework, and a set of helpers for testing Rails applications. The context framework was written because we found the method-based style of Test::Unit tedious, but certain objections were raised to the way RSpec was implemented, and we had existing test suites that could not easily be converted to RSpec. The Rails helpers were written in the form of Shoulda-based “macros” for convenience.
While this approach worked great for those of us using Test::Unit, this meant that the Rails helpers were only accessible in Shoulda-based test suites despite relying on very little in Test::Unit or the Shoulda context framework. In order to bring Shoulda’s Rails helpers to a wider community, they were rewritten as a set of RSpec-compatible matchers, and the existing “macros” were rewritten to use them.
A foot in the other camp
As things in the Cucumber and RSpec community started to pick up, we found more and more that Test::Unit users weren’t getting the latest and greatest. Most of the new assertion libraries we found were written as RSpec expectations, and several improvements to RSpec’s context and matcher DSL made it look too appealing to ignore. We tried out RSpec on a few projects, and found that we really didn’t have to give anything up - Shoulda’s macros were fully supported as matchers - and we found ourselves enjoying RSpec more than Test::Unit.
It was at this point that we decided that Shoulda’s future would focus on RSpec. However, we weren’t going to ditch Test::Unit. We had many existing projects that used Test::Unit and Shoulda, and we certainly didn’t want to abandon the fantastic Shoulda community. After several experiments, we decided not to convert our existing Shoulda test suites to RSpec, largely because they didn’t take advantage of all that RSpec had to offer and didn’t come out any better in the end. This meant that we wanted to keep the context framework functioning. However, we also found ourselves reluctant to spend time and effort on a context framework that we didn’t want to use. Therefore, we needed to come up with a way to keep the matchers working with Test::Unit and make sure the context framework continued to function reasonably, but prevent the overhead of maintaining compatibility with two frameworks.
Rails 3
Rails 3 presented another challenge. Although some of the existing matchers just worked, the general strategy for testing them had to change with Rails 3, and some matchers required conditional forks in the implementations and tests. This was more complicated in the existing macro test suite, and maintaining two test Rails roots for backwards compatibility could potentially slow us down and make it harder to add new features. We preferred the unit-style tests in place for the matchers, and wanted to avoid the rails_root-style test going forward if possible.
Embracing matchers
After refactoring all of the macros to use matchers, we found that the macros all followed a basic pattern: convert a number of hash options into RSpec’s “fluid” syntax, and then create a test that performed one assertion based on a matcher. Based on this discovery, we changed Shoulda’s should method so that it could accept a matcher instead of a test name:
# macro style
should_have_many :users
should_ensure_length_at_least :name, 3
should_not_allow_values_for :isbn, "bad"
should_set_the_flash_to /thank you/i
should_not_assign_to :user
# matcher style
should have_many(:users)
should ensure_length_of(:name).is_at_least(3)
should_not allow_value("bad").for(:isbn)
should set_the_flash.to(/thank you/i)
should_not assign_to(:user)
Converting to this syntax would mean that we no longer needed to support both a set of macros and a set of matchers - all matchers essentially function as macros. In fact, this includes most non-Shoulda macros. We currently support both syntaxes, but you’ll receive a deprecation warning for the older macro style. We’ll also be removing the macros entirely starting with version 3, so now is definitely the time to start embracing matchers.
Separating Shoulda’s contexts
We’re committed to using RSpec, so we don’t want to spend time adding features to Shoulda’s context framework. However, we need it to continue to function for our older test suites, and we understand that some users still prefer the lighter weight Test::Unit framework.
Therefore, we plan on separating Shoulda’s context framework into a separate library starting with version 3. The contexts will still function as always, but we have no plans to further improve this aspect of Shoulda.
Putting it all together
There are a lot of changes here, but they’re not too hard to understand:
- Shoulda is now Rails 3 compatible!
- Shoulda is focusing on the RSpec/Shoulda combination and will primarily support that combination of tools, moving away from Test::Unit
- Shoulda’s “macro” methods are deprecated in favor of using the new should/matcher syntax
- Shoulda’s context framework is moving into a separate library
We’re looking forward to Rails 3 and RSpec 2, and we hope you’re looking forward to Shoulda 3.
