Shoulda Matchers 3.0

As summer approaches, we’re wrapping up another new release of Shoulda Matchers. This time around we’re making some backward-incompatible changes. Let’s talk about them now!

Installation changes

The gem no longer detects the test framework you’re using or mixes itself into that framework automatically. History has shown that performing any kind of detection is prone to bugs and more complicated than it should be.

Here are the updated instructions:

• You no longer need to say require: false in your Gemfile.

• You’ll need to add the following somewhere in your rails_helper (for RSpec) or test_helper (for Minitest / Test::Unit):

  Shoulda::Matchers.configure do |config|
    config.integrate do |with|
      # Choose a test framework:
      with.test_framework :rspec
      with.test_framework :minitest
      with.test_framework :minitest_4
      with.test_framework :test_unit
  
      # Choose a library:
      with.library :active_record
      with.library :active_model
      with.library :action_controller
      # Or, choose all of the above:
      with.library :rails
    end
  end
  

If you’re interested in why we made this change, scroll down to the bottom of this post. Otherwise, read on!

(View commit: 1900071)

Support for older Ruby and Rails versions dropped

We’ve dropped support for Rails 3.x, Ruby 1.9.2, Ruby 1.9.3, and RSpec 2. All of these versions have been end-of-lifed. This doesn’t mean that Shoulda Matchers will stop working immediately with these versions, but we won’t accept any pull requests to fix any compatibility issues with them. (View commits: a4045a1, b7fe87a)

Breaking changes to allow_value

There are two changes to allow_value:

• The negative form of the matcher has been changed so that instead of asserting that any of the given values is an invalid value (allowing good values to pass through), we now assert that all values are invalid values (disallowing good values from passing through). This means that the following test which formerly passed will now fail:

  expect(record).not_to allow_value(
    'good value',
    'bad value',
    'another bad value'
  )
  

  (View commit: 19ce8a6)

• The matcher may now raise an error if the attribute in question contains custom logic to ignore certain values, resulting in a discrepancy between the value you provide and the value that the attribute is actually set to. Specifically, if the attribute cannot be changed from a non-nil value to a nil value, or vice versa, then you’ll get a CouldNotSetAttributeError. The current behavior (which is to permit this) is misleading, as the test that you’re writing under the hood by using allow_value could be different from the test that actually ends up getting run.

  (View commit: eaaa2d8)

Breaking changes to validate_uniqueness_of

• The matcher is now properly case-insensitive by default, to match the default behavior of the validation itself. This is a backward-incompatible change because the following test which incorrectly passed before will now fail:

  class Product < ActiveRecord::Base
    validates_uniqueness_of :name, case_sensitive: false
  end
  
  describe Product do
    it { is_expected.to validate_uniqueness_of(:name) }
  end
  

  (View commit: 57a1922)

• Additionally, the matcher no longer imposes a scope by default. Previously the following test would pass when it now fails:

  class Post < ActiveRecord::Base
    validate :slug, uniqueness: { scope: :user_id }
  end
  
  describe Post do
    it { should validate_uniqueness_of(:slug) }
  end
  

  (View commit: 6ac7b81)

Smaller breaking changes

There are a few backwards-incompatible changes we discussed in a previous blog post that we’re implementing now:

• ensure_inclusion_of, ensure_exclusion_of, and ensure_length_of have been removed in favor of their validate_* counterparts. (View commit: 55c8d09)
• set_the_flash and set_session have been changed to more closely align with each other:
  • set_the_flash has been removed in favor of set_flash. (View commit: 801f2c7)
  • set_session('foo') is no longer valid syntax; please use set_session['foo'] instead. (View commit: 535fe05)
  • set_session['key'].to(nil) will no longer pass when the key in question has not been set yet. (View commit: 535fe05)

Notable bugfixes and features

• We’ve fixed a few long-standing bugs with validates_uniqueness_of concerning UUID and array columns against PostgreSQL (#554, #607, #662).
• validates_uniqueness_of also now works with scoped attributes that are boolean columns (#694).
• #permit no longer breaks the functionality of ActionController::Paramers#require (#675).
• validate_numericality_of now supports the on qualifier (fixed in 9748869; original issues: #356, #358).

As always, you can read the remaining changes in the NEWS file.

Pre-release version available

Since this is such a major release, we’ve cut a pre-release version. We encourage you to try out in your own project, and give us any feedback you may have. To get in touch, you can leave a comment on this post, send us an email, or ask a question on Twitter.

Enjoy!

---

An aside: Why did the installation instructions change?

For a long time Shoulda Matchers required very little setup: all you had to do was add the gem to your Gemfile and you could start using the matchers in your tests right away, regardless of whether you were using RSpec or Minitest. A little magic made this possible. The gem first tried to identify which test framework and which parts of Rails you were using. If a certain Rails component was loaded, it would make the corresponding matchers for that component available by mixing a module into whichever test framework had also been loaded. So for instance, if you were using RSpec and you’d required ActiveRecord somewhere, it would know both of these things, and it would use RSpec’s configuration block to include Shoulda::Matchers::ActiveRecord into every example group.

While this autodetection logic was nifty, it wasn’t foolproof. We received numerous reports from people using RSpec who found they weren’t able to access the matchers (#303, #382, #486, #494, #529, #550, #552). The cause of this turned out to be Rails preloaders (most notably Spring, which was added to the Rails toolchain in the 4.1 release). These tools work by booting the core of your Rails application, including all gems in your Gemfile, and holding it in memory, so that when you run your tests you don’t have to wait for this environment to load over and over again. This isn’t a bad idea on its own – indeed, these preloaders can be useful in larger projects that have a significant startup time.

But there’s a downside: loading your gems up front can interfere with those gems that need to be initialized in a very specific order. In particular, in order for Shoulda Matchers to mix itself into RSpec, RSpec must be loaded first. Without a preloader in place the rspec executable effectively guarantees this happens. We can illustrate the order of things in this scenario:

• Run rspec
  • Require rspec-core (defining RSpec)
  • Require rails_helper
    • Load Rails environment
      • Load Gemfile
        • Require shoulda-matchers (sees RSpec and mixes itself in)

But a preloader changes the game, because now the act of loading all gems happens first, and when that is the case we cannot guarantee that everything is loaded in the correct order. RSpec could be loaded before Shoulda Matchers:

• Preload Rails
  • Load Gemfile
    • Require rspec-rails (defining RSpec)
    • Require shoulda-matchers (sees RSpec and mixes itself in)
• Run rspec

but it could also be loaded after:

• Preload Rails
  • Load Gemfile
    • Require shoulda-matchers (RSpec isn’t available yet!)
    • Require rspec-rails (RSpec is now defined, but it’s too late)
• Run rspec

In order to combat this problem, we were forced to add an extra step to the installation instructions for the gem. This is why we used to tell you to add the gem to your Gemfile but with require: false, and then require it manually within rails_helper. This solved the problem in the short term, but we were never really happy about this, as it made the gem seem more complicated to use than it actually is.

Ultimately, our goal with the gem is to increase happiness and reduce frustration. By removing the autodetection code, we reduce the chance that we will get it wrong, and hopefully we create a better user experience for everyone.