We recently joined an existing Rails 4.1 project that uses AngularJS. We
observed sporadic Capybara::ElementNotFound failures in RSpec feature specs
with JavaScript. These failures were false positives and undermined our
confidence in the test suite, preventing us from using a fluid outside-in
testing process.
Finding the Problem
We saw in Capybara screenshots that the pages (rendered with AngularJS) were
failing to fully load, but only in tests. In order to learn more, we reviewed
test.log, and saw this error:
RuntimeError (Circular dependency detected while autoloading constant MyClass)
This looked like a race condition while loading files. Lazily loading files is
not threadsafe in Ruby, and
since Webrick is multithreaded and Rails.configuration.eager_loading is
disabled by default in test, we suspected that concurrency was enabled.
In production environments, Rails 4.1 uses Rack::Lock if classes won’t be
cached (true in development by default, but false in production by default).
Rack::Lock creates a mutex around requests that guards against threadsafety
issues in multithreaded environments such as Webrick (the test server). We
expected that Rack::Lock would be used in tests as well, but to make sure we
ran RAILS_ENV=test rake middleware, and saw that Rack::Lock was missing.
The Crux
The reason we were seeing the circular dependency error was that Capybara was
making many concurrent requests to a multithreaded server (Webrick) with
Rack::Lock disabled. This became an especially insidious problem in an
AngularJS app, given the number of concurrent requests being triggered via AJAX.
The Solution
We concluded that we needed to force Rack::Lock middleware to be added in
test. This was echoed in this Rails
issue
raised on GitHub.
Rails adds Rack::Lock unless allow_concurrency is true. It follows that we
can explicitly set allow_concurrency to false in our test environment to force
Rack::Lock to be added.
# config/test.rb
config.allow_concurrency = false
The issue will be fixed in Rails 4.2, but in the interim, explicitly disabling concurrency has fixed the circular dependency error we were seeing.
Practicing Red/Green/Refactor and outside-in testing depends on getting reliable failures from the tests, and we didn’t have that. Resolving flakiness in the test suite thus made a notable improvement in our TDD workflow while reviving confidence in the tests.
