Integration Testing with Capybara

If your application relies on your server to generate and transmit all of its HTML, then the structure and contents of those HTTP requests and responses is crucial.

Testing the overlap between Controllers and Views

Out of the box, Action Dispatch depends on the rails-dom-testing gem to provide tests with a way of asserting the structure and contents of a request’s HTML response body. For example, consider an HTML response for an HTTP GET request to /articles/hello-world:

<html>
  <head>
    <title>Hello, World!</title>
  </head>
  <body>
    <main>
      <h1>Hello, World!</h1>
    </main>
  </body>
</html>

The rails-dom-testing gem provides a collection of methods that transform CSS selectors and text into assertions about the structure and contents of the response body’s HTML:

class ArticlesTest < ActionDispatch::IntegrationTest
  test "index" do
    get article_path("hello-world")

    assert_select "title", "Hello, World!"
    assert_select "body main h1", "Hello, World!"
  end
end

In their most simple form, the rails-dom-testing assertions provide a lot of utility in their flexibility. Despite that flexibility, asserting with assert_select can become complicated in the face of Real World features.

For example, consider an HTML response for an HTTP GET request to /sessions/new:

<html>
  <head>
    <title>Sign in</title>
  </head>
  <body>
    <main>
      <h1>Sign in</h1>

      <form method="post" action="/sessions">
        <label for="session_email_address">Email address</label>
        <input type="email" id="session_email_address" name="session[email_address]">

        <label for="session_password">Password</label>
        <input type="password" id="session_password" name="session[password]">

        <button>Sign in</button>
      </form>
    </main>
  </body>
</html>

The ActionDispatch::IntegrationTest cases for this page might cover several facets of the HTML:

The page has "Sign in" in its <title> element
The page has two different elements with "Sign in": an <h1> heading and a <button> element
The page has an <input type="email"> element to collect the user’s email address, and that field is labeled with the text "Email address"
The page has an <input type="password"> element to collect the user’s password, and that field is labeled with the text "Password".

We could cover those requirements with the following rails-dom-testing-capable test:

class SessionsTest < ActionDispatch::IntegrationTest
  test "new" do
    get new_session_path

    assert_select "title", "Sign in"
    assert_select "body main h1", "Sign in"
    assert_select "body main button", "Sign in"
    assert_select %(input[id="session_email_address"][type="email"])
    assert_select %(label[for="session_email_address"]), "Email address"
    assert_select %(input[id="session_password"][type="password"])
    assert_select %(label[for="session_password"]), "Password"
  end
end

These assertions are sufficient to test the page, making sure it meets all of our requirements. Unfortunately, the assertions have some issues.

For instance, the assertions about the relationships between the <input> and <label> elements are extremely brittle. They encode the session_email_address and session_password element [id] attribute directly into the test. If that page’s HTML were to change, those tests would fail even if the new HTML declared the <label> elements with the same text content and in a way that continued to properly reference their <input> elements.

Similarly, the specificity required by the title, h1, and button selectors is tedious compared to how different a <title>, <h1> and <button> are treated by a browser.

These types of issues could be addressed by introducing abstractions to account for the varying semantics of each requirement. Luckily, a tool that already does that on our behalf exists: System Tests!

If we were to write a System Test to exercise our page to make sure it meets the requirements, it might resemble something like:

class SessionsTest < ActionDispatch::SystemTestCase
  test "new" do
    visit new_session_path

    assert_title "Sign in"
    assert_css "h1", "Sign in"
    assert_button "Sign in"
    assert_field "Email address", type: "email"
    assert_field "Password", type: "password"
  end
end

These assertions are not only more terse, they’re also more robust. For example, the assert_button will continue to pass if the <button> element were replaced with an <input type="submit"> element with the same text. Similarly, the assert_field calls would continue to pass if the <input> elements’ [id] attributes were changed, or if they were moved to be direct descendants of the <label> element. What an improvement!

Unfortunately, these improvements come with a cost.

The Challenges of a growing test suite

In practice, a Rails application’s test suite will exercise its controller and view code in two ways: through ActionDispatch::IntegrationTest tests, and through ActionDispatch::SystemTestCase tests.

System Tests are often executed by driving real browsers with tools like Selenium, Webdriver, or over the Chrome DevTools Protocol (via Apparition). The good news: driving a browser through your test cases provides an extremely high and valuable level of fidelity and confidence in the end-user experience. The bad news: driving a browser through your test incurs an extremely high cost: speed.

That cost can be recuperated by configuring System Tests to be driven_by :rack_test. When configured this way, test cases rely on a Rack Test “browser” to make HTTP requests, “click” on buttons, and “fill in” form fields on their behalf. These types of tests are valuable in their own right, but they don’t provide the same level of control as ActionDispatch::IntegrationTest cases.

For example, an ActionDispatch::IntegrationTest can submit an HTTP request with any HTTP verb, whereas a System Test can only directly submit GET requests through calls to visit. Similarly, System Tests can’t access the response’s status code or its headers directly, nor can the read from Rails-specific utilities like the cookies, flash, or session. For example, consider a test that exercises a controller’s response to an invalid request:

class ArticlesTest < ActionDispatch::IntegrationTest
  test "update" do
    valid_attributes    = { title: "A valid Article", body: "It's valid!" }
    invalid_attributes  = { title: "Will be invalid", body: "" }
    article             = Article.create! valid_attributes

    assert_no_changes -> { article.reload.attributes } do
      put article_path(article), params: { article: invalid_attributes }
    end

    assert_response :unprocessable_entity
    assert_equal "Failed to create Article!", flash[:alert]
  end
end

Conversely, ActionDispatch::IntegrationTest cases exercise the application one layer of abstraction below the browser: at the HTTP request-response layer. They don’t drive a real browser, but they do make the same kinds of HTTP requests! On top of that, they provide built-in mechanisms to read directly from the response, session, flash, or cookies.

It can be challenging to strike a balance between the confidence & fidelity of System Tests and the speed and granularity of Integration Tests when exercising the structure and contents of a feature’s HTML. It can be even more challenging if you need to maintain and context switch between two parallel sets of HTML assertions. Both System and Integration Tests are compatible with Rack::Test. What if there was a way to get the best of both worlds?

Combining `ActionDispatch::IntegrationTest` with `Capybara`

Capybara provides Rails’ System Tests with its collection of finders, actions, and assertions. Conceptually, the value provided by Capybara’s finders and assertions overlaps entirely with the assertions provided by rails-dom-testing. Plus, if your application has a suite of System Tests, you’re already depending on Capybara!

If we wanted to share the same Rack::Test session between our ActionDispatch::IntegrationTest case and our Capybara assertions, we’d need to integrate a Capybara::Session instance with the integration_session at the heart of our tests. If we wanted to modify the ActionDispatch::IntegrationTest class, we could do so with an ActiveSupport::Concern mixed-into the class from within an ActiveSupport.on_load hook:

ActiveSupport.on_load :action_dispatch_integration_test do
  include(Module.new do
    extend ActiveSupport::Concern

    included do
      setup do
        integration_session.extend(Module.new do
          # ...
        end)
      end
    end
  end)
end

Within that concern, we’d want to declare a #page method like the one our System Tests provide. Within the #page method, we’d construct, memoize, and return an instance of a Capybara::Session:

 ActiveSupport.on_load :action_dispatch_integration_test do
   include(Module.new do
     extend ActiveSupport::Concern

     included do
       setup do
         integration_session.extend(Module.new do
+          def page
+            @page ||= ::Capybara::Session.new(:rack_test, @app)
+          end
         end)
       end
     end
   end)
 end

Once we constructed that instance, we’d need to make it available to the inner-working mechanics of the ActionDispatch::IntegrationTest case. Right now, the only avenue toward that goal is re-declaring the _mock_session private method:

 ActiveSupport.on_load :action_dispatch_integration_test do
   include(Module.new do
     extend ActiveSupport::Concern

     included do
       setup do
         integration_session.extend(Module.new do
           def page
             @page ||= ::Capybara::Session.new(:rack_test, @app)
           end
+
+          def _mock_session
+            @_mock_session ||= page.driver.browser.rack_mock_session
+          end
         end)
       end
     end
   end)
 end

Depending on Rails’ private interfaces is very risky and highly discouraged. There is an ongoing discussion about adding public-facing hooks for this type of integration (see rails/rails#41291 and rails/rails#43361). Since these are test-level dependencies, changes to the private implementation won’t lead to production-level outages. Ideally, this will only be temporary!

With those changes in place, the last step is to mix in Capybara’s assertions:

+require "capybara/minitest"
+
 ActiveSupport.on_load :action_dispatch_integration_test do
   include(Module.new do
     extend ActiveSupport::Concern

     included do
+      include Capybara::Minitest::Assertions
+
       setup do
         integration_session.extend(Module.new do
           def page
             @page ||= ::Capybara::Session.new(:rack_test, @app)
           end

           def _mock_session
             @_mock_session ||= page.driver.browser.rack_mock_session
           end
         end)
       end
     end
   end)
 end

Now we can re-write our SessionsTest to inherit from ActionDispatch::IntegrationTest instead of ActionDispatch::SystemTestCase:

-class SessionsTest < ActionDispatch::SystemTestCase
+class SessionsTest < ActionDispatch::IntegrationTest
   test "new" do
-    visit new_session_path
+    get new_session_path

     assert_title "Sign in"
     assert_css "h1", "Sign in"
     assert_button "Sign in"
     assert_field "Email address", type: "email"
     assert_field "Password", type: "password"
   end
 end

If we wanted to use other Capybara-provided helpers like within, we could delegate those calls to our page instance:

 require "capybara/minitest"

 ActiveSupport.on_load :action_dispatch_integration_test do
   include(Module.new do
     extend ActiveSupport::Concern

     included do
       include Capybara::Minitest::Assertions
+
+      delegate :within, to: :page

       setup do
         integration_session.extend(Module.new do
           def page
             @page ||= ::Capybara::Session.new(:rack_test, @app)
           end

           def _mock_session
             @_mock_session ||= page.driver.browser.rack_mock_session
           end
         end)
       end
     end
   end)
 end

Then we could use them in our tests:

 class SessionsTest < ActionDispatch::IntegrationTest
   test "new" do
     get new_session_path

     assert_title "Sign in"
+    within "main" do
       assert_css "h1", "Sign in"
       assert_button "Sign in"
       assert_field "Email address", type: "email"
       assert_field "Password", type: "password"
+    end
   end
 end

With this integration in place, there’s nothing stopping us from declaring custom Capybara selectors, or adding a dependency like citizensadvice/capybara_accessible_selectors that declares them on our behalf:

 class SessionsTest < ActionDispatch::IntegrationTest
   test "new" do
     get new_session_path

     assert_title "Sign in"
-    within "main" do
+    within_section "Sign in" do
-      assert_css "h1", "Sign in"
       assert_button "Sign in"
       assert_field "Email address", type: "email"
       assert_field "Password", type: "password"
     end
   end
 end

With access to assertions and selectors like the ones that Capybara or citizensadvice/capybara_accessible_selectors provide (like assert_field, the described_by: filter, and the alert selector), we have an opportunity to make assertions about the structure and semantics of the response to an invalid request:

 class ArticlesTest < ActionDispatch::IntegrationTest
   test "update" do
     valid_attributes    = { title: "A valid Article", body: "It's valid!" }
     invalid_attributes  = { title: "Will be invalid", body: "" }
     article             = Article.create! valid_attributes

     assert_no_changes -> { article.reload.attributes } do
       put article_path(article), params: { article: invalid_attributes }
     end

     assert_response :unprocessable_entity
-    assert_equal "Failed to create Article!", flash[:alert]
+    assert_selector :alert, "Failed to create Article!"
+    assert_field "Body", described_by: "can't be blank"
   end
 end

Hopefully, an integration like this will become publicly supported in the future. In the meantime, if you’d like to add support for Capybara assertions in your Minitest suite’s ActionDispatch::IntegrationTest cases, or your RSpec suite’s type: :request tests, check out the thoughtbot/action_dispatch-testing-integration-capybara gem today!

Testing the overlap between Controllers and Views

The Challenges of a growing test suite

Combining ActionDispatch::IntegrationTest with Capybara

Sign up to receive a weekly recap from thoughtbot

Combining `ActionDispatch::IntegrationTest` with `Capybara`