A journey towards better testing practices

Tests have always been part of my journey with Ruby and Rails. We benefit from a great ecosystem and community, implicated in writing maintainable software. Along the way I have learned good practices, anti-patterns and trade-offs, and I keep learning new practices and tips every day.

When I peek at old code snippets, such as the ones I wrote a few years back, I can’t help but notice how much my tests have evolved, practices I now try to keep away from and goals I work to reach.

Join me in a small journey from anti-patterns to good practices.

Moving away from RSpec DSL

I used to try to use as much RSpec features as possible, thinking these features must be useful to my work if they existed. From my first coding lessons I learned that we should factorize our code, keep it DRY, to avoid hidden bugs.

So I abstracted everything. Any repeated test had to be a shared_example. Any shared context had to define common objects and instructions into let and before blocks. The test subject had to fit into subject so that I could write one-liner expectations.

The following spec is an example of what I could have written a few years back:

RSpec.describe Rocket do
  subject { described_class.new }

  shared_example "it has the right agency" do
    it "indicates the agency collaborates in the ISS" do
      expect(subject.agency_collaborates_in_iss).to be(collaborates?)
    end
  end

  describe "agency" do
    let(:collaborates?) { true }

    include_examples "it has the right agency"

    context "when the rocket has an explicit name"
      subject { described_class.new(name: name) }

      context "when the rocket is named Ariane 5" do
        let(:name) { "Ariane 5" }
        let(:collaborates?) { true }

        include_examples "it has the right agency"
      end
    end
  end
end

This spec is very hard to read and to update. New developers, sometimes ourselves three months from now, will have to jump from top to bottom and to top again to understand what is tested, and how. Any new scenario that cannot comply to this heavily abstracted code will lead to a significant refactoring. But at least, was I thinking, I was taking advantage of the full power of RSpec by using has much of the DSL as possible. After all, if all these methods and tools existed, this had to be for a good reason, and using them had to be a good practice.

I learned that a better approach for maintainable specs was to explicitly define the setup, exercise and verification. We can gain an immediate understanding by reading the test alone of the context we’re building up, what we are testing and what we expect.

The best change in the way I wrote tests to achieve this, was to get rid of nested scenarios. It not only complicates the setup, but it encourages shared code within let and before blocks. The only shared code that remained was shared_examples, but they can be removed easily when we understand we don’t have to factor out specs and it is okay to repeat a similar setup multiple times to have self-contained tests.

I am not saying these DSL methods are not useful. However they can easily bring anti-patterns to our tests and require to learn a RSpec-specific syntax that is not universal in Ruby.

I am still using a lot of RSpec DSL though, such as test doubles or mocks.

Lightweight factories

I used to build exhaustive factories. Large factories are useful when working with tightly-coupled domain objects, when we rarely use an object without associated ones. In feature specs, request specs, or when testing some service objects, it feels safe to immediately have the whole picture.

Let’s take the example of an inhabited space flight and its passengers. A lot of the scenarios we test happen while the rocket is in space, carrying passengers, needing an engine to lift off, so I used to like building such factories for their comprehensiveness:

factory :flight do
  launched_on { Date.new(1963, 6, 16) }
  succeeded { true }
  rocket

  after(:create) do |flight|
    flight.passengers << create(:passenger)
  end
end

factory :rocket do
  name { "Vostok-3KA" }
  engine
end

factory :engine do
  name { "S5.4" }
end

factory :passenger do
  full_name { "Valentina Tereshkova" }
end

I slowly discovered that this came with drawbacks.

First, speed: Each time we instantiate a flight, a rocket and its engine instance will be added to it. If we create it, flight, rocket and engine records will be created, alongside with a passenger record, associated to the flight. These dependencies are unnecessary when we use the flight object in an isolated use case, like model specs. Creating them slows down the whole test suite.

The opportunity to benefit from all these dependencies right away is also counter-balanced by the fact that we often want to customize these dependencies to have accurate testing scenarios. So, in the end, we often have to update or even get rid of these dependencies:

standalone_rocket = create(:rocket, engine: nil)
standalone_rocket.passengers.destroy_all

Instead, what we can do is favor defining light-weight factories that contain only the necessary attributes to be created. In other terms, to pass validations, without sacrificing the convenience of easily creating instances with default associations.

factory :rocket do
  name { "Vostok-3KA" }

  trait :with_engine do
    engine
  end

  trait :with_passengers do
    after(:create) do |rocket|
      rocket.passengers << create(:passenger)
    end
  end
end

standalone_rocket = create(:rocket)
rocket_with_an_engine = create(:rocket, :with_engine)
rocket_with_passengers = create(:rocket, :with_passengers)

Of course, getting there required updating some tests that were implicitly carrying a bunch of dependencies, but this burden is worth it to have a maintainable test suite.

Avoiding randomness

I remember being used to having an unjustified fear of explicitness. In factories, I was afraid of defining fixed strings, thinking they wouldn’t represent the diversity of possible inputs.

For several years I used the faker gem to replace user names, addresses, phone numbers or even numbers. My initial goal was that I would be able to discover new unsupported inputs from failing tests and have to adapt my code to deal with such data.

factory :user do
  name { Faker::Name.name }
  sequence(:email) { |n| Faker::Internet.email("user_#{n}") }
end

That was a terrible assumption as I was only introducing a delicious cocktail of flaky tests. By definition, some tests would randomly fail depending on the generated data. The failing tests were not even always linked to the data itself, but failed as a side effect.

Instead we should favor explicit inputs and test our code over data we understand. I believe there are two solutions against fearing user input:

• Spend time to think of unusual inputs and test them
• Fail gracefully when we don’t support the input

I still use randomness and the faker gem a lot, but not for the same reasons. For example it is useful when creating development fixtures (and not testing fixtures), to easily have a diversified data set that enables multiple features to be used. But I tend to favor explicitness when writing tests.

Testing what the user sees in feature specs

I used to rely a lot on HTML elements, especially classes and ids, when coding feature specs. DOM attributes look convenient to use as they often are unique element identifiers.

One main reason for this was because I misunderstood what I now believe is the real purpose of feature specs. Feature specs differ from other tests in the way we test the big picture, from a user point of view. As a user, I am not interacting with HTML tags or CSS classes, I am interacting with what the browsing software describes to me.

Let’s have a look at the following spec to ensure users logged in from their GitHub account are not offered to edit their password:

describe "when user signed in with their GitHub account" do
  it "doesn't show the credentials edition button" do
    user = create(:user, :from_github)

    visit root_path(as: user)
    click_on(css: "edit-account-link")

    expect(page).not_to have_css("form#edit_password")
  end
end

Relying on DOM attributes makes our tests fragile. These elements can and should be easy to change by developers, designers and anyone else on the team, without breaking any test. They don’t even represent what we really want to test in feature specs, which is to assert elements and events.

We should prefer relying on what the user really sees, such as text, with a bonus point when using internationalization to reduce text coupling.

describe "when user signed in with their GitHub account" do
  it "doesn't show the credentials edition button" do
    user = create(:user, :from_github)

    visit root_path(as: user)
    click_on I18n.t("menu.edit_account")

    expect(page).not_to(
      have_content I18n.t("edit_account.password")
    )
  end
end

This is not always easy, or even possible. It is a good practice. The goal is to tend towards it. By doing so, we stay close to a readable, maintainable and efficient test suite, and we keep our distance with anti-patterns that could ruin the party. And that’s all what it is about.

Keep learning and trying

In this article we learned a few tips to avoid anti-patterns in tests by using fancy DSL sparingly, keeping tests clear and explicit, and avoiding to load unnecessary dependencies.

I used to rely a lot on betterspecs.org to find new and good practices on how to write tests. Unfortunately this source is a bit aging and some of its guidelines are not considered as good practices anymore in the 2020’s.

From one project to another, one team to another, we keep learning new things and finding ways to improve the quality of our tests. Our blog is filled with great content on testing and we even wrote a book on how to test Rails applications. The journey towards good practices continues so that we can keep providing high quality products to our clients and open source projects.