We need to talk about fixtures

Matheus Sales

The use of Fixtures is a typical pattern in Rails applications because it is the framework’s default way of setting up test data. Fixtures seem harmless and helpful in the beginning, just like the Kevin, but as our codebase grows, the coupling introduced by fixtures makes our application increasingly challenging to maintain and extend. Let’s see how connascence as a coupling metric can build us arguments to avoid/remove Fixtures from our application.

Brief introduction to connascence

Connascence is a software metric and taxonomy for coupling. It introduces different types of coupling and metrics to give developers a vocabulary to talk about coupling. Strength, Locality, and Degree are used to estimate the impact of connascence.

Common use of Fixtures

Let’s take a look at these two fixtures files in isolation and see how much coupled they’re using connascence.

# spec/fixtures/users.yml

matheus:
  id: 1
  first_name: Matheus
  last_name: Sales
  role_id: 2
  type: 2

# spec/fixtures/roles.yml

admin:
  id: 1
  name: admin
user:
  id: 2
  name: user

In this case, these simple files contain at least three types of connascence listed bellow from weakest to strongest.

  1. Connascence of Name: We will need to reference our user by the name matheus, that’s not so bad because connascence of name it’s the weakest form of connascence.
  2. Connascence of Position: The user matheus role depends on the position of the Role fixtures because it specifies the role_id.
  3. Connascence of Meaning: We do not know what the magic value of type: 2 means. 
# spec/fixtures/users.yml

matheus:
  first_name: Matheus
  last_name: Sales
  role: admin
  type: <%= User::B2C_TYPE %>

# spec/fixtures/roles.yml

admin:
  name: admin
user:
  name: user

We’ve refactored our users fixture to be dependent on the name instead of depending on the id of a specific role. That’s a nice refactor because we’ve weakened our connascence by moving from a connascence of position to name. And changing the type field to use a constant, we move from connascence of meaning to name. But how will this help us move away from fixtures?

Look at the big picture

Zero duplication is often a goal, and fixtures help on that side, but tests are one area where this is not ideal. We need to focus on treating our tests as documentation. And that’s where we need to think about why we are using fixtures. One of the biggest problems with this approach is sharing global data across all our test suite making the tests not self-contained and introducing the mystery guest testing antipattern.

Fixtures subtly affect the connascence of our tests, often without us realizing it. Let’s talk through an example:

def test_a_very_complicated_scope
  users = User.filter_by_very_complicated_scope

  assert_equal 2, users.size
end

We have some problems with this test:

  1. Fragile test: any change on the fixture file may break this test and multiple ones.
  2. We have a nebulous test that doesn’t tell us a story and helps us understand what the object under testing means (connascence of meaning).
  3. We need to rely on our knowledge of the current fixtures to understand this test because the setup part of our test is implicit (connascence of meaning).
  4. The places where the test data is specified, and used are distant from each other. This is a locality problem (connascence of position).
def test_a_very_complicated_scope
  ## Setup
  admin = Role.create(name: "admin")
  user_role = Role.create(name: "user")
  user_admin_with_last_name = User.create(role: admin, first_name: 'Matheus', last_name: 'Sales')
  user_admin_without_last_name = User.create(role: admin, first_name: 'Matheus', last_name: '')
  user_without_last_name = User.create(role: user_role, first_name: 'Matheus', last_name: '')
  user_with_last_name = User.create(role: user_role, first_name: 'Matheus', last_name: 'Sales')

  ## Exercise
  users = User.filter_by_very_complicated_scope

  ## Verify
  assert_equal [user_admin_without_last_name], users
end

The refactored version of our test does not use any pattern and makes our setup test phase explicit, creating just enough data needed, and telling us a story that the filter_by_a_very_complicated_scope is related in some way to a user being of admin role and not having a last_name. This test is isolated from our test suite, and does not depend on any shared global data.

But this approach might make our test challenging to read and more inconvenient to write. When dealing with objects that have multiple validations on different attributes, we need to pass multiple attributes just to make the object valid, most of these attributes may not be directly related to our test case.

Most of the problems introduced by the usage of fixtures will be spread into our entire test suite. Fortunately, they can be fixed using a design pattern called Factory to generate the bare minimum data needed for that specific test —and sometimes factories aren’t even required to achieve this. Making each test independent and more straightforward is an excellent way to increase the maintainability of our codebases. This approach is quite common and many languages have libraries for writing tests with factories (for example, in JavaScript, Python, or Elixir).

Summary

Connascence is a way to measure coupling. Coupling makes code harder to be readable by humans, and without a human to read it, the code can’t be maintained, extended, or re-used. It can’t even be finished: the developer has to read the code to write it. A good test/suite needs to tell a story, and the usage of fixtures will make this a pretty hard thing to achieve and maintain. Factories will make your tests easier to read, better document your code, remove globally shared data, and reduce coupling and connascence.