Mystery Guest

The xUnit Test Patterns: Refactoring Test Code book has an ASTOUNDING amount of testing knowledge. The patterns are also at xunitpatterns.com.

Here’s an example.

You’re having trouble understanding the behavior a test is verifying.

This testing anti-pattern is called Obscure Test.

A common cause of Obscure Test is the Mystery Guest.

Mystery Guest

The test reader is not able to see the cause and effect between fixture and verification logic because part of it is done outside the Test Method.

The impact is:

• Tests don’t fulfill the role of Tests as Documentation.
• You may have Erratic Tests which result in tests that don’t pass during every test run or pass in the test environment but not in production.

Identify the Mystery Guest

In the test code of Rails applications, the Mystery Guest is usually one of four culprits:

• a Rails fixture
• a fixture file such as XML or JSON saved from an HTTP response
• an instance variable defined at the top of a long context block, or nested up multiple context blocks
• a variable without an Intention Revealing Name

The last two culprits have simple solutions:

• pause and think before creating long context blocks
• prefer flat over nested test files
• pause and think about the intent of your variables, then name them that

The first two culprits require more thought:

Replace fixtures with factories

Historically, the most offensive Mystery Guest has been Rails fixtures. If you’ve forgotten why shared fixtures are bad, remind yourself that the Mystery Guest is one reason.

As Rubyists, Factory Bot¹ may be our best defense against the Mystery Guest.

Do you have fixture files like this?

dan:
  name: Dan
  role: developer
  location: San Francisco

phil:
  name: Phil
  role: designer
  location: Boston

Is it important that these fixtures are role-based or location based? For example: users(:dan) is a developer or that he’s in San Francisco? If the former is true, we should rename the fixtures in order to reveal the intention: users(:developer) and users(:designer).

It’s also tempting to create many fun fixtures in these files. Naming them abraham_lincoln and isaac_asimov is fun but there should be a programmatic reason for each fixture in the test suite.

Since the mass exodus to factories, this testing behavior by Rails developers has been less common. What we call “creating objects with factories”, others call creating Fresh Fixtures built using Inline Setup.

Each test method creates a test fixture for its own private use.

This accomplishes our goal of making it easier for the test reader to see the cause and effect between fixture and verification logic.

Set up only relevant information

Irrelevant Information is another cause of Obscure Test. Again, Factory Bot is our friend. Consider this setup:

context 'user account exists with a matching Facebook uid' do
  setup do
    @uid  = 1234567
    @user = create(:user, fb_user_id: @uid)
  end
  ...
end

We explicitly specify only the attribute on user that matters for this test. We use Factory Bot to create only a user with valid data, and we name a @uid variable to express intent for the otherwise Magic Number 1234567.

External resources

Sometimes we need to assert the contents of a file are what we expect. For example, say we’ve generated the file by dumping some XML from a web service (which we’ll then stub out during test runs, using the file as a proxy).

context 'the job XML from the web service' do
  setup { @xml_job = IO.read('test/fixtures/jobs/1.xml') }

  should "include the recruiter's email" do
    recruiter_xml = '<recruiter>recruiter@example.com</recruiter>'
    assert_contains @xml_job, recruiter_xml
  end
end

In this case, test/fixtures/jobs/1.xml is called a Prebuilt Fixture.

To truly avoid a Mystery Guest here, it seems the XML should be inline but is that going too far?

Writing “good” tests

Debating “good” code is something programmers will do as long as there are humans programming programs. Mystery Guest is memory trick for humans to think about how our test code will be read and understood by other humans. Can the intended behavior of some subset of the system be understood in one glance at the test? Or is there a Mystery Guest clouding our understanding?

Project name history can be found here.