testing your tests

Jared Carroll

Recently I was taking a look at my tests and noticed some things. I’ve been writing tests for my tests without knowing it.

Here’s one I commonly do when testing the ordering of results from an ActiveRecord class side finder.

This is wrong:

def test_should_find_all_posts_ordered_by_most_recent_first_when_sent_recent
  posts = Post.recent
  posts.each_with_index do |each, index|
    if posts[index.succ]
      assert each.created_on >= posts[index.succ].created_on
    end
  end
end

The problem with this test is that Post#recent could return an empty Array, then #each_with_index would not run and no assertions would be run but the test would still pass.

Rewrite:

def test_should_find_all_posts_ordered_by_most_recent_first_when_sent_recent
  posts = Post.recent
  assert ! posts.empty?
  posts.each_with_index do |each, index|
    if posts[index.succ]
      assert each.created_on >= posts[index.succ].created_on
    end
  end
end

Better.

Now we #assert that the collection returned from Post#recent is at least not #empty? However this test can still pass without executing 1 assertion because in order to test ordering you need at least 2 objects.

Rewrite:

def test_should_find_all_posts_ordered_by_most_recent_first_when_sent_recent
  posts = Post.recent
  assert ! posts.empty?
  assert posts.size >= 2
  posts.each_with_index do |each, index|
    if posts[index.succ]
      assert each.created_on >= posts[index.succ].created_on
    end
  end
end

Now its good.

There’s no way this test will pass without testing that the collection of posts found by Post#recent is ordered by most recent first.