We have just added some new permissions logic to a project.
def can_read? article.access_policy&.public? || user.admin? end
We might include some tests that read like:
describe "#can_read?" do it "allows admin to access public article" it "allows admin to access private article" it "allows non-admin to access public article" it "disallows non-admin from accessing private article" end
During code review, one of our colleagues asks
Do we have any tests that cover the
Hmmm, it looks like we forgot! How can we do a better job of making sure we cover all our edge cases?
We can use math to calculate how many combinations of inputs we have. In order to do so, we need to identify each of the inputs that could affect our method, calculate how many different states they can have, and multiply them together.
Here, two variables influence the outcome of our method: the access policy and the user’s role. They can be in the following number of states:
- An article’s public status can be in
3states (nil, true, or false)
- A user being an admin can have
2states (true or false)
Multiplying these gives us
6 possible combinations of inputs. We probably want
6 unit tests to cover all the combinations but our original implementation only
had 4. Time to add the missing 2 tests to cover
Sometimes we may prefer to calculate the different cases visually using a truth table or decision table. Taking this approach is more explicit but doesn’t scale up well to many inputs or many states due to combinatorial explosion.
Calculating combinations is useful for more than just booleans. Consider the following constructor for a query object:
class ArticleQuery def initialize(limit: nil, scope: Article.all) # ... end end
There are at least 4 different ways of invoking this method (2 arguments, each of which can be explicitly passed or defaulted). We probably should have a test for each of these cases.
new(limit: 10, scope: Article.published)
There may be even more cases. Sometimes explicitly passing
nil may be
different than not passing anything. The
scope argument could have 3 states
(default, explicit scope, explicit
Some types of values like strings or arrays have theoretically infinite states. In practice we probably care about a finite number of slices of that infinite domain. For example, our code might only distinguish between empty, single, and multi-item arrays giving us functionally 3 different states.
In our original example, we identified 2 edge cases that weren’t covered in our test suite. One solution is to add some tests so that our suite now covers the 6 scenarios handled by our code.
Alternatively, we could reduce our code to only handle the 4 scenarios described in our test suite. Testing pain is often a sign that we should change our implementation.
In our original example, we may decide to make access policies required on
articles, eliminating the potential
nil and bringing us down to 4 possible
Similarly, we might remove the defaults from the
ArticleQuery constructor. Now
we have eliminated a lot of uncertainty as there is only one way to invoke it. A
win for both our code and our test suite!