Our developers follow Test-Driven Development, or TDD, which is a development process where they write tests for features before writing the code to build said features. These tests outline the steps the users must take to get to their expected outcome.
When starting a new feature, the tests initially fail; one by one, developers begin to build the features that will make the tests pass. Once the tests are passing, the code gets reviewed and then merged into the codebase. Tests are then run when building new features to ensure they don’t break existing ones. This helps prevent regressions in our software and gives our developers more confidence when refactoring code.
I love the assurance the tests give me when I make design changes. It’s clear to me what I might have broken when implementing design for new features and makes sure I don’t introduce bugs into the application. I want to carry over the same calculating testing process to the design of the application as well, to make sure that the features the team is building are solving for the user’s job-to-be-done.
The process for creating design tests
We use jobs stories throughout our design and development process to help us frame and complete tasks. I’ve begun using these stories as my design tests.
Before the developers and I start the time-consuming process of building an application, I conduct design research and run a design sprint to test and validate what the team thinks will solve the users problems. This allows me to interview users and use a prototype as a talking point. When planning for the interview, I create questions around the job stories the team has created to ensure that the user is experiencing the context, motivations, and outcome that the team assume they do.
For example, let’s take this fake jobs story for the readers of this blog:
When I am reading a post on Giant Robots, I want thoughtbot to categorize posts into topics, so that I might find more relevant posts.
Here are a couple of questions I might ask based on the jobs story above:
- How often do you read Giant Robots?
- What kinds of posts have you read about on Giant Robots?
- What are some of the posts that you have enjoyed reading the most?
- What are some of the posts that you have enjoyed the least?
- How do you go about finding content relevant to you on Giant Robots? Can you show me?
Notice these questions are about what users have already done and already liked. They are not leading questions about what they will do or will like. For more about asking the right questions, read What not to ask.
After these questions, I’ll then have them jump into a prototype and ask them to do something like:
- Find a post that you have read or want to read.
- Find similar content to that post.
Figuring out if those tests have passed or failed
I’ll place each jobs story into an assumptions test table and list out the questions and tasks that I’m going to give and what validation would look like. It’s then explicit which tests have passed and which have failed, just like the development tests.
| Broken up jobs story (Assumption) | Test | Validated (Passed) if |
|---|---|---|
| [Context] When I am reading a post on Giant Robots, |
[Question] How often do you read Giant Robots? |
They read infrequently to frequently. |
| [Motivation] I want thoughtbot to categorize posts into topics |
[Question] What kinds of posts have you read about on Giant Robots? |
The posts they read are all in one or two topics. |
| [Question] What are some of the posts that you have enjoyed reading the most? |
The posts they like are all in one or two topics. | |
| [Question] What are some of the posts that you have enjoyed the least? |
The posts the user doesn’t like or skip have the same kind of topics. | |
| [Expected outcome] so that I might find related posts. |
[Question] How do you go about finding content relevant to you on Giant Robots? Can you show me? |
They currently have a convoluted way of finding relevant content on GR. |
| [Task] Find similar content to that post. |
User finds post tags and tag pages to get to more articles that sound interesting to them. |
While they go through the prototype, I’m looking to hear about how they are trying to solve the task I’ve presented in front of them without hinting to how our team decided to solve the story.
Designing learning into the tests
By telling us about how they currently hire for a job-to-be-done, answering questions around their solutions, and using parts of the prototype, users either validate or invalidate our assumptions about their context, motivations, and expected outcome. When the user gets confused or doesn’t complete the tasks as expected, I then follow up with questions to figure out which of our assumptions on the story were wrong as well as what they should have been. It’s a combination of a jobs-to-be-done style interview with a guerrilla usability test.
After the interviews, I take another look at the job’s story on the Trello card and reword it to match what everybody on the team saw in the interview. I bring the card back into our next up column to build awareness that the product team needs to learn more about the job to continue building for it. Many parts of the jobs story get invalidated during this process too: either the team assumed the wrong context or it ends up solving a job outside of the scope of the project, which means I can archive the card all together. This saves us the time it would have taken to build it, the technical debt the code would have cost, and the lack of focus it would have brought to the product. Our product team always leave an interview having learned from it, then apply our knowledge to a new prototype or what we’re building into the application.
After the initial prototype
Once the product team has something built, I begin usability testing with the application on staging instead of a prototype, following the same process I did for the prototyped features. This is similar to our developers running their tests for old features while building new ones. I want to ensure that our initial validation was correct and that any new features we’ve built aren’t polluting our existing features.
These calculated research interviews and usability tests continue to validate that our team is building the correct set of features into the product for the user’s job-to-be-done. They act as acceptance tests that will pass or fail based on specific jobs stories that we have already written.
