Despite our best intentions, spending time developing software that people won’t use can be devastating to morale and to a product’s long-term viability. How can we stop wasting time on developing features (or whole products!) that won’t be used? How can your customers’ most painful problems be identified? Most teams turn to user interviews to flesh these ideas out early on in the product life cycle. With user interviews we can understand what users want and how to best alleviate their problems.
Except it doesn’t work out that way for most teams. Without realizing it we ask questions that yield biased results. That means that if we aren’t careful, we might actually build features or entire products that customers say they want, but actually won’t use.
The mind is a maze
“If I had asked people what they wanted, they would have said faster horses.” -Henry Ford
There are over 150 biases listed on Wikipedia that may come into play during a conversation with your users. The difference between the answers you will get when asking, “Would a feature that lets you add task templates be useful?” and “What problems do you have with managing tasks?” could mean the difference between spending weeks on a feature or realizing that people won’t even use it (and spending no time at all). Without realizing it, we may build something based on biased feedback.
Why do our minds adopt these patterns of bias? Do they even serve a purpose? Cognitive biases enable faster decisions when timeliness is more valuable than accuracy. They can help us to make decisions quickly when accuracy is not as important or when situations are dangerous. When we are asking questions about our products, we want accuracy, not speed. Are the questions you tend to ask during user interviews invoking these cognitive biases or not? Are they causing your users to answer with speed, or with accuracy?
Without realizing, we often ask questions that trigger these mental shortcuts and produce biased responses. Biased responses lead us to build software that is not really what our users need or want. Luckily, research on cognitive biases and logical fallacies have helped us ask questions that produce honest and actionable results.
Hidden in plain sight
Let’s go over some common pitfalls and how to avoid them. In the process we’ll figure out what kind of questions will help us identify the information we need, while avoiding questions that lead to inaccurate responses.
- Avoid giving away too much information. A good question should be goal oriented. Asking someone to, “add a task for sending a proposal to your client” makes it difficult to see how a real person would think through a problem, because you told them how to map their problem to an action in the software, “add a task”. Instead don’t tell them what to do in the interface, give them a problem and see how they think through it. “You want to send a proposal to a client later in the week, what would you do?” This helps you see how your customer thinks through a problem and uses your software to solve it. It may be that they don’t need your software to solve their problem.
- Avoid asking questions that plant an idea in their head and lead them to a conclusion. This is called anchoring. “How useful would task templates be in your project management software?” is a leading question. It plants the idea that task templates would be useful. Why is asking this a problem? Task templates probably are useful, right? So is Tylenol, but only if you have pain. Before prescribing a possible solution, try asking about their pain, the problem they want to solve. “What problems do you have with managing your team’s work?” is a type of question that you will help you find your customers most painful problems. These are the problems that customers really care about.
- Avoid asking questions that limit responses (false dilemmas). These include yes/no questions and questions where you give the person only a subset of possible responses. For example instead of asking, “Would you rather have a task template feature first or would you rather be able to bulk delete tasks?” you could ask, “if you could choose one feature to be included in the next release, what would it be?”
- Beware of the current moment bias. When asking if a feature would be useful, there is a good chance that they can see it being more useful in the moment because they are sitting there with the software in their hands. When someone is experiencing normal life, the pain of the problem they need to solve may not be bad enough to cause them to use your software. Instead ask about past experiences and problems to see if they have felt and noticed the pain point you are trying to solve. Ask what they currently do to solve the problem. It is also helpful to do field tests where you observe people using software in the real world.
- Don’t be afraid to stray from the script and ask, “why?” Knowing why someone says or acts or does things, can be huge. You may realize that their mind took a mental leap to a conclusion that wasn’t really what they thought. Asking “why?” can help you be sure that your findings are accurate. It will require extra work to stray from the script and interpret the responses, but it will be worth the extra effort.
- Be mindful of your own confirmation bias. We want to agree and remember when users say things that we already agree with. Do not do this. Being actively aware of this bias is essential when speaking with your users. Be open to viewpoints that don’t align with what you were thinking.
There are many reasons that a product can fail to gain traction, but products that are based on user’s real needs have a fighting chance. The best user interview questions avoid biases and attempt to find the real problem people are trying to solve. With these tips you’ll be a bit further ahead in creating your next hit product.
