This isn’t a post about convincing you of the benefits of user interviews, or the ills of leading questions. Nor is it a treatise on the one true way to conduct a user test. Instead, these are some observations I’ve made about what works while observing people.
Turn off everything
Put your phone on silent. If you’re on a Mac, switch on Do Not Disturb mode. Quit all the apps that make beeping noises, even when Do Not Disturb mode is activated. Don’t forget to check your Menu or Taskbar for those pesky out-of-the-way yet attention-demanding apps!
Doing so will help keep both you and the participant focused. There’s nothing worse than having your interviewee stop mid-train-of-thought because someone was trying to get your attention in Slack.
Use the buddy system
Having someone there to assist you makes a world of difference.
In the immediate, they can take notes during the session, freeing you up to focus entirely on the conversation. It’s also a nice way to have some safeguards built into the interview session, should you run into difficulties.
In the longer term, it’s someone you can talk with about their perception of the interview. It’s easy to get lost in the moment—being able to have another observer provide their input after the fact can help guard against bias.
Dress for success
A lot of spammy job interview help sites recommend dressing up for your virtual meetings. I hate to admit it, but I completely agree with this advice.
Conducting a day of back-to-back user interview sessions can be an extraordinarily draining process. There might be the temptation to wear your favorite comfortable tee shirt or sweatpants to make up for it. Resist that feeling.
Dressing business casual helps communicate to the participant that you’re a professional. This may chip away at some initial skepticism (“You show people pictures for a living?”) without intimidating them with more formal wear.
Print the script out
Having a physical copy of your script means you can scribble notes down during the interview without getting caught up futzing with a digital interface. This might be personal preference, but I’ve found it makes paying attention easier. Removing digital distractions means you won’t get caught up with things like getting your bullet point nesting strategy just right.
I’ve also observed that participants might notice typing sounds or a change in the quality in the light on your face if you’re shifting through application windows while running the session. This can leave them feeling put out, like they’re not the most important concern.
Remember that your participants are taking time out of their busy day to help you with your project. Resist the urge to multitask and stay focused on them, even if they’re sharing their screen and can’t see your face.
The script is more what you call guidelines, than actual rules
While it is important to develop a script beforehand that will dig into the assumptions you’re making about the product’s design, following it in the order that it is authored isn’t a hard and fast rule.
I won’t lie, this one took me a bit to get comfortable with.
The interview is a conversation, and conversations are fluid things. Trying to force the interview so as to better check off your talking points in the order you authored then beforehand may make things feel artificial and forced. It’s better to memorize your agenda and only reference it for talking points if you need to.
Small talk can be big
Ask them how their day is going, or what the weather is like. I’m serious.
Not only will it help the person being interviewed feel more comfortable, it may segue into topics covered in your script. If you’re free to approach the script more conversationally, you might be able to check some topics off before you even get to the prototype.
Even though we use screener questions to ensure good candidates, we also typically include some background questions in the beginning of the script. While the script questions don’t reiterate what was asked in the screener, the answers you get from the user are sometimes redundant.
This is a good thing! There can be a lot of hidden nuance in those demographic questions—sometimes serving as a launching off point for things you may not have been initially considering. If they’re not, it’s easy to get through them and get on with the rest of the session.
For one client, learning about one participant’s background gave us insight into a high impact portion of the product that made assumptions about circumstances that would have otherwise not have been uncovered. This allowed us to make adjustments that would preemptively address issues about a potentially very sensitive topic area.
Don’t be afraid of uncomfortable silences
Interviews are conducted to get insights. To get insights, you need to get people talking. While it helps to have a friendly and approachable demeanor, sometimes that means staying mute until they fill the void with chatter. It’s an old psychologist’s trick, but it’s a good one.
I’m not saying you should be completely silent for the entire process. You should answer questions if prompted, and should chime in from time to time to prompt or nudge them along. However, you also need to let your new friend squirm through uncomfortable things and resist the very real urge to help them out.
Pump the brakes
One of the times it is appropriate to speak up is when the participant is blowing through a prototype and not saying anything. While it’s great that things are working well enough that the participant can navigate with little effort, the problem is it’s working a little too well.
If they’re not talking about their experience as they go, it’s not a very helpful session. Positive interaction can be just as important as negative, but without the details of what they like and why articulated, it’s difficult to then later quantify this without guessing intent.
Don’t be afraid to walk a participant back and prompt them to explain themselves. A quick, neutral question can be great for getting the conversation flowing (“I noticed you chose X right away. Why’d you go with it?”).
You don’t have to look smart
You’re conducting the interview. Congratulations! You’re a professional!
Trying to demonstrate to the participant that you know your stuff might not work out as well as you think it will. Lording your expertise over the participant may be interpreted by the participant as their opinions not being worthwhile—the exact opposite of what you want.
I try to stick to neutral questions asked in plain language. It can be helpful to do your homework and know what industry jargon means beforehand, but I find that getting the participant to explain what the terms means to them to an outsider can oftentimes lead to important revelations.
Be mindful of the Hawthorne effect
Conducting user interviews is important, in that it helps uncover otherwise unvoiced motivations, frustrations, desires, and needs. That being said, it is very easy for a whole host of biases to worm their way into the process.
One bias I’d like to specifically address is the Hawthorne effect. It occurs when “individuals modify an aspect of their behavior in response to their awareness of being observed.” It permeates the entire user interview process.
Most user interview participants will probably want to give you what they perceive as the “correct” answer—the answer they think you want to hear, even if isn’t personally what they are feeling. This occurs even if you tell them not to be concerned about it. While things like analytics and A/B testing can combat this phenomenon, you have to be sneaky to address it during a user interview.
Use a head fake: ask the participant to complete one task, but make sure that the task’s completion flow covers the thing you actually want feedback on. It’s a little bit devious, but ultimately a victimless crime.
A practical example: Let’s say we want to test the effectiveness of the redesign of a product customization feature. Ask the participant to complete a checkout flow, making sure the redesigned product customization screens are included as part of the session. This way you’ll get your participants interacting with the thing you want to test, with less concern that they’ll be giving you the answer they think you want to hear about the thing you actually care about.
Introverts are people too
The final piece of advice is to remember that while it may be easier to talk with a participant who is outgoing and charismatic, you need to not disproportionately weigh that feedback when compared to the points raised by quieter individuals.
The observations voiced by introverted and introspective participants are just as valid as their more gregarious peers, they just may be communicated in a less outwardly excitable way. It’s your responsibility to determine what feedback is worth integrating into the product during the synthesization process, regardless of how that feedback was expressed.
User interviews, when conducted with care and discretion, are great to help shape your product into something people find desirable and meaningful. Hopefully these tips can help your next interview session be the best it can be!
