You can skip straight to the list of falsehoods if you recognise this article’s genre from the title or you’re not someone who likes caveats and explanations.
Software teams often make incorrect assumptions about user feedback. Being aware of how differently one person can interpret something from another’s intent is powerful.
The bulk of this article is a list of many possibly incorrect assumptions. In most cases they are true, but assuming that they are always true leads teams to over- or under-reactions: dismissing a serious issue, or losing productivity in a scramble to meet imagined emergencies.
The list is in the tradition of many other lists of “falsehoods programmers believe”. In the genre a list of common simplifying assumptions is presented with the intention of making the reader aware of edge cases. You can see a collection of similar articles here.
The list below may seem absurd at times, but rest assured that they are based on real life circumstances. The common thread is that people in the detail (software teams) communicate very carefully about their software, and lose sight of communication patterns in other spheres. Insiders assume:
- preciseness (e.g. assuming that “some” intentionally conveys “not all”)
- shared context (e.g. knowledge of previous work)
- consistency (e.g. everyone uses the same name for the same feature)
… that is not a given in normal communication.
Productive relationships form when there is some give and take where the most involved users start to adopt some of the team’s communication style, and the team learns to more carefully assess communication from those further away. This is doubly true with internal users and layers of management.
And finally, here is the list.
Falsehoods about user feedback
- When asked for feedback about a feature, all user responses will be related to that feature
- When multiple users report a problem using the same words, they are all talking about the same problem
- When a user states “there is a problem”, there is only one problem
- When a user states “there are multiple problems”, all of the problems are different
- When a user states “there are multiple problems”, some of the problems are different
- When a user states “X is broken”, they know how X was intended to work
- When a user states “X is broken”, they have used X
- When a user states “X is broken for someone else”, they have seen someone else use X
- When a user states “I heard X is broken”, they heard someone else talking about X
- When a user states “X is broken”, a feature named “X” exists
- When a user states “X is broken”, they are talking about feature X in your software and not other software
- When a user states “X is broken”, X is broken for all cases
- When a user states “sometimes X is broken”, X is not broken in all cases
- When a user states “sometimes X is broken”, they can communicate all problem cases
- When a user states “sometimes X is broken”, they can communicate all problem cases that they’ve seen
- When a user states “sometimes X is broken”, they can communicate any problem cases
- When a user states “sometimes X is broken” but they cannot communicate any cases, X is not broken
- When a user states “X is broken when I do Y”, Y is the only cause of X’s failure
- When a user states “X is broken when I do Y”, Y is the only cause of X’s failure that the user has seen
- When a user states “X is broken again”, the same bug in X is at fault
- When a user states “X is broken again”, they are referring to a problem that stopped and restarted
- When a user states “X is slow”, they know how quickly X was intended to work
- When a user states “X is slow”, they know whether X does eventually work
- When two users state “X is slow”, they both expect the same speed
- When a user states “there is a new bug today”, they are aware of whether the bug existed before today
- When a user states “there is a critical bug”, they are aware of how severe it is relative to other bugs
- When a user states “there is a critical bug”, they are aware of how severe it is relative to other bug reports they have made
- When a user states “there is a critical bug”, they are aware of how many people it impacts
- When an internal user states “there is a critical bug”, they are talking about code in a production release
- When a user states “this bug is urgent”, they are aware of other work that is less urgent
- When two users state “this bug is urgent”, they need a fix in the same time frame
- When asked “is it fixed?” the users will check the software before responding
- When asked “is it fixed?” the users will respond about “it”
- When asked “is it fixed now?” the users will check the release that contains the fix
- When asked “is it fixed now?” the users will respond about experiences that they’ve had more recently than the fix was implemented
- When asked “is it fixed now?” the users will understand that they’re not being asked about an as-yet-unimplemented feature
These statements might usually be true, but they are not always true. Ignore this at your peril.