Even Good UX Practices can Result in a Poor Experience
Working in UX is a blast: you get to talk with people, get to know their challenges, and work to resolve them. After spending time as a front-end developer, a business analyst and a product strategist, I’m now happily sitting in a role that straddles the how, the what and the why, because I start early (research) and continue through the entire process (designing the solution).
But sometimes, you can do everything right, and still end up with a dud interaction, feature or product. It’s the outcome every entrepreneur, product manager and designer fears. And it happens all the time.
Earlier this year I listened to the audiobook of Annie Duke’s Thinking in Bets: Making Smarter Decisions When You Don’t Have All The Facts. And boy, did it resonate with me!
Early on, she points out the difference between chess and poker (Annie was a world-class poker champion, so much of the book draws from those experiences). With chess, both players have all the information: they can see the board, they know all possible moves, and it’s really a matter of optimizing decisions along the way. As well, there’s a natural feedback loop: a player gets immediate feedback on each move.
In contrast, there’s a lot of uncertainty in poker. You don’t know which cards your opponents hold. You don’t know if they’re bluffing. There’s some level of luck involved with the hand you’re dealt.
In her book, Annie introduces the term “resulting”. This is the problem of assuming too tight a relationship between the quality of a decision and the quality of the outcome. We assume good outcomes come from good decisions, and poor outcomes, well… we prefer to chalk those up to bad luck than our own faults. While luck may play a role in some cases, our unwillingness to recognize our own poor decisions makes it harder for us to improve.
It’s far better to recognize that there are good decisions and poor decisions, and good outcomes and bad outcomes, and they’re not 100% correlated. Other factors, including luck, come into play, yet the only thing we can control is our own decisions, made with incomplete information.
We’re occupying this space of uncertainty when we design products. And while it can’t completely mitigate all risk of failure, UX research can help improve our decision making, and therefore improve our chance of success.
Note above, the only path that doesn’t allow for the chance of a good outcome is to make a bad decision AND fall upon some bad luck. You can’t control luck, so… work on making better decisions!
UX Design is about Patterns
In both chess and poker, we start by learning some basic patterns or recipes. These are plays that apply under certain circumstances, that may lead to positive outcomes. In UX design, these may be standard design patterns or usability best practices. Things that have been shown to contribute to a positive outcome.
Note that I said “under certain circumstances”, and this is where we can get tripped up. Not all patterns work at all times, and sometimes breaking away from a known convention or pattern can be the best chance you have to win big.
How do you know what’ll work? You need feedback. You need to try and learn, and this is where UX research comes in. Try something, see how it turns out, and come back and update your view on that pattern.
The problem is when we try something once and it goes poorly, so we surmise it’ll never work. This is that trap of trying to “validate” a design with a small sample of users.
Qualitative research can only show us what is possible, it doesn’t help us to know what’s not.
A wonderful example cropped up recently for me. I was speaking to a customer about a query she’d run on our system. I started to ask her about how she formatted the query, and she was adamant that this was the only way to do so. Her response was something along the lines of “No! No, no, no. You’d never format it the other way. That’s just wrong.”
Pretty strong signal, right? Sounds like we don’t have to worry about supporting other ways to format the query.
Except… we had data of user behaviour that showed us that some people DID format it the other way. So maybe this customer would never do it, but we weren’t only designing for her. We needed more information to design a solution that worked for more than just her.
You can’t take a spoken “never” from one person and generalize it across the population. The only “never” you should trust is the one you observe [or rather, don’t observe], through quantitative data.
UX Design is About Trial and Error
Even the best poker player loses sometimes. Maybe they make a poor decision, maybe they have some bad luck and it seems the cards are stacked against them 😃.
How do you know which it is? You can’t, unless you try, and learn from it. The best way to improve your decision-making quality is to make decisions and then apply what you’ve learned to refine your model. It IS a model, so it’ll necessarily be incomplete, but at least it’ll be closer than when you started.
When it comes to UX and product design, this means testing and iterating on concepts. Ideally, before you’ve invested months of time and engineering resources to build something for production.
UX Design is About Playing The Long Game
Decisions have cascading effects, and we have to consider the trade-offs and opportunity costs of our decisions. When we choose to do one thing, we are choosing NOT to do other things. We believe that this one thing will give us the best return for our efforts.
This may be choosing to introduce one one feature as opposed to another, or optimizing for one user group rather than another. In the case above, I mentioned our power user customer who didn’t need us to support alternative formats. Are we designing for her, or are we trying to help less sophisticated users by offering them some guardrails?
Over the years, I’ve worked with a lot of product managers who came through the customer support team. They have deep empathy for customers, they know where the system is failing individuals.. but I’ve found they’re often challenged with prioritization and balancing the implications of solutions across other users.
When a customer support team member is helping a customer, they’re troubleshooting the problem for a market of one. Success is whether that customer can get done what he wants to. But that doesn’t seamlessly scale to product management, where changes to the product don’t just impact that one user. You have to consider the inverse market: every customer who doesn’t suffer from the problem with customer has. How does your proposed solution affect them?
And then there’s dumb luck
The title of this article is “Even good UX practices can result in a poor experience,” and it’s the unfortunate truth. Even if you perform generative research, and iterate and test your solution with customers along the way, there’s no 100% chance of success to any product launch.
The competitive environment, your go-to-market strategy and your business model will all play a role in your product’s success. There will be decisions to make along the way in each of those arenas as well, as well as a good chunk of luck.
The best we can hope to do is improve the likelihood of success in the sphere we have control over (in this case, UX design). Gather as much relevant information as we can. Use that to create a model for our decision-making. Acknowledge, but don’t over-emphasize, the impact of luck. Embrace the uncertainty as a challenge, and have fun!
