Data vs Expert Opinion: A guide to better decision making
If you are a designer working in an organization that has a deeply rooted culture of expert judgments, and if you have experienced a battle between data and expert opinion, this write-up is for you.
The Value of Data-Informed Decision Making
When making design or product decisions in general, we try to back the decisions with data. In any design process, we make many assumptions at the beginning and as we go along through continuous experimenting, we validate these assumptions until we reach our goal. As I have experienced, the most straightforward and effective method of validation is to test with real users. In processes such as Design Sprints, it recommends you to validate your assumptions with 5 end users and the user tests are where your assumptions meet the reality.
Fail fast to learn fast
If you do not validate the assumptions before you release your product or even before you build it, the validation that you get at the hands of end-users after the product is built and released is extremely costly. Therefore, data-informed decision making should be at the core of any design process.
The culture of expert judgment
In many organizations, there is a deeply rooted culture of expert opinions and judgments. Without letting the designs to be validated at the hands of end-users, we tend to use subject matter experts to review and validate the designs. You might be validating your assumptions with these subject matter experts due to various reasons such as limited access to end-users or maybe it had been working well so far. If we look at the success story of Apple, Steve Jobs and Jony Ives have been subject matter experts and they probably got it right most of the time? Maybe.
The battle between Data and Expert Judgements
When designing or managing products, most of the time it is given that there is a final decision maker, a subject matter expert, no matter whether you like it or not. When decisions go wrong, everyone points the finger at this person, blame it on her, and wipe their hands off. When a product fails due to a decision made by this decider, the designers usually use it as an “I told you so” opportunity. But at the end of the day, it is the product that fails. So as designers and experts who want to do what is best for the product, what can we do in this context?
3 things you can do to make better decisions using Data and Expert Opinions
1. Appoint a decider
If you are in the context of this conflict, appoint a decider from the very beginning of the project. This gives a clear idea to everyone involved that there is a clear decision-maker. Her decision is the final decision. Once the decision is made, everyone should commit and proceed even if you disagree with it. Make this clear to everyone involved.
2. Give everybody a voice
Even if the decider has the power to make the final decision, it can be influenced by the opinion of others. Without expecting a decision from the decider out of nowhere, help the decider by giving your opinion. This is why in many design workshops, we vote for ideas with transparency for the decider before making her final decision. When the whole team has voted for option A, there should be a really good reason why the decider should go with option B. Encourage everyone to speak up (or silently vote).
3. Always bring comprehensive data to the table
The quantity of data matters. The clarity of data is crucial. The result that you get by testing a design with a single test user can be very easily outdone by a single expert opinion. But if you can gather data from 5 to 10 test users, then it is easier to convince the decider on what the decision should be. And most importantly, make sure that the data is presented in an understandable format to the decider.
I would love to hear your stories regarding this topic. Write to me or send me a message on Instagram and let’s have a conversation.
