News

Overconfidence as the biggest pitfall to innovation success and what to do about it

Overconfidence drives poor decisions. We humans, tend to focus on the evidence for outcomes that we believe and want to happen, while ignoring other possibilities. As a result, overconfidence is the biggest pitfall when it comes to bringing an innovative idea to practice. Now, research has shown an effective strategy to avoid overconfidence: focus on the unknowns.

Innovative ideas come from creative minds. People who can think beyond what is in front of them, dream about overcoming the impossible, and imagine beyond the current horizons. And it requires confidence to act upon these ideas.

Confidence versus overconfidence

To bring innovative ideas to practice, you need an enormous dose of confidence. Otherwise, why start if you don’t believe you can beat the odds and can make it happen!

At the same time, you need an equally large dose of realism, to anticipate and overcome all obstacles and barriers. Otherwise you will fail miserably. This balance between confidence and realism makes innovating so difficult.

No great innovator has been successful upon the first try. Not even Steve Jobs, Jeff Bezos, or Elon Musk. Finding out what does not work – safely, quickly and least costly as possible, while learning as much as possible – is the best approach to achieve successful outcomes.

Overconfidence will hamper this learning process, which is such an essential part of the innovation journey. Overconfidence makes you oblivious to what may not work. As a result, the team will fail to prepare for what may trip up their project.

Innovation teams don’t fail because of what they know, but because what they don’t know!

Given that so many things can go wrong during an innovation trajectory, you don’t want to make any unnecessary mistakes due to overconfidence. Especially not because these foreseeable mistakes will greatly reduce the trust your supporters, sponsors, and funders have in you and your team.

Reducing overconfidence

The most common way to try to reduce overconfidence is appointing a devil’s advocate in your team. That is, someone who will advocate for what could go wrong. However, that is a difficult role to play. At best, the person is the least popular member of the team. At worst, the comments of the devil’s advocate get ignored over time.

Encouraging teams to consider alternatives, is another approach. However, this searching for alternative pathways can often be seen as a waste of time, especially when deadlines are already tight.

New research findings

While the above techniques are useful, there is a better way of avoiding confirmation bias and overconfidence, recent research shows.

 of the Anderson School of Management, University of California, Los Angeles, and his team found that focusing on unknowns reduces overconfidence, while not impeding confidence either.

Great news for our innovation teams!

Solution: focus on the known unknowns

The research by Walters now shows that focusing on unknowns helps to avoid overconfidence. Overconfidence is largely a function of people giving insufficient consideration to unknown evidence. By helping people focus on possible missing information, they become less overconfident.

Even better, the research also shows that focusing on unknown information does not make people under confident. So, there are only upsides from explicating assumptions and defining the unknowns that could strike your innovation project along the way.

What does this mean in practice?

So how do you focus on known unknowns in practice? How do you identify known unknowns?

Known unknowns are defined by all the assumptions you are making about the future, your users, and your solution.

Euro Disney is a famous example of untested assumptions, or known unknowns, that lead to failure.

Euro Disney

Untested assumptions caused Euro Disney (now Disney Paris) to fail the first time around.

An example of an untested assumption that contributed to the park’s failure concerned eating habits. The park was a copy of the parks in the US and Japan, where people “graze” all day. The buried assumption was that Europeans would do the same as part of the American experience.

Euro Disney’s restaurants therefore were designed for all-day streams of grazers. When floods of visitors tried to follow the European custom of dining at noon, Disney was unable to seat them. Angry visitors left the park to eat and conveyed their anger to their friends and neighbors back home. The lack of dining options contributed to the failure of this first version of Disney in Europe, failed (McGrath & McMillan 1997).

Explicating assumptions

With the teams we coach and in our training sequence, we place a lot of emphasis on explicating assumptions that teams make about their market, their technology and their organizational context.

Explicating assumptions may sound easy. However, in practice it is challenging as often assumptions do not get articulated. Everyone just assumes.

These assumptions are your known unknowns. You suspect them to be true, but without further research or testing, you cannot be 100% sure.

The good news is that you just have to be diligent about the assumptions you make as a team. You don’t need to appoint a devil’s advocate or set aside time to come up with alternatives. You just have to be aware and keep track. Both are very doable for any team that is serious about their success.

Now, go out there, explicate your assumptions, innovate successfully, and change the world for your clients, organization, and profession!

More on the research

Below the abstract of the article and a link to the full paper.

Abstract:

“We propose that an important determinant of judged confidence is the evaluation of evidence that is unknown or missing, and overconfidence is often driven by the neglect of unknowns. We contrast this account with prior research suggesting that overconfidence is due to biased processing of known evidence in favor of a focal hypothesis. In Study 1, we asked participants to list their thoughts as they answered two-alternative forced-choice trivia questions and judged the probability that their answers were correct. Participants who thought more about unknowns were less overconfident. In Studies 2 and 3, we asked participants to list unknowns before assessing their confidence. “Considering the unknowns” reduced overconfidence substantially and was more effective than the classic “consider the alternative” debiasing technique. Moreover, considering the unknowns selectively reduced confidence in domains where participants were overconfident but did not affect confidence in domains where participants were well-calibrated or underconfident.”

A full version can be obtained online through Management Science.