This blog is a call to action to use better innovation metrics. Innovation means many things to many people. I have always suspected that was because without proper metrics it is difficult to define what makes innovation. Take for example success. What is innovation success? An example of a successful new product is the first iPhone. Yet, arguably, the paper vaccination cards that were widely used during the pandemic had more economic impact, since these paper forms facilitated a safe return to work and school by identifying vaccinated individuals.
Return on investment and market share
For firms, the most often looked-at success metrics for innovation projects is the return on investment followed by market share captured. For the iPhone these metrics are relevant – the iPhone enabled Apple to bring down market leader Nokia at the time. However, the vaccine card's success hinged more on widespread acceptance.
The limitations of return on investment and market share as metrics are extensively discussed in management literature. In essence, these metrics can be valuable to evaluate success in hindsight, but they are useless for managing ongoing innovation projects or evaluating alternative options.
While each innovation project is unique, the process leading to a solution follows a similar structure, regardless of the product. Both the iPhone and the vaccination card required exploration, scoping, investment decisions, testing, building, and scaling. These universal activities form the backbone of each innovation journey.
Table 1: The development of the iPhone versus the Vaccine Card
|Scout for technologies
|Ask other countries how they are going to track and check who has received vaccines.
|Define project purpose and requirements
|Define the scope & how much to invest
|Define what the solution should do. What are the requirements to make widespread acceptance possible
|Test critical assumptions and technological challenges
|Build mockups to get user feedback, test critical parts of the technology
|Create different versions of the card – to both understand what should be on it and the quality of the paper to be used.
|Create a working solution with minimal resources
|Put everything together in a small box (phone case) and ensure the user can easily understand how to operate the phone. Build the production process.
|Settle on the final design, create distribution channels, and put guidelines & regulations in place for the use of the card. Create a printing process and distribution plan for the card
|Expand the solution to the widest audience
|Market the solution, build sales channels, scale production, and create a distribution network.
|Ensure all places that provide vaccines use the same administrative process. Print and distribute the cards.
Note: This table provides a very simplified overview and is based on estimations, not verified activities.
The table outlines the phases for both projects, emphasizing the uniformity of phases but acknowledging the wide variations in what these activities entail across innovation projects.
These variations are what make measuring the progress and performance of innovation projects so difficult. No two projects are alike.
To enhance uniformity, projects can be categorized at the start based on type (new product, service, process, incremental, radical), with specific metrics developed for each category.
However, even with categorization, the unique nature of activities within each project poses challenges in evaluating the progress and performance of each project team.
Assessing Progress & Performance
Traditional project management tools, whether using Waterfall or Agile approaches, require past experience for estimating task durations and resource needs. Data that is typically missing for projects like the iPhone and the vaccination card.
For the teams working on the iPhone or the vaccine card, they could estimate how long for example it would take to take the solution into production, but they had no reliable data to back up these estimates.
Thus, when holding innovation teams responsible for typical project progress and performance metrics, you are evaluating their estimating abilities rather than using a meaningful metric for predicting project outcomes.
Stage gates provide moments to reevaluate the progress and performance of innovation projects, aiming to approve further investments. After each phase, project teams reflect on accomplishments, failures, and lessons learned, guiding management in investment decisions.
Stage-gate processes typically involve 5-9 check-ins with project teams over 6 months to 3-5 years. That means that the information that is available about the progress and performance of each team is available sparsely and typically outdated.
If innovation teams consist of professionals – that is people in an R&D or innovation lab whose task it is to bring ideas to practice – the rather low frequency of check-ins is probably good enough, as everyone knows what is expected and the careers of these employees are closely aligned with the success of the projects they are working on.
Yet, what we see in practice, is that most innovation teams consist of employees who have never engaged in an innovation project before. Being part of an innovation team comes on top of their job duties and is not something their career depends on.
For such teams, the infrequent Stage Gate checks are rarely sufficient. They need more support and frequent check-ins to maintain momentum.
Many papers and books have been written about stifling innovation and creativity by imposing too much control.
To provide each team with the best balance when it comes to support, autonomy, and oversight – Stage Gating is currently the best available option. It gives teams the autonomy to define their course of action in between each phase and come up with the best solution for the tasks at hand. Management knows what the team is working on – based on the information provided at the previous gate – and can provide the kind of support the team needs to succeed.
However, when it comes to oversight and progress reporting, the gates are not always frequent enough to ensure resources are not wasted on projects that are going nowhere or other frivolous activities.
Data that is available
For progress reporting, many innovation managers grapple with determining relevant questions for teams to report without creating bureaucratic complexities.
Monthly progress reports typically inquire about the tasks accomplished, the support needed, and the obstacles encountered. The answers to these questions will certainly help to provide better support for each team. Unfortunately, this information won’t inform management about the likelihood of the project team’s success.
Innovation metrics that management needs
As such, despite all our knowledge about how to manage innovation, there is still a lack of basic tools and metrics to assess if resources are allocated to the most promising teams and whether the right investment decisions are being made.
What management needs, are insights into the progress and performance of innovation teams on an ongoing basis and in a way that will provide information about:
- The likelihood the team will succeed
- The support the team needs to increase the likelihood of success
- If future investments are warranted.
Lack of this kind of progress and performance information results in lengthy and opinionated discussions about which projects deserve resources and future funding. It makes decision-making political and managers reluctant to invest – because they don’t like betting on the wrong horse.
Fail fast to succeed sooner
The mantra of "fail fast to succeed sooner" does not help in this regard. The acknowledgment of failure as an integral part of the process often masquerades or justifies bad decisions. Worse, it has normalized ineffective and inefficient innovation processes.
It seems that the management of innovation projects has remained a black box for too long. The current status quo is acceptable because failing is considered an essential part of the process, because each project is considered unique, and because there are no reliable innovation metrics to track the progress and performance of innovation project teams.
Taken together, it makes innovation projects cost most organizations more than what they will ever get in return from these investments. As a result, in many organizations, innovation has turned into a nice to have, instead of a process that reliably can be used to secure growth, profit margins, and guarantee future success.
Better Innovation Metrics, please!
Does it need to be this way? For the past 20+ years, I have asked myself that question, and the answer is a resounding NO.
Let’s take a closer look at Table 1. While the activities of the projects differ widely, the objectives of each phase are similar, even for these widely different projects of the iPhone and Vaccine Card.
Providing guidance to hundreds of teams has confirmed that while each team is unique in terms of what activities it undertakes, there are behavioral and learning patterns that inform the progress and performance a team is making in each phase. These patterns can be tracked and used to identify how a team is doing in real-time.
From these patterns, indicators can be created that provide insights into which team needs intervention and why. Most importantly, these indicators can demonstrate which teams deserve funding because they are more than likely onto something.
Metrics to Money Program
Ten years after founding Organizing4Innovation, we are close to perfecting these innovation metrics – and are looking forward to sharing what these innovation metrics are and how to use them in our Metrics to Money program. This program will start March 6th.
For more information about the Metrics to Money Program, see: Metrics to Money 2024