Buzzwords or phrases are great to try to convey an idea quickly and easily to a wide number of people, however, they should not set your policy on how to do things. One particular phase “Fail Fast, Fail Often” has been used a lot when talking about Innovation, Agile Development and now Dev Ops. I hate it!
It is important to handle failure within an innovation program differently than your normal day to day business. And most companies struggle to separate different types of failure. It is very hard to praise failure as suggested by some innovation leaders, so there is now the focus on faster learning. However, I think it is better to try to ensure you see each iteration for what it is an experiment.
If we think of traditional experimental research you have three outcomes; confirm, disprove or inconclusive result. Each one is used to describe the result of a question or theory that is created prior to the experiment. There is no reason why innovation idea incubation can not use the same approach.
The first task is to be objective. The person defining the experiment needs to be somewhat removed from the idea creator and so does the person that evaluates the results. It could be the same person that defines the upfront goals and then checks the results, but like testing code it is always better to have different people do unit and functional testing to ensure that you check all options and remove assumptions. You should definitely not have the idea creator be the sole person defining and reviewing experiment results. Split up the responsibility.
The second more skillful task is how you define the test. A little like survey question writing, this is something that you should not leave to anybody. Defining the test needs significant skill and understanding to get it right. A bad experiment will lead you to the wrong conclusion. I find that it is helpful to think of what would happen if your new great product or service was not there and how the experiment would react. Taking this subtractive approach helps you think of how your new offer or service adds value and therefore you can focus on measuring the value it adds. You just need to try to understand what the minimum level of value makes sense.
Innovation or any iterative design process is an experiment. For you to get meaningful results you should spend as much time defining the experiment as creating your prototype and ensure you can objectively justify taking the next step forward based on the results.