Undersampling of Failure
A couple of months ago I consulted a very successful company where the leadership team was convinced that it could create a competitive edge by understanding its success better and deeper than it did.
The team’s idea was to do an in depth analysis of the successful outcomes that the company had created over a 12 month period in order to understand what factors, people and patterns drove the success. However, there is one big problem with this approach. The problem is called “undersampling of failure” – also know as the “survivorship bias”.
Imagine the following scenario for a moment:
Companies that pursue risky strategies, for example by putting all of its resources into one technology, typically achieve either high or low performance. They succeed BIG TIME or they fail BIG TIME. As time passes, the successful companies thrive and the failed companies go out of business. Someone attempting to draw lessons about successful companies would therefore see only those companies that enjoy good performance and would infer, incorrectly, that the risky strategies led to high performance. However, the truth is that the corporate cemetery is full of failed companies with risky strategies, but this evidence is silent.
My point is that if you want to understand success it is not enough to study successful outcomes. In order to really learn about success we need to consider a full sample of strategies and the results of those strategies. That is why the approach of the leadership team I mentioned in the beginning of this blog post was wrong.
You don’t want to study success to learn what strategy was used but rather study strategy to see whether it consistently led to success.
If you want to learn more about this idea, HERE is a great article about it.