Thoughts on A Scientific Approach to Entrepreneurial Experimentation
What a fantastic read from Camuffo, Cordova and Gambardella! If you haven’t read A Scientific Approach to Entrepreneurial Experimentation, you’re missing out. It’s a great read. And not only because it reinforces my own preexisting biases, but also because there are challenging bits in there.
The core finding is “We find that entrepreneurs that behave like scientists perform better, pivot to a greater extent to a new idea, and do not dropout less than the control group in the early stages of the startup.”
The authors focus on a key behaviour that scientists exhibit. A scientist has two types of skepticism – skepticism that something is true, and skepticism that something is not true. Those represent two types of error, helpfully numbered 1 and 2. The false negative and the false positive are pervasive and they mess up our relationship with reality. It’s tough for scientists to manage. It’s especially tough to manage skepticism through founderhood.
A founder believes. Often, they’re convinced that they have product-market-solution fit. Or, if they haven’t, they’re convinced that they can discover it quickly, and achieve traction. Often, they believe so much that they discard evidence that doesn’t fit into their belief system, and embrace evidence that they do.
Kuhn observed that scientists can behave in that way, too. Evidence that confirms the dominant paradigm is added to the puzzle. Collect and contribute six pieces, have them recognized by the top tier guardians, and you are rewarded with tenure. Evidence that doesn’t fit into the puzzle is shoved to the side and often ignored and usually derided. Scientific revolutions occur when too many pieces are contradictory to the main puzzle and a group of people decide to play their own game with those pieces.
And that goes for entrepreneurs too. Revolutions occur when too many pieces, especially those that are coloured red with negative revenue, are collecting at the side of the table. For a startup, the accumulation of false positive pieces kills it. Scientific communities can persist for decades beyond any useful contribution. The skill of thinking like scientist is probably even more valuable to an entrepreneur than it is to an academic scientist.
How does a founder think like a scientist? By starting with well defined hypotheses – those that are clear and falsifiable:
“An explicit definition of clear hypotheses helped to identify the key areas that needed validation, reducing the potential biases deriving from keeping assumptions implicit: if the hypotheses are not formulated, entrepreneurs can look for (and get) only general feedback and forego important aspects of the business model, for example weighting equally components of the business model that instead contribute asymmetrically to value generation.”
(p. 4)
Think logically. Design experiments objectively. Look for biases. List biases.
The bit that I really like about the study is that those who were taught to think like scientists and got training in some of the core orthodox frameworks used in tech entrepreneurism, earned more revenue. In fact, average revenue was 3,000 euros higher. That’s huge because we’re talking about the earliest stage tech companies. That lift is significant. There’s more detail in the tables down in the appendix that certainly suggests (but doesn’t prove), to me, that it’s a huge factor in success or abject failure.
Clear hypothesis creation and bias checks can help the entrepreneur manage themselves. The benefits appear clear, given the stakes and the risk.
There are challenging bits. The authors spend quite a few paragraphs talking about the effect of drop out. It’s easy to empathize with why it’s important, as it introduces a kind of bias, and that can certainly cause a bit more skepticism about the validity of the results. After reading their explanation on how they assessed the difference between failure and drop-out, I’m comfortable with the risk.
Conclusion
It’s a great read, with solid advice, because it gives the entrepreneur more control over their destiny. As the authors themselves note, so much of the literature in explaining success and failure of startups is rooted in exogenous, environmental factors, and much less attention is given to the engine of the startup itself: the founder(s). Any tool that an entrepreneur can pick up to tilt the odds in their favour is a valuable one.