An orthodox Software as a Service (SaaS) business is, in part, math that an organization tries its best to manage. Think about all the math that goes into the construction of a typical SaaS firm. At the core there’s some customer with a job: a goal against which the customer wants to make progress. They can have a mathematical representation in a database somewhere. A bunch of technologists write some code, which is all math, and a bunch of creatives take a few photographs, which expresses itself a mathematical representation, and some data is Created Read Updated and Destroyed in a database somewhere, which is all just more math. And it’s all abstracted by yet more math at the processor[…]

Let’s start with a story. Daan did a traditional fast follow. He calls it Netherflix. His story was: “It’s like Netflix…for The Netherlands!”. At first, he buys rights on the cheap, pays for digital subtitling, and has a successful kickoff. He gets through to 10% household penetration, or roughly 700,000 subscribers, with an annualized gross revenue of about 60 million Euros. The strength of the Euro lets him raid the Anglosphere and he can stock 10,000 hours of content reliably [1]. He gets through the struggle of getting his stack to deliver content and minimize churn. He’s able to host and deliver 10,000 hours reliably, in spite of supporting video players across 11 different front end platforms, and the costs associated with hosting,[…]

How do you, and those around you, deal with failure? Because if the answer is anything but “well” or “every fail has a lesson”, then progressive, iterative, experimentation and AB testing really, really isn’t for you. Go away. Stop reading. It’s not everyone. The way to build an experimentation and testing bench is entirely by setting sail for fail. How to build your Experimentation and Testing Bench Your bench will be built: 95% with culture 5% with technology The technology is well thought out and several vendors are excellent in this space. The technology isn’t a problem. Sure, there are a few pretty shockingly bad systems out there. Every industry has a phony. Building a testing bench begins with policy.[…]