The Value We Place On Certainty

It seems like a lot of people value certainty. People buy a lot of products and stories for certainty. Insurance. Investment advice. Forecasts. Indulgences.
Many entrepreneurs, in particular those in data science, sell certainty. What else is an F1 score other than a measure of certainty on some level? Given some inputs, our machine transforms them some way, which produces some statement about the past, present, or future, with some quantifiable amount of certainty, so that you can do something with confidence (or feel more secure). We sell certainty. And yet isn’t it curious about how much insecurity we’re creating while we do so?

It has always been easier to sample data from the past, pull a heuristic from it, and generalize it into some sort of policy to be applied in the future. (A lot of us do that a lot – training organizations instead of machines.) Originally, bookmaking, the act of creating a prospective experience for people (gambling or gaming), was all about that. The earliest era of the use of statistics to inform public policy focused on examining records of the health of children to write heuristics about alcohol consumption. Tracing the price of rice futures in Edo Era Japan was another massive application at the time (though I don’t think it really worked). Risk pooling was really an arbitrage play on how people think of the certainty in their lives. Claude C. Hopkins taught us the value of the experimental copy-writing in Scientific Advertising. What changed in the late 20th century was the reduction in cost of machine readable data, thanks to the semiconductor and the CRUD (Create, Read, Update, Delete) design pattern of databases. Suddenly we could record huge volumes of data, run them through mathematical models invented in the 1700’s (and really hyper-refined in the 1950’s), to generate smaller matrices of policies and prediction. We could order the world by ordering the data. And we used culture is to to scale the decisions coming out of that order. You wrote a policy for the bureaucracy to execute, and you set up a second bureaucracy to independently watch it. We used scientific management to drive productivity gains, capital accumulation, and improve society.

Later on in the 20th century and into the 21st, we used networks to reduce the latency between sensing the world and affecting the world. Gradually, we got it down from batches of data, to soft-real time flows of data, and later, to hard real-time systems. These capabilities accelerated the pace at which decision automation has spread. The most well known and threatening uses has been around self-driving vehicles, killer drones, and the surveillance state applications. The lesser well known use cases are logistics (from poop to books), power distribution, high frequency trading and yet better risk pooling. Automation is how we scale routine decisions.
And what of the cultural impact of that?

Much of this technology is feared because it creates uncertain futures. And yet, so much of what we sell, as data scientist entrepreneurs, is certainty. Certainty for who? And why?

For the data scientist leader, there’s greater certainty in training a narrow machine intelligence to automate a decision than there is in writing a policy, doing it and showing it, saying it and telling it, have them try it, praising them, and then having them do it by themselves. The cost is also a lot more certain. And, there’s an underlining certainty that there is value creation by automating something that is terrible for humans to do. The great fantasy is a core product team to do the creative work, and everything else gets outsourced and/or utterly automated. Wouldn’t it be great if narrow machine intelligences could do all your QA for you? Wouldn’t it be even greater if systems self-healed after they were accidentally botched?

So while we’re remaking the world the way that want it, creating the the type of certainty we want to see by simply going ahead and changing the world, what of the impact we’re having on others?

For the citizen, we make less certainty. Are they going to be replaced by a machine? Are they going to teach machines to work along side them? Are they going to experience better lives because of their work with machines? Are machines going to help them become more creative, more often, more reliably?

Creativity isn’t so reliable or certain. It’s affected by so many other factors. Think of all the things in your life that can crimp your creativity. Terrible weather. Terrible work environments. Terrible commutes. There’s less certainty in generating and sustaining a citizenry that can be consistently creative. Indeed, so many of the absolute worst impulses of capitalism is to increase productivity by doing just the opposite — by creating less security, less safety, and less liberty for people.

Coal country or the sharing economy? Can you tell the difference?

There is nothing really fundamentally different new about this wave of automation. Steam powered rotary motion, the railroad, the diesel tractor, replaceable parts, the assembly line, the radio, and robotics all disrupted base jobs throughout the economy. They transformed societies then, just as the narrowest of machine intelligence is transforming society now.

How did we handle those previous advancements? Leaders didn’t get ahead of mechanization and look at what happened to Europe in the 1800’s. Other leaders didn’t get ahead of it and look at what Europe did to them in the 1800’s. In looking back at Canada, sometimes our leaders did very well (1963-1968, 1939), and sometimes they stumbled very badly (1837, 1881, 1917, 1990). So there’s good reason for the instability we all feel.

It may be the case that our institutions have never been stronger to handle some of these transformations. This isn’t the 19th or 20th century. The social capital accrued into our institutions and our societies, in the West, is substantial. They’re not as brittle as they were back then. And while local governance continues to get roughshod by waves of trade liberalism and monoculturalism, what if they were able to adjust to absorb these shocks before it all dissolves into an opioid riddled nightmare?

At the core is a contradiction. In striving to create a better world and greater certainty through data science (turning data into product) and applying narrow machine intelligence, we’re creating a lot of uncertainty. We’re at the core of that. And there isn’t as much confidence or trust right now to ease us all over into the new world.

It would be fantastic if more people had the liberty to grab a pen and author their own place in it – to create the same degree of certainty for themselves as we gain for ourselves. That would be a better future.

What certainty are you creating for yourself, and what value do you place on it?