Prioritizing Knowledge
The inspiration for this post is John Cutler‘s excellent twitter thread on prioritization. It’s well worth the read. This post builds on that inspiration using Roger Martin’s concept of the The Knowledge Funnel.
One big takeaway of John Cutler’s thread is when deciding the sequence of what to do in product management, consider the big picture and think of the impact of what you will do next on what you will know next.
What I like about Roger Martin’s concept on knowledge funnels: consider the big picture and think of what you know about value.
Product management and data science is all about managing the knowledge funnel. Your ability to manage this funnel is predictive your ability, and those you lead, to succeed.
Let’s start at an axiom and a judgement. There are a set of things that are unknown. It’s true to the point that it can be an axiom. We don’t know a lot of things. And isn’t that exciting? There is a lot of mystery that’s left for all of us to discover. The judgement is that isn’t something to be afraid of. That’s something to be thankful for.
Then there’s a set of things that have become known over time. We know a lot better today than we did before. This knowledge and be divided into two classes: firm knowledge and secret knowledge.
There is some secret knowledge that is held by people within the firm, but can be hidden away from the firm. This may include knowledge like how to build a brand. Or how to run a forensic audit of the books. Or how to build an effective API strategy. Or how to lead a product development group. Some people are often recruited for their knowledge and their capacity to apply their knowledge. Some of that knowledge is never learned by the firm. Sometimes the firm wants that knowledge. Sometimes the firm doesn’t want it.
There is some knowledge that the firm teaches people. It’s a set of knowledge that’s known well enough to be reliable enough to be useful. Most of this knowledge comes in the form of heuristics. And if you work in an organization, you learn loads of heuristics that you are told to use every day.
Sometimes you learn heuristics from blogs. Like this one. Why prepare for a meeting. Why build an agenda. Why using an [ACTION REQUIRED] tag is effective. The bullet point. There are tens of thousands of these rules of thumb for working. Some heuristics are more valuable than others. Some are kept secret from the firm.
Sometimes a firm competes on its ability to transmit these heuristics from one generation of management to another. Some of these webs of heuristics are extraordinarily difficult for other firms to imitate. Some firms compete purely on their ability to transmit their heuristics and maintain them.
And then there’s a set of knowledge that is so well known that it’s standardized and optimized to the point of an algorithm. This is known to the firm. It runs on the firms’ servers. It runs all day and all night. It’s documented and it is run and maintained by a group of people.
A lot of machine learning and productized data science focuses on converting heuristics into algorithms, and teaching machines to make those algorithms better. Prioritizing this section of the knowledge funnel is technically and mechanically straightforward. Feature engineering is known technology. Managing technical debt is known technology. Cleaning instrumentation is known technology. All of those processes are known or knowable. They generally vary a bit on context. But they’re known or knowable.
What makes it hard for many firms to drive a heuristic into an algorithm is multi-faceted. It’s mostly traceable to human factors. Socially and emotionally this is a very hard process for people. All of those processes are less known or knowable. They vary tremendously on context. They vary tremendously on the culture of the firm.
The front section of the knowledge funnel – converting mystery into heuristics – is much more difficult and less straightforward. It’s hard technically. It’s very hard socially. Because it involves the unknown, it generates a lot of fear in people. Because it isn’t linear, it generates a lot of aggression and impatience. There isn’t a clean, clear, way of doing this work. It isn’t reliable. There isn’t a guaranteed path to success. Failure is very common.
This poses a considerable management and leadership challenge.
What makes Cutler’s thread awesome is how he explains a way to manage the relationship between what you know, what you think you know, and what you do not known.
A lot of energy in product management goes into managing the certainty. Often, prioritization, the act of sorting activities that are worth doing now over activities that are not worth doing now, is focused on the certain and the appearance of certainty. Many customers are very certain that they want a given feature right now. Stakeholders can exhibit remarkable certainty in what they want. And yet, prioritization should also take into account the uncertainty and driving knowledge through the funnel.
Cutler calls out making sense of systems about value, generating new ideas about it, and sorting on that basis. And, he offers a nice insight into how the unknown can be managed by way of bets.
Cutler makes the distinction between Level 0, 1 and 2 bets. Level 0 bets are fundamental – to be confirmed over 1 to 3 years. Level 1 bets are important, to be confirmed or rejected over 3 to 9 months. And Level 2 bets are interesting, to be confirmed or rejected over 1 to 3 months.
Those bets roughly align to a systems understanding of value, a lever based understanding of value, and a settings-level of value. OKR setting happens at Level 2. Big mysteries about big systems are covered in Level-0. Medium sized mysteries are explored in Level-1. Small mysteries are covered in Level-2.
It’s great because it helps leaders manage the unknown and the fear.
Takeaway: Consider what You Know, What You Don’t Know
Think about what you don’t know and what you know when you are prioritizing. If you’re on the left side of the knowledge funnel, pulling mysteries into heuristics, you might prioritize knowledge of riskier Level 2 bets over easy Level 2 bets. If you’re on the right side of the knowledge funnel, pulling heuristics into algorithms, you may prioritize more conservative Level 2 bets.
It also helps to put the smaller bets into larger context, and offers a nice frame. It can help you manage the fear of the unknown against the anger of the certainty. It offers a way for leadership to set the term and timing of major, paradigmatic, conflict. And, it offers a way to retain, or even accelerate, the rate of learning.