More data than we know what to do with

It was only towards the tail end of the second year of university when anybody tells you about the pearson tables. It’s a glorious thing, all alone in there, hidden away in PASW. You run it for a number of variables, and it gives you a beautiful matrix showing the strength and direction of relationships among them.

It can be disastrously misleading. Violence can dull sensitivity. Still, it can be used to rapidly validate mental models and rule others out.

I cope when I’m confronted with a large dataset.

I identify what is it that I’m trying to figure out. Then lay out all the independent variables that I think might relate into explaining that variation in something dependent. I keep a few notes. I’m about to go in deep and if I’m not careful, I risk being fooled by randomness.

(The longer you randomly look for a connection, you more likely you are to find something randomly significant. The risk is real.)

Then I run pearsons to quickly rule out the ones that don’t look promising. Then validate the two that do make sense. Chances are good that I’ll come out with a much better understanding behind the drivers of something.

It’s a coping mechanism when confronted with too much data. And it takes time to do.

There are other ways of coping, too. The average, the median, the mode. Standard deviation, skew and kurtosis. For the visual: the histogram and the normal curve.

The Key Performance Indicator is always supposed to be coping mechanism.

We’re confronted with more data than we know what to do with. It’s so much even with a very high degree of statistical literacy. It’s akin to knowing how to read and being confronted with a Library of Congress. Even reading the back covers is time consuming. Possibly unrewarding. Why would you want to?

I think most cope by simply tuning out.

And so do I.

I cope by selectively tuning out.

Instead of wanting to know everything, which is implausible, I simply try to know something useful. And something useful typically relates to a competitive advantage. I actively practice selective ignorance.

I need something better than that. Because somehow I don’t think the banner IGNORANCE IS STRENGTH is a winner. Do we like amnesia?

Perhaps the construction of new mental models mandates temporary amnesia or selective intelligence? Sometimes, to move forward with a fresh perspective, to innovate in a big way, you need to temporarily forget much of what you know?

It’s more data than we know what to do with. But someday it won’t be enough.