Two statements and four questions about Big Data Analytics: Part 5

Consider the following two, distilled, points of view:

Statement 1:

“Big Data Analytics is going to change the way we do business. Sure, a lot of it will be routine “I’m okay!” status updates from sensors, but making sense of the key parts of it, like “help me, I’m failing”, will be extremely useful. Companies that were previously exempt from competing on analytics will be disrupted by new entrants who will compete better, either by being more effective or being more efficient. Big Data Analytics is already having a disruptive impact in marketing, where it never used to before, and is gaining huge traction in medicine. There is reason to believe that Big Data Analytics will cause better decision making in the organizations that chose to invest both in the physical infrastructure and in the cultural infrastructure that’s required to truly succeed.”

Statement 2:

“All the big industries that rely on data already have Big Data. Airlines, casinos, Internet arbitrage firms, logistics firms and especially finance already have all the data they need. Indeed, all of that data has made them dumber, not smarter. In fact, even in those sectors, there’s little evidence that executives use all of that data to make substantially better decisions, especially when it comes to big strategic decision making. Did anybody see the economy after 2008? This is all just a second wind of hype coming from the Business Intelligence industry, which has so far failed to make anybody smarter. Don’t buy the hype. The companies that have long competed on analytics, since the 1960’s, have nothing new to learn from this next wave of Big Data Analytics. Just ask the line managers what they need, they’ll tell you.”

Four Questions

• Did anything really go wrong with Business Intelligence generally and Web Analytics specifically?

It didn’t meet expectations. The technology failed often. The people failed often. Read part 2 for the expanded version.

• Where does the assumption that better data causes better decisions come from?

It’s hard wired into the scientific method, and, more recently, into Operations Research. Read part 3 for the expanded version.

• Is that assumption credible?

That depends on the people. Good evidence on good managers makes a difference. Good evidence on willfully ignorant managers is a waste. Read part 4 for the expanded version.

• What questions really matter that would cause statement 1 to come true and mitigate the concerns expressed in statement 2?

It’s the attitude of the people using the data.

There are those that view Big Data Analytics as a tool for advancing their personal aspirations. For instance, “I really want to go to big fashion shows for free, so, I need to find evidence that a co-sponsorship with big fashion shows are really going to move our bottom line. Go find me that evidence and don’t come back with an answer to the contrary.”

There are those that view Big Data Analytics as a tool for advancing their personal aspirations. For instance, “I really want to increase gross revenue by 10%, so, I need to find pathway and evidence to support that objective. I have a few questions about how the firm really makes money and from who – go find me that evidence.”

It’s greatest barrier isn’t really the technology. It’s the people.

What’s really different this time

This is the third effort in several to express this point of view. Here it is:

In 2000, to build a data warehouse to mine all the IRC and ICQ chat logs, you would be looking at a $30 million investment.

In 2012, to build a cloud to mine all the IRC and ICQ chat logs, you would be looking at a $300,000 investment.

The cloud, plus open source distributed computing technologies like Hadoop, plus the rise of a generation of data scientists who understand the power of decentralization and know how to use it, is what has changed. It has reduced the costs, increased the imagination, and is making possible a Cambrian explosion in startups.

There are big things happening on the technology side.

If North America is old enough to remember BI and really won’t change its attitude towards data and the way it makes decisions, if that’s what people who are in favor of Statement 2 are really saying, then that’s sad.

There’s a whole bunch of people in China, Brazil, Poland and India are too young to remember.

Thanks for reading this five part series on Big Data Analytics. If you want to leave a comment, challenge an assertion, or raise a point, you can do that right below.

***

I’m Christopher Berry.
I tweet about analytics @cjpberry
I write at christopherberry.ca