The Schism in Analytics, A response to Carrabis, Part II

The central scar, the central schism, as I view it, is in the disconnect about what analytics should be and what it actually is.

There are those who look to the past. It is perfectly possible to do very thorough analysis about why what happened in the past, happened. There’s a large amount of valuable competitive advantage to be had that way.

There are those who look to the past only to find evidence to confirm what they remember having thought. These are proof-seekers or justifiers. No further analysis over and above the baseline amount of proof is required. And, if the proof is unsatisfactory – then the data must be inaccurate. Frequently, all that is required is a simple, static report listing a few numbers.

There are those who look to the future. It is perfectly possible to do very thorough analysis about what could happen in the future and optimize against those scenarios. There’s a large amount of valuable competitive advantage to be had that way.

There are those who look to the future only to find evidence to justify what they want to do next. These are validation-seekers. No further analysis over and above the baseline amount of evidence is required. And, if the proof is unsatisfactory – then the data must be faulty.

I think there’s a huge market for justification. In fact, I think this is why so many vendors go where the market is. They just respond to the market, don’t they? And for most people, it’s purely about justification. The tools aren’t set up to explain anything in the past because that’s not where the market is at. The result is predictable. Tens of thousands of reports generated daily, going unread and ignored: all a function of market demand.

To pin the blame solely on vendors is like blaming obesity on fast food companies. They’re only giving the market what it wants, negative externalities be damned.

(Who is anybody to resist market forces?)

I think there’s a smaller market for validation, largely because most analysts don’t really play that game. One of the reasons why web analysts are infrequently invited to the table is because they kill creative ideas and sometimes, are completely disconnected from how managers really make decisions (this is directly to Hillstrom’s point). Because web analysts are generally not very good validators, they’re condemned to the bowels of the company, reporting on the past, and infrequently asked about the future.

That’s the schism: between Scientist-Practitioners and Everybody Else.

And Everybody Else is kicking our ass.

There’s reason for hope though!

I think there’s a growing market for actual learning and competitive advantage – driven by science. This isn’t justification seeking or validation seeking behavior though.

This is driven by upper management – people like Alan Wurtzel of NBC Universal (September 2009) who literally tired of drowning in data. If anything, they’re looking for a consolidation of data by way of scientific methods.

There’s a generation of HBR people too – who actually know what innovation really means. They know you have to get there through science.

Therein lies the gap.

Can anybody say, really honestly, that Web Analytics, as it is practiced by 80% of practitioners, is Scientific? Of course not. And we find fault with everybody else: tools, skillsets, and market demand.

Look, the tools have been there all along: SPSS, GGOBI, R. The courses have been available online for a long time. So, alright, SPSS is expensive and budgets are tight, alright, fine. GGOBI is free and so is R. But if that isn’t enough, there’s more help too:

One of the things that Google is doing to make it easy, part of their grand plan, is to introduce statistical functionality into the baseline tools. Google is creating a world of abstraction – where a web analyst won’t need a degree in quantitative methods to be able to operate the scientific method. Truth be told, an analyst doesn’t need to be a statistician to use Google Website Optimizer. No, an analyst only needs to have the political skill set an alien ambassador on Babylon 5 to get the tags actually put in.

(Hint to those threatened: That entire political world – about getting things done through a large org chart – is where a whole world of value-add can be found.)

I’m under frieNDA about what is going on elsewhere. They’re a huge part of the solution too.

And, I’d like to believe, on sunnier days, that I’m also part of the solution.

Look, you don’t need to understand it to use it.

And, to touch briefly on the whole ‘Cult of the Amateur’ Easy/Hard Towards/Away debate that Carrabis touches upon in his “hard” versus “easy” passage –

Web Analytics didn’t exist when I was growing up. The new economy giveth.

Web Analytics, as it is presently practiced, won’t exist someday. The new economy giveth away.

Sundry reportage – the generation of justification and validation – will probably take at least twenty years to be destroyed because of the a stubbornly long S curve. And you know what: good enough is good enough for huge swaths of the economy. I understand that there are companies out there that turn trees into toilet paper. I salute them and believe that there are analytical products that are perfect for them. I won’t dare call those products “science”. (And when they’re ready for real science someday, I’ll be there too).

I think there’s a solution in the schism: honesty and retitling.

If a web analyst has the drive and desire to actually be a real scientist-practitioner, and their company isn’t going to go there, then they have the duty to get out or STFU.

If a web analyst doesn’t have the drive and desire, then I’d argue that we should retitle that segment of the industry as ‘web reporting’. It’s not worthy of the term ‘analytics’ at this point.

I think that vendors who are clearly in the business of web reporting need to come out and say, “we do web reporting”, and that vendors who do analytics need to come out and say, “we do analytics, scientifically”. That said, we need people who are honest enough and loud enough to call bullshit when a vendor is just that. If it gets nasty, so be it.

I’m convinced that there’s a large market for real science – for real strategic value – for real actual learning. It’s pent up and generally angry with the web analysts fighting each other.

Whether or not we take the same people who are in the industry now will be with me in five years, is the next cause for debate.

(Monday Morning EDIRT: Eric Peterson wrote “Are you ready for the coming revolution“. Some of his sentiment is echo’d in this post. Check out his post and white paper.)