Category Archives: Analytics

Analytics and Inside Pool

You may or may not have been hearing about a debate going on in web analytics.

To most, it might seem like a lot of inside pool. And I suppose most of these things are.

I want to talk a little bit about some of that inside pool.

Over the course of my WAA Research Committee work last week, I stumbled upon a paper entitled “Assumptions, Explanations, and Prediction in Marketing Science: “It’s the Findings, Stupid, Not the Assumptions” by Eric W. K. Tsang.

In it, he replies to a debate that’s been going on for a long time, but what natural scientists had settled a hundred years ago. Richard Staelin back in 1998 said that there’d always be debates about whether analytical models needed to have realistic assumptions or not. Shugan came out in 2007 and argued that it wasn’t about the assumptions. I can remember reading that paper back then. It had an effect on me. Let’s fast forward to 2009.

I don’t quite know how it happened, but I ended up sitting at a table with the megastars of Marketing Science research at an informs conference. Dr. Lehmann was there – as was Dr. James Lattin. From what I gather – they’re pretty distinguished researchers. I didn’t know it at the time, and I doubt that it would have changed my behavior much. Maybe only outliers would ever dare sit with that group. That’s how I met Alex.

Two outliers at a table of high insiders.

Alex is an economist out France. I won’t go so far as to call him a French Economist, but, regardless, there it is. :) We got into a discussion about how irritated I was with stupid assumptions.  I understood that without invalid assumptions that the math wouldn’t work: but maybe there’s no value in the math that doesn’t work. That unless I could use the model to understand the world, or at very maximum: predict the future in some way – that wasn’t of any use to a practitioner or to a scientist. Alex explained to me that the Math unto itself could help science chip away at the edges of complexity – and if something adds understanding, then it is of value: but maybe not to a practitioner.

I still accept where Alex comes from. I think there’s a role for trying to understand complexity by way of deliberate simplification. How those assumptions get selected still bothered me, and I continued to want to shout down anybody who had selected, in my judgement, a stupid assumption for such little gained value.

Back to Tsang, in his paper, where he takes Shugan on. Apparently there’s an entire school of thought that dismisses my belief that science should have at least a goal in making accurate predictions about the future. Tsang carefully deconstructs Shugan’s 2007 arguement, and in the end concludes that “although Shugan (2007) rightly stresses that it is inappropriate to dismiss a model or a theory based only the realism of its assumptions, realism does matter, and it matters a great deal for model building and theory development.”

And I happen to agree with Tsang. He’s helped me immensely in being able to reconcile some of that inside pool.

A lot of the inside pool going on right now in Web Analytics is very similar to Tsang-Shugan and Christopher-Alex. There are huge disconnects between what many web analytics practitioners want analytics to be, what some of the industry titans want it to be, what customers of web analytics outputs want it to be, and even within the broader analytics community (data miners, revenue managers, and market researchers are in the same neighborhood) want it to be.

All this – within an industry that couldn’t possibly employ more than 50,000 people in total.

Inside pool is important because it’s about values and refining the definitions that are in use by a community.

Talent Supply in Web Analytics

Eric Peterson wrote yesterday about the coming revolution in web analytics.

It’s worth a read and it sparked off a lengthy twitter exchange.

I think we have a huge talent supply problem in the web analytics industry.

Web analysts are very specialized in terms of their understanding of the Internet, websites, tracking technologies and reporting methodologies. And necessarily so.

There really aren’t that many of them.

Sure, there are plenty of people who have Google Analytics on their blog. And I’m glad that they do. It’s great to have so many people interested in Web Analytics. But there’s a gap between the interpretation and the turning of that data into actionable insight. In fact, many of the things that look easy really aren’t: such as interpreting ‘time spent on site’.

It’s quite another thing to talk about the leap into statistical web analytics. It’s a different world.

Currently, 90% of web analysts are not asked heavy statistical questions. The industry just isn’t there yet.

To be sure: the demand is growing, and will continue to grow.

We already have a talent shortage in web analytics. Baseline web analytics.

The coming revolution, as Peterson puts it, will put unprecedented demand on the existing talent.

I welcome it. I don’t know how many people will really be able to respond to the challenge (immediately).

Markets have time lags.

Adobe Buys Omniture

My initial feedback:

YO #OMNITURE I NO U JUST MERGED AND ALL N IMMA LET U FINISH BUT SPSS’S MERGER WAS THE BEST ONE THIS YEAR – Kanye West

So what does Adobe really get for its 1.8 billion?

A company in the top 5 of web analytics tools providers for one. A great client base for sure.

But there’s a black lining to that white cloud.  Even with a client list that most analytics companies dream about – Omniture is hemorrhaging money. Even with strong revenue growth, it hasn’t been able to make marginal profit on that growth. It should just confirm what all honest practitioners admit at a Web Analytics Wednesday: the software is frakencode that’s a time vampire to install and keep running with any degree of adequate accuracy.

Dissatisfaction is running high: and I don’t think I’m the causal variable of it. (I’m OCP at that!).

If Adobe really is competing in the same sandbox as IBM, Microsoft and Google – and this is somehow a measurability / analytics play: I think Adobe could have done better.

I just don’t see Omniture being able to innovate around Google at this point. I think new entrants just have too many advantages going for them.

In sum, I hope for sake of myself and my friends that Omniture is run better under Adobe. Adobe has a lot of heavy lifting to do to turn it around.

Of Personas and Market Segments: A reply to Hamel

David Hamel wrote:

The issue behind it all is that the web isn’t static, it is constently changing.  Ever heard of AOL?  Of course you have.  Know anyone still using it? Probably not.  What about MySpace?  Also there is the inevitable the march of time.  Your persona for Bob has his age at 52.   In five years time, will Bob still be useful?  Probably the difference between 52 and 57 isn’t that large.  But what if your target demographic is 22?  There is a much larger difference between the interestes of a 22 year old and a 27 year old…In conclusion, use personas but don’t let them get stagnate, your personas represent people and people change.

Hamel rightly points out that personas come into being and are sometimes printed out on huge boards and laid out in the office. And then they’re taken down after the redesign and we forget about them. They don’t evolve. They don’t breathe.

I’d argue that the typical market segment, “male, 56-65, rural” is incredibly bland and not adequate at all. I think somewhere along the line, perhaps over the decades, we forgot why market segments are supposed to be powerful: like people talk and they are self-referential. This was demonstrated with the original Word of Mouth marketing studies focused on seed distribution. Word of mouth is critical. No company can possibly afford to pay for every conversion.

The segment should describe, instead, where rural older males are talking and how do you get them to refer you along. A person is so much more than just an age, gender, income bracket, number of kids and general DMA. Quantitative marketing science is capable of real contributions and advancement in that field.

If personas impart empathy in design, then segments should impart empathy in marketing.

And directly to Hamel’s point: they should be periodically revisited.

That might not sit well with the common “rip’em down and reinvent them from scratch” mentality that dominates the world – but I’m very comfortable with the notion of optimizing segments and personas over time as a program of ‘learning’.

I’d like to see more of that.

Why yes, it is summer event today

It’s summer event today at CM Toronto.

It’s normally a very good day. As with anything, the 80/20 rule applies to it.

We have a town hall, where a member of the executive comes and presents. It’s actually really good. When I started out as an analyst, the only time I was ever really fully brought up to speed (I felt) was during these presentations. Things have long since changed for Marketing Science folk.

Then there’s some component of field trip or activity fun. Those are always fun.

And then there’s an evening of more fun – typically featuring Captain Morgan.

What I value most is getting to really talk to people from the other offices. So often, they’re voices on the phone, and very often, you’re not exactly meeting under nice circumstances (they’re calling for a reason, and typically there’s distress involved). In sum, it’s nice to see people in person and listen to what pains them. And then fun to have fun.

Social Analytics and the Triple Bottom Line

I hope to embark on some Internet Serious Business work that links community with government with some industry. There’s a large social analytics piece in all of this that I’m looking forward to.

The triple bottom line can be summed up as “profit, people, planet”. Basically, accounting for social and environmental impacts as well as the profit motive.

There’s a story about a young graduate student and an old econ prof walking down the street together. The young graduate student, clearly cash starved, spots a dollar on the street and says, “look, a dollar” and goes to reach for it. The prof holds him back, and replies “nonsense, if there really was a dollar there, somebody would have already picked it up by now.”

The point being: sometimes we assume, wrongly, that if an action was profitable and optimal, we would have already figured it out and the market would have already cleared. Again, as a marketing scientist that understands the problems of optimization – that assumption is totally false.

It’s probably very likely that you can’t maximize profit, social and environmental benefits simultaneously with existing technology. What’s required is research and development geared to bring those incentives into alignment. It’s perfectly possible to calculate social and environmental harm and introduce market mechanisms (thereby forming the basis of a tax credit system to guide the invisible hand towards more good). There’s a careful policy way out of this. And, as we’ve seen time and again in Canada (in particular): policy is powerful.

I have much of the profit stuff figured out. It’s the two other components, planet and people, that deserves some mental attention.

Michael Jackson and Honduras

JacksonAndHonduras

Whether it’s Elitist or not to point out the disparity, the fact remains that these kinds of media flares have become standard.

Lynn Rosenvall taught me that the world is divided up into large media zones. There’s no conspiracy about them – regions of the world just have different points of view and are dominated by different stories on different days.

Our little neck of the woods up here is dominated by the Michael Jackson story. I don’t know if we’re all the more richer or poorer for it. Would the news be greeted by yawns or passion? After all, there’s no oil buried underneath Honduras…