eMetrics Toronto Summary, Segmentation and Future

For Syncapse, eMetrics Toronto was a success.

This post is long, and divided into three parts: a summary, a response to Glinski, and then a few thoughts about the next eMetrics summit to come to Toronto.

The first presentation was Theresa Locklear of the National Hockey League. She demonstrated just how far that team had come in just two years. She presented real data – how it’s really presented – across multiple parts of the organization. I applaud that degree of transparency and I applaud her in particular for bringing her entire team. And her analytics team is simply brilliant. They’re well inspired and well informed. Solid.

I’ll start with the Quant/Qual mix panel on Wednesday night. There was no real controversy generated. I chalk this up to the panel being composed of five excellent, well balanced marketing professionals. Jim Novo expressed a strong preference of beginning with an insight derived from a quantitative method, exploring it with a qualitative method, and then communicating it to decision makers by way of a quantitatively derived story. If an ethnographer was around – well – we’d have a much less polite discourse. Issues with social media datasets was suppressed until the evening discussion.

Syncapse and Unica sponsored the elbow rub reception. It was well attended and I saw a lot of smiles. I met Jacques Warren in the flesh for the first time and instantly got along. Jacques is, essentially, the Avinash of the Francophone web analytics world. John Lovett, who I’ve talked to for years on the WAA Research Committee and Forrester calls/interviews, was there in the flesh. Major discussions centered on the Certification Exam, the impending INFORMS collaboration / collision, and how our economic ecosystem is changing.

The entire measurement science team from Syncapse was there too: Paul Cowan, Nadia Kim, Greg Araujo, Zoe Siskos, Hemash Bhatti, and Kevin Richard. A very entertained Meg Burns, my right brain, was pulled from circle of talkers to circle of talkers as we worked the room. Andrea Hadley, the tireless organizer of eMetrics Toronto, hosted a dinner, and, in an aspect that continues to make me really happy to be at Syncapse, our lead developer on Science attended.

The next morning we had the Avinash of the Anglophone world. That is – Avinash Kaushik himself. He was excellent, and will, someday, come with Patrick and I to the top floor of Bar Wellington. Avinash directly referenced that in sentiment analysis, positive/neutral/negative doesn’t work – and that more dimensions were needed. Again we’ll echo that point.

I spent time Thursday talking to Ned Kumar, Stephane Hamel, Jean-Paul Isson, Marko Hurst, Graeme McLaughlin, Mike Sukmanowsky, Simon Austin, Michael Helbling, and Breanna Wigle – among many others. Much of the discussion, impressively, centered on communication of insights, institutional models, education, strategic analytics and a few methodological concerns. In effect, talking about a single tool or a single vendor just didn’t happen in any of those conversations. However, it was great to talk to prospective customers and the talent in our industry.

On Friday, Sionne Roberts asked me to join a panel at the SMX conference next door. He was nice enough to show me his notes in advance, and it went well. My POV on mobile search marketing is pretty simple – we’re reaching an inflection in the adoption S-curve. When it hits, the cost-curve is going to shift quickly as we march up that hype-cycle. Same cycle is same.

Patrick Glinski then presented an excellent Aviva case study. There was real transparency there and beautifully communicated. You could see how the strategy was interwoven with the results. The imagery was beautiful. It was easily one of the best presentations of the entire conference. I’d love to see more of that. I sure wouldn’t want to have followed Glinski. 😉

I presented real data from the WAA and Syncapse. Jacques was kind and called for more presentations like that. Novo sat in the back of the room and did his best not to shout or troll. His mouth was gushing with blood because he bit his tongue so hard.

The final panel was well attended. We had been joking that only five people would be there at the tail end. There were many, many more. And the discussion was open and lively – with the audience actively participating. And it was a power panel – Hamel, Warren, Lovett, Novo. It was a real honor to join them up there. And it was a really great to have dataminers: Emma Warrillow (the regent of weak-ties), Wigle, and June Li contributed heavily to the discussion. I think we expressed frustration that excel just wasn’t cutting it and there are real issues with workflow.

Name dropping and summarization aside – I’ll indulge in some cross-blog talk and some deeper discussion.

Patrick Glinski wrote on the IC blog:

On the “dread” side, I can’t help but feel this conference is going to be the end of my formal association with the digital measurement community.I’ve been a practicing Web Analytics analyst for about 6 years now – an old guy in digital terms. But even in that space, I always felt a bit like an outsider as a user experience strategist (researcher) first and an analytics practitioner second – a divide that is becoming an argument of validation versus prediction.

Meanwhile, many of the dominant conversations are distant and foreign. As Web Analysts, we focus our efforts on what will derivate the largest impact. Usually that means optimizing the user experience around the highest revenue-generating opportunities. Rather than focus our efforts on any one individual, we analyze patterns because it’s a better use of our time. But it seems as though we’ve lost our own way – now focusing in on technologies and methods designed to derive actionable results around strategies that most clients can’t afford to build in the first place.

I want to unpack that.

Glinski (bowtie pictured above) has left in his wake a thread of practitioners that I still work with to this day. I’ve inherited some of them. The digital measurement community is severely bifurcated. There’s now (finally) an impressive cohort of people with 5 years+ of digital measurement and ux strategy experience. In almost every way – you can’t be in the digital measurement industry for that long without picking up a lot of UX experience. I’ve always thought of this cohort as the ‘second wave’ of analysts – the post dot-bust generation that have been heavily influenced by the pre-busters that survived and by the new design thinkers. Glinski is part of that cohort. He’s a big part of that cohort. What he’s expressing is a frustration that nobody else really understands that cohort.

I’m relating to his experience. He’s now a social innovation strategist. I’m now a strategic social marketing scientist. I can relate. However, I don’t think that most people will.

There’s now the post-Davenport wave that’s coming into their own. They have a moderately different slant. I refer to people who have been heavily influenced by the book ‘Competing on Analytics’. They tend to be people from a marketing or finance background (as opposed to a research background), who are beyond the tool and the people. They just want to get to the good stuff. They just want the good insights. They don’t want to read a paragraph of text – unless of course they disagree with the insight – at which point they want material to disprove it. They want the Harrah’s. They want the Fedex. And, they’ve been led to believe, rightly or wrongly, that it’s a fairly efficient and easy operation. Analytics is easy. Davenport says it’s so.

You also have people who are very new and are looking for tools. A big function of an industry that grows in the double digits year on year is that there are new people.

There’s a big difference between analytics for the sake of analytics, analytics for the sake of convenient justification or perception control, and analytics as a strategic source of competitive advantage. That is to say, a difference between analytics as a means to a means, a means to a political end, and a means to a strategic end. The purpose of analytics depends on who you ask. Validation versus prediction? Oh yes. It’s there. And it’s common in DW and BI circles too.

Glinski writes on:

“But it seems as though we’ve lost our own way – now focusing in on technologies and methods designed to derive actionable results around strategies that most clients can’t afford to build in the first place…

For the rich few, yes, these new methods and technologies can (and will) create a competitive advantage. But for the poor greater, this only further creates barriers to entry – introducing alienating languages, foreign skills, and high costs. I’ve heard many people in this industry talk about how wonderful Web analytics is because it’s accessible to marketers and (with a little help), it can be understood all the way to the top. Because of this, there is power in our recommendations.”

Glinski is right. The analytically rich are becoming richer. Most companies do not have a Goal Alignment Strategy in place or an analytics department. They do not have a unified understanding that spells out goals and business objectives and alignment with KPI’s and a cause-effect architecture. Sushi menu style analytics is the order of the day, and many analysts, both on the BI side and the WA side, have taken to retreating behind a wall of no. The strategic – tactical misalignment persists well into 2010. The lack of a solid GAS remains a major barrier between reporting and analytics success.

And the creation of such alignment remains a foreign skill.

It continues to have very real consequences.

As for methodological innovation: it’s necessary. The problems I’ve faced down in social analytics mandated the adoption of new methodologies and new ways of bridging the qual/quant divide. It isn’t innovation for the sake of innovation. It’s innovation for the sake of solving an actual business problem. And – innovation doesn’t always have to be expensive. But it usually is because anything impacting the status quo always consumes a lot of energy. A big part of presenting the initial part of EMV was answering a very real business problem.

Glinski goes on:

While I can’t be certain yet, I get the sense that digital measurement is going the way of the great black box – a fiefdom for a chosen few to understand. It’s also focusing on things most clients simply can’t resource against. While, yes, the decision support may push us towards greater accuracy, we can’t forget the politics behind all of this. I’ve been in enough Marketing Business Intelligence briefings to know (whether right or wrong) who makes the final decisions. In the world of mixed quantitative and qualitative data, whoever tells the best story usually gets the most attention. Black boxes don’t make for good stories.

Three points to tackle: black boxes, story telling and chosen few.

The push for black boxes is driven by profit and a fairly weak patent protection system. It’s also distrust. I’m seeing more of it. Expect more black boxes because of weak patents and weak trust.

We don’t do a good job at telling stories. The medium of powerpoints are designed to tell pretty good stories if we can make them sing. I’ve screamed repeatedly in this space that dashboards don’t tell stories, and yet, they’re our principle output. There’s some innovation in terms of visualization to tell stories too. To Patrick’s direct point, we’re not doing a good job. I’ve asked for help for 5 years and spent much of that time helping myself while detractors boo and hiss. I certainly hope my detractors continue to enjoy clucking themselves and I continue to be thankful to those who help me tell better stories – creatives and the marketers on my team in particular. We don’t have a systematic way of doing it, and what is common sense to great storytellers is a nightmare of ill-defined rules of thumb for those who are not.

If those who tell great stories are frustrated with those of us who don’t – it’s a two way street.

Finally – the chosen few and eMetrics.

eMetrics was a success for Syncapse because we worked really hard at it. I’m really quite introverted and I really don’t like to meet unfamiliar people. So it’s always a stretch for me to go out and work a room. To just walk up to a group of people standing in a circle, butt in, and immediately beginning to talk and shake hands is a stretch. I think it’s a stretch for most others there too. It’s a stretch.

It was also a stretch for much of my team. Except Kevin. I continue to be amazed by Kevin Richard’s ability to work a room. Many people knew Syncapse before I got to them because Kevin already knew them and gave them the story.

I got some of the best value through these interactions with people. I get the best value when I generally stretch myself. Some of the presentations were very good. Some of the presentations were not relevant to my interests. I got to spend 45 minutes with one of the best minds in database analytics on the esoteric problem of behavioral WOM. (Yeah. For real!). I got to spend a solid hour with Wigle as we shared tips on social segmentation. I also got to spend an evening with two of the most knowledgeable minds in analytics. I hope that I’ve begun really solid professional associations and friendships. It was an incredible experience that was facilitated by the conference.

I also enjoyed talking to many of the newcommers. I loved hearing about their specific challenges, their questions, and helping them out. I also look forward in telling specific people more about what we got planned at Syncapse.

80% of the most valuable experiences happened outside the main conference.

Andrea Hadley and I have had many conversations about this effect while sitting outside in the lobby of another conference. (See??!?!?!?)

You have a group of people who don’t want to talk about tools or black boxes. They want to focus on communication, institutional integration with UX and Qual, and strategy. These are your 5 year+ analysts. There aren’t many of them. They have real problems and concerns, and there’s real value in getting together with them, face to face, and talking about these types of managerial and strategic concerns.

So here’s one of the paradoxes of what Glinski is alluding to:

You have a group of people who are grappling with issues like “how should UX and Evidence Based Marketers work together?”, and another group that grapples with issues like “why doesn’t comscore match up with Omniture and how do I explain that?”. One subset of question isn’t better than the other. They’re just different. Because we’re at different points in the lifecycle.

There are already fiefdoms because there are segments.

So how does eMetrics, as a web optimization summit, retain its pull?

I think it would be an incredible experience for 20 experts to gather into a room and talk.

The downside is that it would be a ghetto.

I would hate for the perception of the newcommers to be that we’re all crusty, exclusive, and inaccessible. After all, I was a newcommer once too. And I benefited incredibly from having access to those experts.

From a marketer of Syncapse’s point of view – it’s twofold. We want good reach. We want to scout out talent and prospective customers. We also value getting perspectives. I also value having the team there so they can learn what others are doing and saying.

I don’t know what to suggest to Andrea aside from some sort of hybrid approach to keep the Glinski’s, Hamel’s, and Dykeman’s coming back and getting value. Both segments are worthy, equal, and should be facilitated and supported.

Would an advanced track, open to everybody, be satisfactory – even if many who came on in were ultimately alienated by the content?

Glinski’s frustration doesn’t appear to be just with his eMetrics experience. It’s with digital measurement writ large. I don’t think there are enough of us within the WAA, within INFORMS, or within the broader SiG community to achieve any sort of reach or impact. There might be all of 400 people on Earth who stand at the analytics/ux/social/strategy/business juncture: and 200 of them are in India. Not talking to each other. 😉

I don’t begrudge the 6 billion others who aren’t there with us, and who don’t get it. And I don’t judge the 50,000 who will start to get it really soon. It’s a young industry.

That said, it’s a young industry from which many move on.

The image macro below is about the best representation I can find to express my thoughts.

Thank you to Andrea Hadley, June Li, and Alex Langshur for an excellent eMetrics Toronto. If any of you reading this would like to contribute thoughts – feel free to post a common here or cross-blog-link it.