The Complex Quest For Simplicity in Social Media Measurement

The Quest for Simplicity in Social Media Measurement (#smm) is one that will dominate the year.

Trying to produce something simple out of something complex is…complex.

There are seven axioms that are guiding a lot of my thought in dealing with that complexity:

1. The purpose of analytics is to derive competitive advantage for the organization / firm / entity.

It follows that the purpose of Social Media Measurement is to drive competitive advantage. If the end result isn’t competitive advantage – then it has no value. That unto itself is a value statement.

Simplicity drives competitive advantage because simple is more actionable than complex. I’m often asked questions that have very complex comprehensive answers. I have to sort out that complexity based on relevancy and action-ability. Reality is always so much more complex. And yet, people can’t act on the complexity.

They act on simplicity. And if action is the vital link between the insight/competitive advantage gap – then this mandates a simplified approach.

2. Data alone does not yield competitive advantage.

A major brand might be mentioned 2.5 million times a week on Twitter alone. Having all of that data in a database is of no value if it doesn’t result in competitive advantage.

I’ll go ahead and make a statement: very few people on Earth have the capacity to read and understand what 2.5 million tweets mean on a monthly basis.

3. A sequence of progressive hypothesis testing is the most efficient and effective method to derive competitive advantage from data.

I still hold that the scientific method is the best one we have for learning right now. Someday, somebody will figure out a better algorithm. Until then, the scientific method has this wonderful blend of flexibility, creativity, and evidence.

Progressive hypothesis testing means acting deliberately with marketing messages. The goal might be known – like ‘drive sales’, but the opportunity to message a community becomes all the more useful when, over a sequence of messages, a specific hypothesis is testing. One really basic test might be: “will the community respond more to content about special features instead of content about where our spokesperson is going to be”.

Acting deliberately isn’t always possible, especially in a reactive world, but there’s opportunity to derive learning or insight that can drive the next wave. In social media, the tempo is that much higher. This isn’t 2-year website redesign land.

4. Predicting the future requires an understanding of cause and effect.

At the core of prediction is previous cause and effect. If I touch a hot pan, it will cause my hand to burn. Therefore, I can predict, by touching a hot pan, my hand will burn. Very predictive.

Not everything, especially in marketing, is so clean. At some of the more basic roots – If I spend 500,000 dollars on commercials and run them constantly, I will get 11 GRP. If I get 11 GRP, I’ll move 25,000 toasters.

Statisticians, or Social Science Statisticians, are so incredibly jaded by such simple linear models. Sure, you might get 11 GRP’s, but not all GRP’s are made the same. Moreover, what type of commercial are you going to run? Will it resonate with those who are already looking for a toaster? Will it cause people to suddenly desire a toaster who do not have one? Will it cause people who want to judge others to go out and buy the toaster so they can have a plank to judge? Will it cause people who already have a perfectly good toaster to want to buy, and remember, that toaster – five years down the line to buy that brand?

So frequently, especially when a cause-and-effect model doesn’t jive in our own minds, will we go out and try to discredit other models by introducing other factors that we ourselves deem salient to the situation.

In the end, it comes down to R Square. The percentage of the variation our model predicts the outcome of a variable we care about. A big reason why I rattle on about the importance of goals and KPI’s is because we can anticipate a world where everybody will care about the R Square.

This is especially true in Social Media Measurement. Many people speak of things ‘going viral’. Yet, how many people have truly explored the causes of going viral? There are multiple causes of why something goes viral.

Predicting anything comes from cause and effect.

5. Correlation is not always Causality.

Even a high R Square doesn’t guarantee truth. There might be a great correlation between affinity for John Cena and a love of peanut butter – but I’d be hard pressed to derive a clean causal link between the two. (Perhaps John Cena’s fan base is concentrated in regions where peanut butter is given to young children early?). Unlikely.

Correlation is useful, but without overarching respect for your own theory and your own mental models – it’s dangerous.

This is especially true in Social Media Measurement – where correlations abound – but causality can be fleeting.

6. Accuracy over Precision.

Would you take a thermometer that is right 95% of the time and you were fairly sure that it was always off by 5 degrees, or would you take a thermometer that is right 50% of the time and you were fairly sure that it was always off by just 0.01 degree?

In Social Media Measurement you can have it both ways!

7. It is possible for there to be two optimal, equally true, answers to a problem. (And Sometimes More!) (X^2 = 4, x=-2, 2).

If there are two equally true answers to a problem, surely there could be millions of wrong ones. I’m certain that will make certain people happy to hear.

In Social Media Measurement, it is perfectly possible for two solutions to be both equally right.

A specific instance would be the sentence:

“The boy crossed the busy road carefully.”

I’ll ask you: What was that sentence about? I can see a situation where one of you says, “The boy” and another person says “The road”.

Well, in my view – they’re equally true.

There are multiple right answers. There are multiple wrong ones too.

Simplexity.

The quest for simplicity is complex.

Simplification involves obliteration. It’s possible to take a column of 300,000,000,000 numbers, a massive amount of information, and summarize them into a single figure. In fact, there several numbers that can describe the central tendency of all that information: mean, median, mode. We have a number that describes dispersion of that data: standard deviation. We have numbers that describe the peakyness: kurtosis.

What should get obliterated in the quest for simplicity?

Going back to Axiom 1, variables that do not matter to competitive advantage should be obliterated. Going to Axiom 4, you need to identify the variables that cause a desired effect, in particular, looking for reinforcing effects, all the while knowing that Axiom 5 applies (your theory of how the world works could be wrong even if mathematically it works) and Axiom 7 – it’s perfectly possible for two models to be equally right.

It all comes down to an acknowledgment that Axiom 2 is right: data alone isn’t going to yield competitive advantage, and Axiom 3 is the best way to turn that data into insights that drive competitive advantage – a sequence of progressive hypothesis testing.

I don’t believe we’ve even begun at the beginning yet: what is salient in social media measurement?

We’ll need to get all of those on the table before we can talk about causality, reinforcing effects, and come out to a resolution. I’m pessimistic that there will be a single resolution that will suit everybody: but there is probably a solution that will satisfy 90% of the situations.

What say you?

You can edit this ad by going editing the index.php file or opening /images/exampleAd.gif

Community Seeking and Online Gaming in the Early 2000’s

Communities played an important part in the online gaming experience during the early 2000’s, and I think there are lessons in there for today.

Time for a story. It’ll be fun and egregiously self-deprecating.

My first Real Time Simulation (RTS) game was Age of Empires I, back in 1998 or so. And I loved playing it online. Problem was – the online experience really sucked because most of the players were jerks. The experience sucked and the game lagged like hell.

By 2000 I had joined my first gaming community. They were referred to as gaming clans, and you could identify its members by having a telltale tag at the front of a name. MNPE_username, JCV_username…and so on. There was also this entire notion of being anonymous. It was highly prized. Back to this in a bit.

Gaming clans were never really intended to be part of the multiplayer gaming experience. At least, I don’t think Ensemble Studios ever really foresaw them. People, in a way, spontaneously formed them. The most dedicated ones would then register a domain name and host Internet forums. I don’t think we even knew the term social media at the time.

I regret not keeping a long enough list or better records – but there were many, many, many clans. The vast majority had very short half-lives. They would be founded, exist for 3 weeks, and then die a cold death. Some would persist. They’d grow and thrive. And then later, they’d fragment and explode. They died a hot death.

A very few would be self-sustaining. In effect, they’d be founded, they’d get 7 members, and from there they would be successful…continuously refreshing the membership over time and maintaining just the right size to insure familiarity and community.

Different clans had different utilities for their members. Some of them were Elite Clans. In effect, a small group of 7 to 15 people would get together and trade insider skills and knowledge. You had to have a certain rank to even apply to be part of them.

Other clans were friends only. You were invited in.

Some clans were open. Anybody could join. So long as you were fun to play with, you were pretty much guaranteed to be in.

Clans served a few purposes. There was a reason why so many people joined them back then.

For one – they provided a source of gaming quality control. The online experience sucked back in then – with people flamming and dropping, cheating and yelling. More often than not it was unpleasant. If you knew a group of people who behaved reliably well, then the experience of the game would be all the more better.

Secondly – they provided a source of quick games. If a group of people routinely logged in at the same time, chances are they’d play together and have a good game. The tag provided the boundary for friendship. Or, it guaranteed a certain level of competitive quality in team play – which was important for people’s ladders rankings and tournaments. Gaming was (and probably still is) serious business.

Thirdly – many evolved to form actual communities over time, and evolved a set of social norms and commonality. For a very long period of time, pre-Skype-Facebook-Twitter-YouTube-MySpace, they formed a type of safe-place. You could remain anonymous and still be part of a community. And it was as though somebody in Korea was just next door.

After a game had become stale – the community frequently remained.

You might ask – “Why didn’t people play with their friends?” (Like they do now – like on Xbox Live…?)

Not everybody had broadband connections at the time. It was still rare in countries other than Canada, South Korea, and Netherlands. Secondly, not many people played RTS games, either. For sure, some people knew other people ‘in real life’, but this was the exception, not the rule. Finally, Xbox live didn’t exist. Console games were consoles. Online games were social in a different way. The distinction, I suppose, in the transition from kids in front of Nintendo to young adults drinking beer in the basement playing a console game – and the notion of virtual community anytime/anywhere gaming: was a distinctive split. You could only get an online social experience through the PC. This unto itself caused a different dynamic.

You could have an authentic community experience through a clan while being anonymous. Many people back then wouldn’t imagine doing anything online unless it was through a pseudonym. (This was pre-Facebook).

Many of these communities gradually faded. For one, Ensemble Studios began incorporating clans directly into the gaming experience. While this was nice to a certain extent, they removed a few sources of quality control, and an important concept of continuity of the community in case the leader leaves. Ensemble Studios, much to their credit, actually did what we now refer to as ‘community outreach’. And they were actually quite proactive about it at the time.

Clans continue to thrive in the First Person Shooter (FPS) genre, where the quality of the game is heavily dependent on the quality of those you play with, and where dedicated servers are highly desirable. Sony, through it’s massive FPS game ‘MAG’, might still accidentally cause a renaissance of the early-era clan.

There are quite a few takeaways for social media.

People sort other people. Not always. But often. In the instance of clans, people sorted themselves into groups based on the experience they wanted but couldn’t get otherwise. I suppose if the parts of what remain of Ensemble wanted to innovate, they could introduce an jerk-sort into the experience. Those who have propensity to ruin an experience for others could be grouped together and be miserable with one another in random match-ups.

Communities are much more than joint-utility seeking entities. To see them only in that light would be too simplistic of a model. But communities do, at least, seek joint-utility. Even 4Chan has its bouts of concern about the cancer that is killing /b/.

Communities might also self-organize so as to compensate for a deficiency in a corporate offering. Perhaps some companies can understand which deficiencies exist.

Finally, communities can outlast their original purpose. For how long they can survive I’m unclear about. But they do.

Community Seeking and Online Gaming in the Early 2000’s were really interrelated. I’m curious to watch how this next wave of social gaming will change the landscape that much more.

Social Media Return on Investment – A reply to Jim Novo

Jim Novo wrote in response to the last post:

This is an interesting line of thought Christopher, perhaps I can help with a bit of a framework. And you’re right, product is the root of Marketing decision making. I hope my attmept at a chart below makes it through the CMS without breaking…

Brand for any product is a continuum between Product-centric and Image-centric, example:

……….Product Centric………..Image Centric
Beer…….Sam Adams………………Budweiser

Image-Centric Brands tend to have commodity status, which begs the need to differentiate by creating some kind of unique Image. Product-Centric Brands differentiate on hard Features and Benefits.

If you think about the Marketing for Sam Adams, it’s all about ingredients and customization. If you think about the Marketing for Budweiser, it’s all about wanting to be like or associating yourself with the people or images in the spot – “Yea, that’s me!”.

Now, if you think about Social success stories, you find that they really gravitate towards Product stories, and not Image stories. Image stories are too easy to destroy in the social fabric; product stories bubble up *from* the social fabric.

So the success of social will largely be determined by where your Brand is on the continuum between Product-centric and Image-centric.

And here we arrive at a bit of irony.

Many of the most successful Social “Campaigns” happen when the company does absolutely nothing overt in the social space – see Apple, and many other Product-centric Brands.

And some of the lamest and most clueless Social campaigns have been from commodity Image-centric products that tried to do something overt in the social space – see various packaged goods.

Meaning, you don’t really have to *do anything* to get ROI from social if you have a successful Product-Centric Brand – the ROI is infinite because there is no spend. And the ROI for an Image-Centric Brand is likely infinitely negative – any spend will never generate enough incremental sales to pay for the spend.

As far as Marketing discplines go:

……….Product Centric………..Image Centric
……….Direct Marketing……….Mass Marketing

Direct has always been a Product-Centric approach; it has to be or the Math doesn’t work; it’s Feature / Benefit driven.

That’s not to say companies employing Direct do not have “Brands”, they most certainly do. But the Brand is very tightly tied to product, not so much with “me too” Imagery.

Make sense? Help in your quest?

This both makes sense and helps.

The nature of the product is certainly a dimension in this problem, and your model rings true.

So much of what we really consume is in our heads. Certainly, there’s often a physical aspect of most products. For certain products, it’s about how we feel about them. And to a certain extent, how we feel about a product is what we are and what our friends are like and how much our friends really influence ourselves. I can point to product diffusion studies in marketing science around artist popularity and DVD sales to back that up. The nature of the social graph is as important as the social impressionability of that social graph.

If you’ve listened to any commercial music since the advent of the Bit Torrent, you know that there isn’t really a qualitative explanation as to the variation in popularity of a given DVD. What we consume is sometimes, though not always, in our heads.

For a commoditized product that really isn’t differentiated based on the general attributes or quality of it – I can see where a company will want to compete in your head and for your friends. Of course, not everybody is as susceptible to social effects as others. Beer choice amongst friends versus beer choice amongst professional associates might be one.

I think we have to consider where a product is in terms of adoption is important too. Just because something on its own might have better features than another – it doesn’t guarantee successful diffusion.

I’ll need to think longer and harder whether or not the ROI curves asymptotic.

But, as of now, we have ourselves at least three dimensions.

Social Media Return On Investment

I’ve been fairly obsessed as of late with quantifying Social Media Return on Investment, or sROI for short.

At the root of the issue is a clash of belief systems.

Marketing thought is dominated by two rather large models of thinking. You have the Direct Paradigm and you have the Brand Paradigm. By Paradigm, I mean simply a way of looking at the world. Let me take one step back, and then one step forward.

People, in general, can only hold so many variables in their heads at the same time. So, we abstract. We’re supposed to derive some forms of causality that are important, throw that into some overarching architecture, and then use that framework to make decisions in a quick manner. When two people first approach a problem, and come at it from different paradigms, sometimes it can get nasty because there’s some questioning root assumptions.

The language you find in the Direct Paradigm is that the last action somebody took towards a sale is the most important. They point out, quite rightly, that repeated human behavior matters the most. A human in motion will tend to remain in motion. I wouldn’t make the accusation that all Direct people can only hold the short term in their minds at any given time. In fact, some of the best contributions and strongest predictors of campaign success are based on a very sophisticated understanding of time.

The language you find in the Brand Paradigm is that how somebody feels is the most important. They point out, quite rightly, that if somebody hates a brand, they won’t buy that brand. There’s a set of key performance indicators, invented in the early 1930’s to handle radio measurement, that attempt to quantify that. Likeability and message recall are the two big ones.

The Direct Paradigm tends to value deductive reasoning, and this is form and function as a result of having all the data. It’s inherently about data mining.

The Brand Paradigm though relies on inductive reasoning, because they’re forced to use sample statistics to perceive the world.

If you were to review what adherents of the Direct Paradigm are saying about sROI, they’re wagging their fingers and their tongues. They point out that Dell only made a fraction of their direct sales from Twitter. They point to the lack of conversion from the sales is proof positive that social media ROI is too low to justify intense spend.

If you were to review what adherents of the Brand Paradigm are saying about sROI, they’re clapping. They point out to successes like Best Buy and argue, quite rightly, that it impacts how people feel about a brand. They point to a fragmenting attention economy as being the main reason for intensifying social media spend.

So, which belief system – which paradigm – is right?

I’d argue that they’re both right.

I’ve read about an era in marketing when both the direct response and word of mouth dynamics were scientifically optimized. In my minds eye it was quite an exciting time.

I think we have to understand, fundamentally, that the Direct People have it right. They’re quite right that if you don’t have the right message to the right customer at the right time – you won’t get a sale. It’s about having the opportunity to convert being there at the right time.

I think the Brand people also have it right. How people feel about a brand is important. I’ll go so far as to say that how the friends of certain people feel about a brand, and how they consume that brand, is also a factor. Sure, you might put the right message to the right customer at the right time – but if I hate that company because they pulled their sponsorship for the Reading Rainbow…I’m not buying.

I’m optimistic that within social media measurement, through this quest for social media return on investment – that we’re going to find a satisfactory model that will be easy enough for 95% of the population to understand. That it’s going to incorporate just enough from the Direct Paradigm and just enough from the Branding Paradigm to work. In fact, what I’m seeing is a real opportunity for a Third Paradigm.

What if, under this Paradigm, we selected the most predictive elements, instead of what would be the easiest elements? What if, built into the model, we had a larger number of variables to chose from in constructing our general causal model? What if we acknowledged the dual nature social media?

I’ll add one factor that I think is especially salient: the nature of the product or service itself. There are certain products that are completely social in nature. It’s for this reason that I believe sROI is actually going to vary quite a bit depending on the sector and the competitive set.

The ultimate calculation will depend, quite heavily, on how much is borrowed from both Branding and Direct.

Little Things that Make Big Impacts

The cleanest way I could explain the Butterfly Effect was to say:

“Let’s say my shoe is loose. So I decide to bend down and tie it really tighter, inadvertently creating a knot. Let’s say the next morning, I have a hard time getting my shoe on – for let’s say, four minutes. Then let’s say that I miss my bus by just one minute. And the bus has a frequency of thirty minutes. Well then – one seemingly unrelated decision, made 16 hours before and taking all of 2 minutes to execute, has a 30 minute tardiness impact 16 hours later. That’s pretty much like the Butterfly Effect. Writ Small. And Mundane. Without bad acting.”

The Star Trek: TNG way of saying it would be “There’s a cascade failure in the warp core”. But enough of the Laforging.

Cause and Effect dynamics are devilish. After all, my lateness could have been chalked up to not being ten minutes early as I normally am. Or it could be chalked up to the bus being on time, which is unusual. I like to think of the world as a whole bunch of cones converging on a single point. Taken from this point of view, there are as many explanations for something happening as there are people. We all have our perception and are all entitled to own opinions. Though, we’re not entitled to our own facts. (wink).

It’s just a matter of which model has the greatest predictive strength. Normally I’d head down the rabbit hole into a bias about multiple regression…but no. This isn’t going to be a statistical rant. No. I have something far funner to read. (I hope).

And of what implications for the social systems we create?

Twitter is an excellent laboratory to study for that.

And that’s where we’re going to get into a lot of trouble with each other, as social media scientists.

‘How one seemingly innocuous tweet could cause a cascade failure in the warp core?’ will be one of those great analyses someday. And it will be contested. Loudly. By very educated and sinecure analysts.

It won’t necessarily because they won’t accept that little things can make such big impacts. I’ll be referring them on back to this post at that point. And surely, every very educated analyst should be familiar, and indeed, should have experienced such dynamics in their own lives so as to be able to relate. The Butterfly is in the Sky.

Rather, the debate might be how much causality to attribute to the originating tweet, and how much causality to attribute to the reinforcing effects. And indeed, this sub-branch of analytics, of reinforcement-attribution theory, is still very young in marketing science literature. (I salute those of you who have made contributions. It’s just that I wish we had a unified language to describe it.). Someday I’d like to be able to say: “Take a look. It’s in a book.”

How do we understand cause, intervening variables, and effect – and how much we decide to respect where each other is coming from, is by and large going to paint future debates. I’m optimistic that there will exist one school of social media measurement practitioners that will rely on evidence to make assessments. And I’d like to be in that school. I’m certain that we can go twice as high.

There was a little theme running throughout the post.

That’s how little things can make big impacts. And how something little will make something big.

Roger Martin, Michael Porter, and Re-imagining the Production Possibility Frontier

Michael Porter, in “On Competition”, appears to emphasize the importance of trade-offs.

Roger Martin, in “The Opposable Mind”, appears to de-emphasize the importance trade-offs.

Porter defines strategy is the process of making choices about activities that results in sustainable competitive advantage. Both books make reference to activity diagrams – so there’s unity and acknowledgement that choice matters. At the core: Porter explains the ‘why’ of strategic decision making, and Roger Martin describes the ‘how’ of strategic decision making. The ultimate way of showing trade-offs, in my view, is though the Production Possibility Frontier.

What is very elegant about the production possibility frontier (PPF) is that it’s two dimensional and tells a very clear story. There are trade offs between quality and quantity. Luxury and Economy. Bread and Guns. Reporting and Analysis. Accuracy and Precision. Very easy to understand.

Simplicity wins when you’re communicating. And yet, we’re rarely handed a manual when it comes to seeking elegant solutions.

Instead of just plotting two trade-offs and calling it a day, and instead of heading into n-space and Riemann spheres and unicorns and stickers – there is an elegant way of increasing salience while retaining understability.

If you use GGOBI or PASW, you can select ’scatterplot matrix’ and you’ll get a n x n chart of all the activity-relationships. So, if you have 10 activities with tradeoffs and functions interrelating them, you’ll get a 100 charts, of which, 45 will be of use. (45 are just the mirror image, and 10 are straight line functions).

Through the magic of the post-it note, you can expand the number of activities under consideration quite a bit, and then start sorting them out on a very large wall.

What you might end up finding are a number of strategies that are very close any number of PPF curves (at least, the ones that matter). From there you can derive activity maps that correspond to any number of selected PPF curves. The elegant solution would be plot it against a single PPF curve, but I’m pessimistic (at this point) that such a solution would be common enough.

Instead of a single PPF, what if  it’s more of a Library of PPF’s?

eScience

I’ve just learned of eScience as a result of a book entitled “The Fourth Paradigm”.

While I don’t have that much to say about the essence of the Fourth Paradigm yet, I have to admit that I feel immediately at home with this group within eScience. One of the best quotes in the book is:

“Need driven versus curiosity driven. Basic science is question driven; in contrast, the new applications science is guided more by societal needs than scientific curiosity. Rather than seeking answers to questions, it focuses on creating the ability to seek courses of action and determine their consequences.”

Substitute ’societal needs’ with ‘business needs’, and I have myself a nice bridge between eScience and commercial eScience. I suppose that’s been one of the fundamental misunderstandings about the Scientist-Practitioner: that they were only poking about out of curiosity. Science for the sake of science.

What if we were transparent about the intent to use science for purely commercial gain? Sounds Edisonian I suppose?

Much of the literature seems to be about very huge computing problems, like analyzing the data from the LHC. I’m not necessarily as concerned with problems of that order of magnitude. In fact, most business problems are fairly modest by comparison. What will, however, hold back commercial eScience, are the same forces that will hold back eScience. That is to say, the lack of unification among the fundamental tools.

At any rate – this field looks attractive.

WAA Research

Preliminary results from a membership survey suggest a strong level of satisfaction with the work coming out of the Web Analytics Associations’ Research Committee. And that’s heartening, since the volunteers do a lot of work.

I’ve participated in some of that research over the years, and it’s always pretty enlightening.

It’s good news.

Complexity

I’ve spent a lot of time this week managing complexity.

And it’s gone well.

I think looking for simple and remembering the end goal are two key ingredients. Backcasting happens a lot. Expecting exogenous shocks instead of being all outraged when they happen is another.

That’s all that’s really on the mind.

That and how much code I have left to write. :)

The Seven Axioms and Predictive Validity

I published seven axioms over the past week – in a not so humble fashion. I’m taking the James Burke line to heart and just putting it out there.

The Seven Axioms are:

1. The purpose of analytics is to derive competitive advantage for the organization / firm / entity.
2. Data alone does not yield competitive advantage.
3. A sequence of progressive hypothesis testing is the most efficient and effective method to derive competitive advantage from data.
4. Predicting the future requires an understanding of cause and effect.
5. Correlation is not always Causality.
6. Accuracy over Precision.
7. It is possible for there to be two optimal, equally true, answers to a problem. (And Sometimes More!) (X^2 = 4, x=-2, 2).

They might appear to be fairly straight-forward. And they are. In my opinion.

A statement like Accuracy over Precision was certain to cause problems. And it has.

If you look at the language around cause and effect, causality, and there being many correct right answers to the same problem: you get the point. It follows from the Axioms that, to derive competitive advantage, you need to be able to make predictions about the future, and the only way to really get there is through progressive hypothesis testing with accurate data, and understanding both complexity and causation.