I’m testing three themes:

• How to fight nature’s pesky way of inhibiting our ability to make clean causal statements.

• The importance of imagination in identifying independent variables.

• The role of evidence in decision making.

Simplification of a message is not pandering. However, many pandering statements are deliberate simplifications.

If your optimization objective is to gain followers:

• Post often.

• Post simply.

• Post what people want to hear.

I’m choosing simplification while avoiding pandering.

Let’s see how that unfolds over the next 60 days.

Consider the following deliberately simple scenario:

March 2012 Impressions:

• Total Digital Impressions Delivered: 100,000,000

• Total Impressions with Chicken Creative: 25,000,000

• Total Impressions with Beef Creative: 50,000,000

• Total Impressions with Pork Creative: 75,000,000

Something doesn’t make sense. I’m telling you that 100,000,000 impressions were delivered in total, but each component of that figure: 25 million, 50 million, and 75 million, don’t actually add up.

That’s because creative can have multiple attributes. An ad may feature Chicken alone, Beef alone, or Pork alone. An ad may feature Beef with Pork. An ad may feature Chicken with Beef. An ad may feature Chicken with Pork. In a crazy twist, perhaps some creative features all three! (The madness!). Attributes can cause such complexity when it’s possible for a single thing to have multiple attributes.

The next scenario demonstrates complications that arise because of instrumentation:

March 2012 Impressions:

• Total Digital Impressions Delivered: 100,000,000

• Total Impressions served to Males: 60,000,000

• Total Impressions served to Females: 10,000,000

• Total Impressions likely served to 35 to 50 year olds: 1,000,000

All people have attributes, but not all people have attributes that can be measured.

It might very well be that for the XBOX Live component, Microsoft can report with greater certainty, owing to profile information, that the content was served to more males. And, because that particular app was geared towards males, there’s greater certainty on that end. It also might be the case that another component was on mommy blogger ad networks, however, the knowledge of the ad targeter was really ethical, and wasn’t uniquely tracking everybody, so, the ‘missing 40 million impressions’ aren’t missing.

The same goes for the age component. We may hypothesize because of Quantcast data that those impressions served on mommy blog networks were heavily 35 to 50 year old females, but, there’s nothing in the instrumentation itself that confirms that hypothesis.

Just because it may be measurable doesn’t guarantee that it will be measured.

Finally, consider the complexity imposed by time:

March 2012 Impressions:

• Total Digital Impressions Delivered: 100,000,000

• Total Impressions from Affiliate Program: 10,000,000

• Total Impressions from the RayRayHayHay campaign: 8,000,000

• Total Impressions from the A campaign: 1,000,000

• Total Impressions from the Eh campaign: 1,000,000

Well, CLEARLY the A campaign and the Eh campaign failed – since the affiliates didn’t use those creative treatments much at all. What we don’t know is time.

• Date the RayRayHayHay campaign creative was posted: January 5, 2012

• Date the A campaign creative was posted: March 1, 2012

• Date the Eh campaign creative was posted: March 28, 2012

That’s 1 million impressions served in 3 days for the Eh campaign. That’s 1 million impressions served in 31 days for the A campaign.

Such component analysis is made particularly tricky when we’re trying to do it using a monthly report or some other arbitrary unit of time.

In sum:

Channel performance analysis is not channel component analysis. These are two distinct types of analytics, aimed at answering two different classes questions. For the reasons listed above, attribute overlap, instrumentation limitations, and time, the sum of the components may not add up to the total. This is not a devastating realization if you understand the differences and how to think of them.

There’s a general optimistic sense that drillability, the ability to drill into any metric and see its components, is possible in all contexts. It is possible in some contexts. It is not possible in all contexts. Privacy and technical disruption impose long run constraints in ever being able to achieve that.

It’s not likely to be perfect any time soon, and, in some cases, the components won’t ever add up.

***

(Note to fellow analysts: I chose impressions to keep it really simple. On-site and post-click analysis is required. Statistical analysis exists for a reason, so, even armed with impression and CTR data, you may analyze performance across multiple attributes. Moreover,you ought to be aware of the biases that exist in your data set – is it the case that males really did respond better, or, is it the case that the instrumentation is just better at identifying males?)

But first – three notes:

• You may be familiar with Reddit. If you’re not – you can read this explanation about what Reddit is.

• To answer that question, I downloaded a dataset that was built in early 2011 or very late 2010. The dataset is a 29MB gzip compressed and contains 7,405,561 votes from 31,927 users over 2,046,401 links. You can read about the methodology here.
• The file contains three columns – a vote, a userid, and a link. Only people who had their privacy settings set to open had that data read by an API. There is no meta-data about who these people are in real life (IRL) or even what was the nature of the content they were upvoting and downvoting.

So, who’s downvoting you on reddit?

To find out, I took that huge file transformed it into another one – boiling it down into a single user name, how many times that username vote (numberofvotes), and the average of all their votes.

You can see below that _mike voted 26 times, and, if you take the average of all his votes, +1 for an upvote and -1 for a downvote, it turns out to be -.92. Basically, _mike didn’t like a lot of what he saw. In fact, _mike upvoted once (+1) and downvoted 25 times (-25). So (- 25) + (+1) is -24, and -24/26 is -.92.

There are over 30,000 usernames here – and that’s a lot of data. It’s really important to visualize the data before you really get into any analysis. One way to do that is to run a histogram.

To read the histogram below, remember:

• Frequency means ‘the number of usernames that fall into this category or range’.

• Numberofvotes means ‘the number of times a username voted.’

• Mean is another word for average.

There are three takeaways from the histogram above:

• The average number of votes by a username was 234.

• A large number of usernames didn’t vote very many times at all.

• There are bumps at 1000 and 2000 votes. (If you’re interested as to why – see the Methodological notes. Incidentally – this is why you should always visualize your data.)

A histogram is built from a Frequency Table, which we’ll see below.

The way to read a frequency table is:

• The ‘Valid’ column means ‘how many times a username voted’.

• Frequency means ‘the number of usernames that falls into this category’.

• Percent means ‘the percentage of all the usernames that those in this category represents’.

There are three takeaways from the Frequency Table above:

• 4877 of the usernames only voted one time (It’s likely they submitted a single link and never returned).

• Note how both the percentages and number of usernames in each category decrease.

• 50.1% of all the usernames voted 20 times or less. (Look at the cumulative percent column and make sure that makes sense to you. We’re going to use this column later.)

You may have heard the term ‘long tail’ many times before. This is a demonstration of what that means. The bars on the histogram falls away to right.

Recall that the average of all the votes a username made is called ‘averagevote’. If somebody was persistently downvoting links, they’d have a negative number. If they upvoted everything they saw, they’d have an averagevote of +1.

Read the histogram below.  The three takeaways are:

• Negativity follows a reverse long tail. (It really happens – see how the figures fall away to left)
• On average, usernames upvoted what they saw (average 0.79).
• There are bumps at 0 (related to a methodological note) and at -1.

By now, two of my good friends in London are screaming at the screen. Means are a horrible way to explain long tail distributions. You can see that now too. Means are giving us a pretty skewed view of the world.

The table below is a byproduct of our Frequency table. It’s aptly labeled ‘Statistics’, and compares these two variables, numberofvotes, and averagevote, side by side. I’ve thrown a yellow box around ‘percentiles’. Recall the cumulative column from previous frequency table.

• 22.8% of all usernames voted 2 times or less.
• 40.8% voted 9 times or less.

The program I’m using is giving me ‘break points’ for those percentiles.

Two takeaways:

• The median gives a better summary of what’s going on here – half of the usernames voted 20 times or less, and, another set of usernames always upvoted what they saw.
• If I know that roughly 80% of all usernames posted 325 times or less, then I know that 20% of the usernames in my sample posted 325 times or more.

We’re going to use those percentile cutoff points to inform a segmentation, next.

Segmentation

A segmentation is a grouping of records, usually people, into categories. There is not prescription for how to do this. If you talk to a modeller, they’ll tell you about their clustering algorithms. If you talk to a machine learning scientist, they’ll tell you about bump-hunting or unsupervised machine learning clustering. Those are all very good algorithms. I use them myself.

I’m going for simplicity here. I have these four percentile cut-off points that evenly cut people into five categories. And, for further simplicity, instead of referring to a group of people who posted between 9 and 48 times as ‘those who posted between 9 and 48 times’, I’m going to call them Average-Andy’s. And I’ll just keep on calling them that.

At this point, I don’t know if they’re male or female. (And we won’t in this thread). And it’s controversial to use alliteration. But it’s done.

So, mapping the percentiles against a segmentation, based on how many times a username voted, we have:

• 1 time: One-Time-Oliver

• 2 to 9 times: Vanity-Vanessa

• 9 to 48 times: Average-Andy

• 48 to 325 times: Frequent-Fred

• More than 325 times: Power-Pauline

Take a look at the result below – a variable I’m calling ‘equalseg’ – short for ‘equal segmentation’.

Takeaways:

• There are 4877 One-Time-Olivers, representing 15.5% of the usernames in the sample.

• Vanity-Vanessa’s represent 23.9% of the usernames.

• The last three segments are pretty equally divided – the first two are more lopsided.

Even though I aimed to have five groups of people with equal numbers in each, you can see the division between One-Time-Olivers and Vanity-Vanessa’s are off. This happens very often when segmenting a long tail into equal groups. And, while not ideal, it’s okay for our purposes.

Next, we’re going to examine each segment individually.

One-Time-Olivers

There are very efficient ways that statisticians quickly summarize and understand the relationship among variables. The aim here isn’t to be efficient – but to be clear. In that spirit, I give you the histogram below.

Takeaways:

• All 4877 One-Time-Olivers voted exactly one time.

You should lol. It makes sense though, right? And, the segment name should make a lot more sense.

The histogram below summarizes how, on average, One-Time-Olivers voted – positive or negative. Since they only voted one time, it’s either an upvote, or a downvote. A +1 or -1 average.

 Takeaways:

• One-Time-Oliver’s tend to upvote once, and are never heard from again.
• In answering the question – “Who’s downvoting you on Reddit”, it isn’t One-Time-Olivers.

 Vanity Vanessa

Vanity accounts frequently enter Reddit, they flicker, and they go out. They get discouraged. They never really commit to the bit. That’s what happens to them. The histogram below takes on that familiar long-tail curve.

Takeaways:

• There are lot of Vanity-Vanessa’s, some 7,527 of them.

• Most of them posted only 2, 3, or 4 times.

So, how did they vote?

The histogram below summarizes the story:

Takeaways:

• Vanity-Vanessa’s upvoted nearly everything they saw, with very few exceptions.

• Very few persistently downvoted everything they saw.

• They’re not the ones downvoting you on Reddit.

Average-Andy’s

Recall that the average username votes 326 times, and yet, I still labeled Average-Andy, ranging between 9 and 48 votes, as average andy. That’s because the mean number of votes that Average-Andy’s cast is 22.25 – which is close to the median of 20 for the entire set.

This mixing and abstraction of median, mean, and segmentation isn’t something that I expect most people to consider or think about, but I can foresee some getting hung up on it. When you think about an equal segmentation though, it makes sense that the mean of your middle category should be close to the median of the entire set.

For everybody else – just know that you’re you’re looking at the “average joe redditor” here.

Takeaways:

• Average number of votes is 22.25, close to the median of 20 for the whole set.
• Familiar long tail.

How do they vote?

Takeaways:

• A majority of Average Andy’s liked everything they saw – they upovoted everything.

• They downvote more often than Vanity-Vanessa’s or One-Time-Oliver’s, but not massively.

• They aren’t downvoting in such a huge way to say that these are the ones downvoting you on reddit.

Frequent Fred

By now you’re pretty much a pro at reading these histograms. Frequent Fred’s vote frequently. Look at the histogram below.

Takeaways:

• Classic long-tail continues.

• Averaging 139.3 votes.

• The unusual bump at the beginning of the series is just magnified by the scale from the previous vote frequency histogram. (It’s fine).

How do they vote?

Takeaways:

• Far fewer of them are likely to upvote absolutely everything they see.

• There’s significant flattening of the long tail – the average is .74.

• More of them, on average, are disposed to downvoting.

Power Paulines

Power Paulines are the most difficult group to analyze, but the easiest to summarize and understand. Take a look at the histogram below.

Takeaways:

• The long tail is holding – there’s significant clustering at 1000 and 2000.

• The cause is related to rate limiting within the Reddit API.

• The longest part of the long tail – those power users with thousands and thousands of votes, are all bundled and clustered together at 2000.

• There are around 500 of such power users, representing some 1.5% of the total usernames.

So how do they vote?

 Takeaways:

• The bump at 0 is caused by 1000 upvotes getting averaged out by 1000 upvotes.

• 0′s aside, which are tugging on the mean, Pauline’s are on average more prone to downvoting.

• Power Paulines are downvoting you on Reddit.

Putting a bow on it

The chart below summarizes the relationship between segment and their average vote. You can see a clear negative direction. The more one uses Reddit, the more one downvotes – even if the mean is exaggerated in the Power Pauline segment.

To really hammer the point home about the origin of downovotes, take a look a the table below. It’s broken out by the segments you understand. It also contains two new variables – upvotes and downvotes. That is the total count of the number of upvotes and downvotes made by each segment.

Takeaways:

• One-Time Olivers as a group were responsible for 175 of all the downvotes cast.

• Vanity-Vanessa’s as a group were responsible for 1781 of all the downvotes cast.

• Average-Andy’s as a group were responsible for 13,258 of all the downvotes cast.

• Frequent-Fred as a group were responsible for 120,758 of all the downvotes cast.

• Power-Paulines as a group were responsible for 1,672,368 of all the OBSERVED downvotes cast – but are probably responsible for a lot more in aggregate across all of Reddit. (This sample contains a bias, but bias doesn’t mean I can’t say anything at all about anything.)

Note the differences in order of magnitude between each group. 1781 is roughly 10 times greater than 175. And so, a bit imperfectly on the way up to Frequent-Fred’s. There’s an order of magnitude difference here in terms of the amount of weight each group casts.

The greatest power users users of Reddit are the ones who are downvoting you – and it’s an exponential power.

But wait, there’s more.

Recall, however, that there over 7 million votes cast. 1.8 million were downvotes, and 5.5 million were upvotes. Read the statistics table below to verify that.

Takeaways:

• Upvotes outnumber downvotes.

• The interface of Reddit itself causes upvotes to accumulate.

• Reddit itself is a cause of a bias – probably by design.

The histogram below is by links – the content getting upvoted or downvoted. There were just over 2 million links submitted. On average, each link received 3.62 upvotes. Given everything you know about long tails, think about just how deceptive that 3.62 mean figure is. Note how you can’t even see the bumps in the tail. And be in awe of the efficiency of the collective Reddit behavior that causes popular content to disproportionately promoted while even ‘good’ or ‘average’ content gets relentlessly shifted to the left – all by a very small group of people.

Takeaways:

• The long tail is long and powerful.

• This small group Power-Paulines are far more likely to downvote because of a much higher frequency of use.

I’m thanking Reddit for making so many API’s publicly exposed and enabling this sort of analysis and exploration. Thank you.

Portions of this post appeared on Eyes On Analytics the week of February 5, 2012.

The tl;dr summary:

“For Samsung’s 7500 and 8000 series TVs, all you have to do is say “Hi, TV,” when you walk into a room for the TV to turn on and know who’s there.”

“Think of it: The tech means an advertiser or TV programmer could, for the first time, know which members of a Nielsen household are watching a show or an ad. Cisco has even developed a system meant to read facial expressions and determine whether you’re entertained or bored.”

“Many people in the living room are multitasking with other devices. “We’re paying for that,” said Rex Harris, innovations supervisor at SMGX, a unit of ad agency holding company Publicis Groupe. “If you’re looking at other screens, then you’re not paying attention. We would like to know if we’re getting accurate impressions.”"

Commentary:

Alright – so – a simple innovation, the webcam, is jumping from the PC/DVR into a TV, and we get a few folks who come out and speculate what it could mean. It all ends up sounding like a 1984 telescreen idea, which, I’m 99% certain, is not what Samsung has/had in mind.

Broadcast isn’t digital.

Repeat: broadcast. isn’t. digital.

This has implications:

• There is enough inventory for targeted ads and offers in digital because the technology enables the creation of multiple ad treatments at scale. No such technology exists in the broadcast industry.

• People already effectively segment themselves by TV show preference.

• On Demand technologies like Netflix, and time shifting technologies like streaming and DVR’s, are already eroding the concentration of key market segments.

• Plot the S-curve adoption rate of the technologies driving market fragmentation against the adoption of new, Big-Brother enabled telescreens, and see which wins. (Hint: it’s time shifting and on-demand).

• You’re paying for junk impressions because we’re developing ad blindness, just like we’ve developed banner blindness.

No amount of surveillance is going to change that fact.

It’s also the first get together after eMetrics New York, which was a major, and had big time Canadian attendance. These tend to be among the more interesting evenings. It has also been some three months since the last WAWTO event, so there should be quite a few fresh stories.

It’ll be great to be back in New York.

There are a few people that I’m looking forward to seeing: John Lovett on social media, Melinda Driscoll on web analytics, Shari Cleary on media, Joseph Stanhope on mobile, Alex Langshur on government. And then there’s Michael Healy, Patrick Glinski, and me.

I’m presenting with Michael Healy on sentiment. Michael Healy is among the best thinkers in this space and is just great. There have been a few very recent breakthroughs in sentiment analysis over the summer (and as recently as last week), and I’m looking forward to explaining how to treat the measure. I understand a core problem with the application of the metric – the gap between what some want the metric to mean – and what the metric actually really measures.

I’m with Patrick Glinski of Idea Couture on Friday – presenting “Communicating Data to Designers”. It’s a really different topic, and something you won’t see anywhere else. It’s not on the radar yet as a differentiating competitive advantage. It’s new, it’s different, it’s fresh – and even a bit risky. So come on out. We won’t bite.

I’m looking forward to seeing you out.

It’s an excellent way to present data in a very accessible, very explorable way. I found it inspiring.

The call to action is “create your chart now”. A very good, honest, call to action.

The technology adoption S-curve can be a slow beast, and expectations of growth have persistently outstripped actual adoption, at least in North America, and especially in Canada. Adoption has a few drags on it in North America and Europe. No such drags exist in Asia.

The chart below compares all the countries smartphone penetration. (Click to embiggen)

That chart masks underlining maturity in each country. The chart below compares m-commerce ‘at least one time’ usage across Germany, China, and the United States. The big three economies. (Click to embiggen).

That’s a good snapshot. M-commerce is thought of as a pretty big risk in North America.

In many ways, the melding of couponing with check-ins was the right bridge into mobile for the times. As we all watch Groupon careen into the inevitable mess, we all ask ‘what’s next’. I ask, ‘where’s the utility’, ‘how can mobile be used to salvage previously wasted parts of my day?’

I look to m-commerce as being predictable. Certain firms, like Plastic Mobile, have extensive experience with eCommerce, and understand mobile. They’re not going to replicate the pain of the nineties. There’s an inevitability to it.

Though, painfully punctuated.

This would have happened sooner if it wasn’t for the double whammy of policy plus recession.

John Lovett predicts that Moneyball will put analytics on the map.

It’s likely. It’s just so damn relatable.

Ideas have a long journey from conception to popularization. Nash had been known to game theorists and a sub-set of political scientists who don’t understand people, since the beginning. Most didn’t learn of it until ‘A Beautiful Mind’ came out. Moneyball is that movie.

To extend the lesson from Moneyball -

1. Everybody has the same mental model of the way the world works with respect to some phenomenon. (Experience E used to perform Task T with performance P as the outcome).

2. A dominant model dictates a dominant set of Key Performance Indicators.

3. Somebody discovers a better model and validates it through analytics.

4. Somebody effectively competes better as a result of the new model.

5. Others resist the new model (lock-in), they persist in the market until they die or are defeated.

6. New model becomes the new dominant one.

7. Return to (1) and repeat.

***

I’m Christopher Berry.

I bridge the gap between marketing science and data science.

I welcome connections on Twitter and LinkedIn.

If you define analytics as being ‘the scientific method applied to data to generate sustainable advantage’, then there are three major concentrations of practitioners: finance, marketing and operations.

The financial sector breaks out into the risk management and the speculation fields. There’s a higher self-referential graph amongst the risk management people in insurance than there are on the banking side. The speculation analytics folks are at severe disadvantage against their New York counterparts. If there’s a thriving hedge fund section in our community, I don’t know about it.

Marketing is divided between startups, CRM vendors, agencies, and client side (which includes data mining and web analytics). There are very few deep pockets of specialization – but they do exist in social analytics, mobile analytics, and web analytics. Extreme subject matter experts in Omniture, Coremetrics, and Webtrends number fewer than 10. Subject matter experts in Google Analytics exceed 30, however, that’s not to be confused with ‘people who have google analytics on their blog’.

There’s a thriving operations section of analytics – supply chain/logistics, transportation logistics, and internal operations. I also lump the impressively large gaming analytics (online gaming and gambling), and the minuscule dating optimization folk into this space.

There’s variety and specialization in analytics just within Toronto. And it’s pretty awesome.

The best way to break into analytics is to practice analytics. People who practice analytics in a context specific to something they enjoy are more likely to enjoy success and attract attention.

The best way to find somebody who knows analytics is to attend an industry event. They are frequent and welcoming.

We need both physical technology and social technology to practice analytics at scale.

There are three grouping of physical technologies:

• First, there’s instrumentation technology that we use to measure  and record the world around us.

• Second, there’s analysis technology that we use to understand the data that’s coming.

• Third, there’s presentation technology that we use to communicate a world view, and what to do next.

On the instrumentation technology side, we’ve all had a few challenges with instrumentation as of late. Specifically, the understanding of definitions, their impacts, and the unexpected impact of bugs. Empathy from one technologist to another on this front. Instrumentation is not easy.

On the analysis side, SPSS, R, SAS, Datameer, Python. Amazing technologies, some of which may be used as controllers, some of which are used by analysts to peer into the deepest, most chaotic systems.

On the presentation technology side, we have excel, powerpoint, keynote, and certain dashboarding technologies. They have pros and cons. XML or JSON API’s ought to be the future, or some version of it, here. It doesn’t seem like a big problem. But it a fairly wicked one, because credibility and authority are bound up in aesthetic.

Getting these three physical technologies right, linked up together, is very important to practice analytics at scale.

Social technology at scale

People are incredibly important because they’re the ones who generate insight, and ultimately cause beneficial change that results in sustainable competitive advantage. It’s not the physical technology of software and hardware. The institutions that cause them to behave in very specific ways is a social technology. And it must be in place to scale.

There are a number of problems that an organization creates for analytics professionals.

For one, most organizations don’t know what they don’t know about analytics. They don’t understand that instrumentation is still young and buggy. That truth isn’t absolute. That sleuthing is part of the role. That it’s not just “pizza and spreadsheets”. That it takes time to put together a series of recommendations that make sense in a given a system or context. That not every convenient reasoning business case can be generated, or generated quickly. That not everything is recorded by the instrumentation. The first three months setting up any new analytical institution is entirely about resetting expectations.

There are a number of problems that analytical leadership causes for their organizations.

For one, most organizations don’t know what they don’t know about analytics. Bad behaviors result. They hive off the data. They clam up indiscriminately. They refuse to engage with regions of the company for extended periods of time without a strategy in place for such cut-offs. They hire the wrong people. They don’t secure headcount for enough people to be successful. They’re unable to demonstrate their own ROI. They don’t say no often enough to be able to cause their own ROI. They don’t champion their own work. They acquire a siege mentality. They don’t publish and they don’t share their successes with industry. They churn rapidly.

Not all leadership is bad and not all organizations are poisonous to analytics.

If we accept the premise that organizations want sustainable competitive advantage from analytics, and that analytics leadership wants that same outcome, we can construct a physical technology stack and a social technology stack that achieves that end.

Current thinking on that end:

1. Mediums and Medias are fragmenting. The most progressive thinking on the subject is towards medium planning (Syncapse, Teehan+Lax), and as a result, analytics leadership that resists new, novel, mediums are likely to be viewed as obstructionist. Instrumentation will fragment as a result. This is okay. Derive a medium measurement strategy. It’s what our collective leadership must become good at.

2. Analytics means having an analytical tool to use. SPSS is preferable because of usability. R is preferable in certain environments because it’s free. Firms that actively compete on analytics may require a big data stack to data mine very large sets.

3. Recommendations are communicated in powerpoint. The business schools have decreed this. Collaborate on problems without a powerpoint presentation. New thinking from the business schools have decreed this.

4. Every single organization has a C-level that always asks for go-pher analyses. The rest of the organization typically pays no cost to support an analytical seat. These ad-hoc seats are an excellent way to hire new talent out of the universities and are good training opportunities, under supervision. Allow your experienced guns to find the insights, and let your inters-juniors do the gophering. Record the output of the gophering.

5. Nobody in the organization has incentive to acknowledge the analytics departments for insight discovery. The leadership of that department must make sure that there’s a solid internal culture that rewards insight.

Most organizations desire analytics at scale – which means handling both the intelligence and ad hoc sectors of the business – simultaneously. The way to get there is by combining social and physical technologies that enable that scale. It will be an ongoing losing war, but every battle should bring victories.