ChristopherBerry.ca

Can I ask about what you think of #measure, which is the main analytics channel on twitter? Are you happy with what that channel has become? *** I’m Christopher Berry.I’m taking refuge over at #msure.I tweet about analytics @cjpberryI write at christopherberry.ca

On occasion, I use data generated from surveys to experience some empathy with groups of people who I don’t frequently encounter and interview about their realities. A beautiful, publicly available data set is the PEW Internet and American Life Project’s August 2011 Apps and Adult SNS Climate data set. You can access the dataset here. Thank you PEW. You’ll see the tables as I see them. I’m using column percentages. Note that there’s a confidence interval on either side of those percentages. If you don’t understand what a confidence interval is, don’t tweet about the figures you’re seeing. Don’t quote them. We’ll get to that in a bit. (In general, you don’t use tables to communicate with general audiences. Since[…]

It’s awesome to watch Pinterest grow. A post at High Scalability reveals just how much they’ve grown. TL;DR: 80 million objects stored in S3 with 410 terabytes of user data, 10x what they had in August. EC2 instances have grown by 3x. 150 EC2 instances in the web tier 90 instances for in-memory caching, which removes database load And a few notes about technology that caused a smile: Written in Python and Django  Hadoop-based Elastic Map Reduce is used for data analysis and costs only a few hundred dollars a month One of the fastest growing sites in history. Sites AWS for making it possible to handle 18 million visitors in March, a 50% increase from the previous month, with[…]

It’s the Victoria Day long weekend in most parts of Canada, and, whereas our colleagues in Moncton and Halifax will be working, many analytics practitioners in Toronto and Vancouver will be playing German board games and drinking beer. One of the most popular of the German board games is “The Settlers of Catan“. If you know the game, skip ahead to the next bold title. If you don’t know the game, read on for a painless summary. Here’s the TL;DR Summary (Too Long; Didn’t Read): The objective of the game is to win 10 victory points before anybody else does. (Keeps everybody in) You earn two victory points for every city you build, 2 for having the longest road, 2[…]

Consider the following two, distilled, points of view: Statement 1: “Big Data Analytics is going to change the way we do business. Sure, a lot of it will be routine “I’m okay!” status updates from sensors, but making sense of the key parts of it, like “help me, I’m failing”, will be extremely useful. Companies that were previously exempt from competing on analytics will be disrupted by new entrants who will compete better, either by being more effective or being more efficient. Big Data Analytics is already having a disruptive impact in marketing, where it never used to before, and is gaining huge traction in medicine. There is reason to believe that Big Data Analytics will cause better decision making[…]

Is the assumption that better data causes better decisions credible? It depends. If evidence to the contrary of an individuals aspiration comes to light, and that individual refuses to update their expectations or aspiration, then even the most pristine, accurate, precise and real time data will fail to change their mind. If evidence to the contrary of an individuals aspiration comes to light, and that individual updates their expectations or aspirations accordingly, then it will be effective at changing their mind. The key element that decides the effectiveness of data is the human. Great data can cause great managers to make better decisions. Great data doesn’t cure ignorance. Maybe the broader commentary on the value of Big Data has more[…]

Where does the assumption that better data causes better decisions come from? I don’t know for sure. But I can point to two possible sources: The enlightenment and the scientific revolution Robert McNamara and the Whiz Kid movement The entire scientific method is predicated on data. A hypothesis is either accepted as truth or rejected as false based on the data. There is no other arbiter. Faith or strength of opinion has nothing to do with it. The data decides. The assumption that greater knowledge causes greater outcomes flows from that fact. That, if you’re honest about being wrong, that everybody benefits. (Disturbingly, that trend may be reversing in The West, as negative findings have been disappearing from most disciplines.)[…]

Yesterday, I summarized two statements and asked four questions about Big Data Analytics. Today, we knock off the first question. Did anything go wrong in Business Intelligence generally and Web Analytics specifically? The extreme skeptics towards Big Data Analytics argue that Business Intelligence failed. Others point to cases of success. Who’s right? Today, we see: Major decisions about pricing, like those recently at Netflix, continue to be made without any analytical support (or, the analytics were completely ignored). Few people, to this day, really understand the numbers they’re looking at. (Few can explain the accurate definition of ‘time spent on site’, for instance). Hundreds of thousands of firms continue to compete just fine without any analytics at all, without any[…]

Consider the following two, distilled, points of view: Statement 1: “Big Data Analytics is going to change the way we do business. Sure, a lot of it will be routine “I’m okay!” status updates from sensors, but making sense of the key parts of it, like “help me, I’m failing”, will be extremely useful. Companies that were previously exempt from competing on analytics will be disrupted by new entrants who will compete better, either by being more effective or being more efficient. Big Data Analytics is already having a disruptive impact in marketing, where it never used to before, and is gaining huge traction in medicine. There is reason to believe that Big Data Analytics will cause better decision making[…]