Yesterday, I defined what a KPI is, and explained the existence of KPI creep. A List of KPI’s Is NOT a System or a Model A list of KPI’s, 10 to 15 in a young programme, is not a system or a model. Take, for instance, the following list for a standard eCommerce shop: Net Revenue Gross Revenue Conversion Rate Number of Conversions Average Revenue per Checkout Number of Checkouts Started Checkout Abandonment Ratio Number of Carts-Started Number of Carts-Abandoned Cart Abandonment Ratio Average Items Viewed per Visit Number of Visits That list, unto itself, does not constitute a system of thought. If I turned this on its side, and drew arrows between the factors, then yes, it would constitute[…]
There’s a problem with Key Performance Indicators (KPI’s) as the general religion of Digital Analytics. Specifically: (1) On their own, a list KPI’s does not constitute a system of thought or logic. (2) A list of KPI’s is extremely difficult to optimize against as a whole. Statement (1) compounds the problems in Statement (2). Definition of KPI: Let’s go back to the 2007 Web Analytics Association Standards document for the definition: KPI (Key Performance Indicator) — while a KPI can be either a count or a ratio, it is frequently a ratio. While basic counts and ratios can be used by all Website types, a KPI is infused with business strategy — hence the term, “Key” — and therefore the[…]
You should read the full post, entitled “How Reddit algorithms work“. It’s a great read. TL;DR: Reddit harnesses the power of the recency logarithm in its ranking algorithm. Reddit weighs the first ten upvotes equally to the next 90 upvotes. Reddit values Recency above all else. The logarithm in the Reddit algorithm favors early pick up. Getting 10 upvotes is no easy task. Getting the next 90 is as hard. It’s the same math used to express the strength of earthquakes. Reddit’s bias/variance towards recency has a cost. High frequency/low recency power users are prone to scream ‘repost’ on a four hour old article. Casual redditors, who are slowpokes, are more likely to be annoyed with power users, who cause[…]
If you don’t know about Octave, it’s new to you. Octave is a high level language and real time compiler, all nicely contained into a single package. It’s free and well documented. I use it to: Run functions against relatively small datasets. Rapidly visualize complex functions fitted against that data. Answer questions that I’m not going to have to answer repeatedly. I use Octave to prototype and check functions quickly, and move on into the heavier languages to the repeatable bits. If you don’t have SPSS or the upfront patience for R, Octave is a pretty good quick start language. Check out Octave. *** I’m Christopher Berry.I tweet about analytics @cjpberryI write at christopherberry.ca
Phil Mui wrote a nice piece yesterday, introducing a new Google Analytics feature that links upper-funnel visits-from-social sites to downstream conversion events. They’re calling it “Assisted Conversions”, and it’s a good thing for Digital Analytics. The new report, called social value, enables you to see assisted conversions. That is to say, conversion events that in some way started or intervened over the course of a consumer journey, on their way to a conversion. If Google Analytics is set up properly, either by way of eCommerce pass through, or, assignment of a dollar value to a conversion event, Google Analytics will calculate a social value amount. Example: Say that you sell boardgames through your eCommerce store. Say you post something about[…]
This piece from 2005, entitled “Why software sucks“, recently gained prominence once again. I’m happy it did. Two choice quotes from the piece, and then an editorial. “The three things that make [software development] difficult are: Possessing the diverse skills needed not to suck. Understanding who you’re making the thing for. Orchestrating the interplay of skills, egos and constraints over the course of the time required to make the thing.” And: “If you look deeper, you’ll find that when people say “this sucks” they mean one or more of the following: This doesn’t do what I need I can’t figure out how to do what I need This is unnecessarily frustrating and complex This breaks all the time It’s so[…]
Some very good progress on what a Data Scientist is, and isn’t. @neilraden and @teddy777 have contributed, and here is where it started. – and where we’re at now. Some people say: The definition of a scientist is somebody who does original research and publishes in peer reviewed journals. Most people who call themselves data scientists aren’t actually scientists. Data scientists should be stratified depending on the sophistication of the tools they use. A few points to make: Science is a learning algorithm. If you’re executing the algorithm, then you’re doing science. If you execute the algorithm frequently, then you’re a scientist. Science is what you do. Most people aren’t scientists because they don’t actually use the scientific method. Consider[…]
Sometimes the components of a marketing channel will not add up to equal the total performance of the marketing channel. This is caused by any number of realities and limitations imposed in part by nature, and, in part, by you, the marketer. Consider the following deliberately simple scenario: March 2012 Impressions: Total Digital Impressions Delivered: 100,000,000 Total Impressions with Chicken Creative: 25,000,000 Total Impressions with Beef Creative: 50,000,000 Total Impressions with Pork Creative: 75,000,000 Something doesn’t make sense. I’m telling you that 100,000,000 impressions were delivered in total, but each component of that figure: 25 million, 50 million, and 75 million, don’t actually add up. That’s because creative can have multiple attributes. An ad may feature Chicken alone, Beef alone,[…]
Some people who rely on their gut argue that data driven decision making causes analysis paralysis. Some people who cause analysis paralysis have very good reasons for appearing to be paralyzed. I can think of three classifications of inquiry that correspond to three levels of information sufficiency. Specifically: Convenient reasoners, those who know what they know, and are looking for evidence to support their case, have enough information when they feel like they have enough compelling evidence for their case, and no more. Those who have a hypothesis will be temporarily satisfied with a firm accept/reject. The very next inquiry will either be based on convenient reasoning, or, another hypothesis. Those who don’t know what they don’t know and have[…]
The 2011 Canadian Election Study is available for download. You can get the file here. (If you don’t have SPSS, you can load it into R using the SPSS import functions.) I invite you to explore it. What is it? An entire generation of Canadian market researchers and pollsters grew up on the Canadian Election Study (CES). And there are a lot of them! There were federal elections in Canada in 1997, 2000, 2004, 2006, 2008, and 2011. That’s 6 elections in 14 years! It generated an incredible amount of publicly available data about political attitudes in Canada. Research uses aside, the CES is used to teach Canadians about electoral behavior. It is among the most studied data sets in[…]