On Error

There are at least five types of error related to analytics. These are instrumentation error, algorithm error, transposition error, statistical error, interpretation error.

1. Instrumentation Error

When the instrument is measuring a phenomenon incorrectly. This is not to be confused with a human mistaking what an instrument really actually measures. Rather, this is when the instrument itself is only recording half of something. Or not measuring something at all. It’s akin to saying that thermometer is broken.

Instrumentation has varying degrees of accuracy. For instance, the unique http cookie is subjected to fault as a result of a deteriorating cookie retention curve. The instrument continues to work just fine – it’s just that user behavior has changed, affecting its accuracy. To be clear, that’s not the fault of most instruments that use http cookies. The instrument is just fine. The underlining behavior is making the instrument less useful.

2. Algorithm Error

You get an algorithm when a calculation just goes wrong. Take, for instance, the intent that rate of change is ((present-past)/past)*100, but sometimes gets mixed up. Sometimes, in certain complex algorithms, unexpected (unintended) cases slip through, and they, in turn, are a source of error. It’s akin to saying ‘getting a rule wrong’.

3. Statistical Error

There’s the instance of error that is caused by the forced necessity to sample the world instead of absorbing it all. These types of error are imposed by the universe and fairly bad random table generators. It’s somewhat akin to saying that you’re being fooled by randomness.

4. Transposition Error

Transposition error happens when a human, usually an analyst, has to manually move a set of figures from one medium or location to another. That is to say, from a tool into excel, or from one area of excel to another area of excel.

5. Interpretation Error

Interpretation error happens when a human, usually an analyst or a reader, believes a number to be something that it is not. For instance, confusing unique cookies for people.

The five types of error are omnipresent in analytics. It’s best to acknowledge them and manage them as best as you can.