On the construction of metrics

The argument is as follows:

• There are an infinite number of potential metrics offered up by your standard analytics software.

• In spite of there being an infinite number of metrics, the actual amount of knowledge or information is limited.

• The value of a metric should be based on how much it contributes to the understanding of a system.

I’ll unpack that.

There are an infinite number of potential metrics. Take, pageviews. Take visits. Now divide pageviews by visits and call it ‘pageviews per visit’. Now apply a filter and look at only New Yorkers. You get New York City pageviews per visit. Now run a 31 day exponential moving average on the measure. You’d then get the New York EMA(31) pageviews per visit metric. Now, let’s make that pageview figure a little bit more descriptive by weighting all the pages that were visited by the average conversion contribution factor. You’d then get the New York Average Conversion Contribution Factor Pageviews per Visit EMA(31) metric.

Anybody can add complexity to a metric by exposing two or more metrics to each other, applying a subset constraint to it, through values through matrices, and applying time smoothing to it. (And so much more!) Understand that we can continue adding complexity to a metric in an infinite manner. There are more ways of composing a metric than there are atoms in the universe.

A metric only needs a definition. That definition doesn’t have to immediately useful to you, personally, or anybody else for that matter, to be a metric. In other words, a metric as a specification, on its own, may or may not be useful.

The set of useful metrics is much, much smaller than the set of all metrics.

The ultimate value of a metric is in how useful it is to your context. You don’t have to agree with everybody elses’ construction, either. To some, an exponential moving average is complex and doesn’t serve a need. To others, it’s great. The value of a metric is in the utility it brings to a context.

If it’s useful in the context of a system that you’re trying to understand, then it’s a useful metric.

If you’re constructing metrics, you should be aware that just because you can imagine complexity, it doesn’t mean that it’s useful.

***

I’m Christopher Berry.
I’m a Data Scientist.
This is what I’m working.