Measuring Innovation

Challenged with the question: “But how can you possibly measure innovation?”

Alright, here’s my stab at it.

Do what most web analysts do – ask another question, definition related. “What do you mean by innovation?”

If I may quote Wikipedia (and I will)

The term innovation may refer to both radical and incremental changes in thinking, in things, in processes or in services (Mckeown, 2008). Invention that gets out in to the world is innovation. In many fields, something new must be substantially different to be innovative, not an insignificant change, e.g., in the arts, economics, business and government policy. In economics the change must increase value, customer value, or producer value. The goal of innovation is positive change, to make someone or something better. Innovation leading to increased productivity is the fundamental source of increasing wealth in an economy.

Alright, so Innovation is characterized by two things:

1. A positive increase in productivity, value, customer value, producer value.
2. It’s substantially different than existing product, process, thinking, or services.

And…we’ll accept a Kuhnian bias here: you can have innovation incrementally. Or, an innovation can be a paradigm shift.

Not all changes are positive.
Not all changes are substantially different.

And this is how we start to get into the weeds, right.

Is an increase in productivity of 0.0000000000000000001 dollars, after transaction cost, really positive? (By definition, it is).

And, how do we know that something is ‘substantial’? Is there an innovation referee that decides? Is that referee subject to being publicly abused by a rabid hockey Dad Tuesday night at the Centennial Arena?

Is innovation like obscenity? (I don’t know how to define it, but I know it when I see it?) Well…let’s not just throw our hands up in the air just yet.

We have to start laying down some operationalizations. They don’t have to be perfect. They just have to be excellent enough that their validity can’t be easily attacked.

Isolate the most important outcome first. Let’s say that it’s value.

Let’s operationalize ‘value’. Let’s define ‘value’ as “a rise in customer satisfaction based on the weekly rolling satisfaction poll, blended index of 10 service touchpoints.”

Let’s set thresholds. A positive increase, to ‘register’, or to ‘count’, has to be in excess of one standard deviation of the mean over a 6 week period, off of the 52 week moving average.

Let’s operationalize a “substantial change in process / thinking / product” (etc) as a board with stickies on it. Everybody is free to post to it. Staff votes on them. If something is executed or adopted, it goes into the excel sheet, and we track the impact on customer value.

Yes. BUT!

Oh, yes. Attribution.

With so many innovations going on, how do you know which is impacting what?

You’re right in that we can’t properly use linear regression (we’d be dealing with ordinal data as the dependent variable, nor could we guarantee that each innovation has nothing to do with one another.)

If customer perception of value is going up, and our ten satisfaction questions are well enough aligned, or are asking directly about what we’re changing, then we can certainly argue, more likely than not, and using control tables, that innovation is the root causal variable.

So, that, in sum, is how I’d go about it in a very narrow situation.

Define, dissect, operationalize, link with causality.

Then get buy in.

So much of this is making sure all the players of the game agree on objective rules before the game starts. Not easy in many organizations, is it. 🙂

Alright, fire away. 🙂