The Quantitative, The Qualitative, The Total Consumer Experience

A tier one MSI topic focuses on how should quantitative methods and qualitative methods be combined to understand the total consumer experience.

It’s an excellent topic.

The two worlds aren’t natural complements. They have radically different systems of activities, tools, and methods, which in turn affects their own experiences, and how they see the world.

However, if the stance is unified, in the form of understanding the total consumer experience, the sum of the two approaches produces such more.

That focus creates the cohesion.

Facts, Experience, and Anecdata

Have you ever been asked how many people need to be in a focus group before their statements become statistically significant?

It’s a pretty neat question. What are they really asking when they ask such a question?

I think it has to do with how a fact is generalized into a fact about population.

A fact is a verifiable/verified truth.

It might be a fact that a focus group of eight men, aged 60 to 75, did not enjoy that episode of Family Guy. It may be verifiable because the transcript, backed by an audio recording, proves that every single participant in that focus group used terms like “repugnant”, “disgusting”, “toilet humor”, and none of them would recommend the show to a friend, or anybody they know for that matter.

The usefulness of that fact in making a decision is what motivates the question:

“Is it a fact that that all men, aged 60 to 75, will not enjoy Family Guy?”

The quant will reply of course not. A sample size of 8 is insufficient to make a truthful fact about the preferences of a large set. It is, after all, called the law of large numbers. Somebody with experience in survey methods might point out that the answers are not standardized in a focus group, nor, is the environment of a focus group likely to generate an unbiased result. Somebody with experience in the dirty world of behavioural data might point out that nobody actually really tested if men aged 60 to 75 would watch a whole episode if nobody was watching them.

The desire to project a fact, to generalize one, onto a very large population, is important because it reduces uncertainty. It’s not necessary to get to 100% certainty to make a decision. By the time you get to 100%, it’s too late. It’s only necessary to get enough facts to cover between 40% and 70% certainty (Colin Powell’s rule).

To a qual, or a design thinker, the discounting of their experience with the experiences of a group of people, to disregard it as mere anecdata, is very harmful and hurtful. There’s value in their experiences in nature.

It shouldn’t be discarded so quickly by quants.

After all, quants engage in qualitative realities too.

Simulation

Many curious quants engage in simulation.

My favourite quants took what they observed during their experiences with university administration duties and turned into a FORTRAN program to simulate and expand their understanding of organizational decision making.

Quite a few quants turn to simulation to experience outcomes and to iteratively ask what if. Many and then use those experiences to expand their model. That exercise in experience can be treated with skepticism, but it produces useful outcomes.

Some quants may choose to qualitatively experience a model.

Every Qual chooses to engage with nature itself.

The Consumer Experience

Qualitative and Quantitative methods, when taken together, can be accelerate understanding the entire consumer experience. That extra speed is especially important in digital, where technological advances change consumer behavior and preferences quickly.

Qualitative methods are excellent for testing the first handful of hypotheses, like the initial aspirations and preferences of decision makers. When you show a small set of people a lot of questions, you minimize the risk of alienating mass audiences (which can be a concern) and you temporarily increase the number of questions that can be behaviourally validated. Competitive analysis, the activity of evaluating the choices that others have (or not) made, is a very useful input. Tailights can be a valuable input if you’re behind. Qualitative methods are also very useful for finding out why behavioral patterns that mysteriously appear in the data appear after launch.

Quantitative methods were once constrained by scale because people only have so much patience for surveys, and most CRM databases were incredibly resistant to entropy and change. Many people spent much of the 20th Century torturing a 300×3000 matrices for ideas about why people said what they said. Thanks to advancements in digital technology, we probe matrices that are much larger. But they’re sometimes not interactive. That is to say, a query is limited to what’s gathered by the instrumentation.

The two can be intertwined and work together, and work iteratively, getting to better results, faster.

If both the qualitative and the quantitative are united in the singular mission of understanding the total consumer experience, they can work wonderfully together.

Their strengths are complementary in the mission, even if their respective tools are not.