When we deliver corporate training, we spend a fair amount of time talking about the differences between market research (MR) and competitive intelligence (CI).
In the simplest terms, MR is about extrapolating across a population, and CI is about getting an answer.
For example, if 600 people out of 1,000 think the most important feature is X, then 600,000 out of a million will also think that. That’s a simple MR calculation.
Now let’s say we’re going to determine the competitor’s standard discount percentage, or find out when the competitor will launch the next version of its product. You can’t get your answers through extrapolation; there isn’t a population to generalize across. There’s only one “true” answer for each. These are CI questions.
The problem becomes evident when you’re presenting findings. For example, if you say, “We believe their standard discount is 15%,” a market research person may ask, “What’s your sample size?” You respond, “Three people.”
“N = 3???” thinks the market researcher. “What’s the margin of error on a sample size of 3???”
Wrong question. That’s like asking how many miles per gallon a cell phone gets. MPG is a nonsensical formula in this context. Likewise, margin of error is nonsensical when you’re not generalizing across a population.
The real question is this: what are the odds that three independent people gave the same answer? This is something you can calculate.
First, what are the values that a respondent could have given? With industry expertise, you might determine that the lowest reasonable number anyone would have given is 10%, and the highest that wouldn’t be totally odd is 30%.
And you determine that people probably aren’t going to break numbers down in intervals smaller than 5%. In other words, you wouldn’t expect someone to say a standard discount is 18.275%. So there are five possible answers in your range: 10%, 15%, 20%, 25%, and 30%.
Now say you’re having a conversation with someone who seems knowledgeable, and you ask about the standard discount percentage. He or she says the discount is 15%.
You have no idea if this number is right. You can’t present this as “the answer”. The person might have just made it up. So you talk to another person and they also say “15%”. What are the odds that two independent people both pick the same number? The probability formula is:
Probability = (The number of ways an event can occur) / (Total number of possible outcomes)
With two people choosing from five options, there’s 5 ways they could both pick the same number out of 25 possible combinations. So the odds that they just happen to both guess the same number are 5 / 25 = 20%. That means there’s an 80% chance that if they both chose the same number it’s the real number, and a 20% change they both just guessed the same. That’s pretty good, but not great.
Now let’s say you talk to a third person, and they also say, “15%”. What are odds that three people pick the same number? There’s still 5 ways they could all pick the same number, but there are now 125 possible answer combinations:
5 / 125 = 0.04, or 4%
With only a 4% chance that they all guess the same number, that means there’s a 96% chance that they’ve given you the real number. 96% is at or above the bar for typical market research, so it should be treated with the same amount of confidence you would place in a good quant study.
By Sean Campbell
By Scott Swigart
Get in touch
"*" indicates required fields