You have a new B2B product or service that you’re rolling out to the market, but adoption is slow. How could you have avoided a slow launch? What should you have known before the launch?
Concept testing is the way to find out if your product or service will sell before you bring it to the marketplace. There are a number of concept testing methodologies to test your idea, each with their own pluses and minuses.
And if you have more than one idea, you can test multiple concepts with your prospects, too.
Concept Testing Lets You Ask Why
Qualitative methodologies allow you to ask open-ended questions that uncover important insights about your target customers.
For qualitative concept testing methodologies, an analyst conducts a series of in-depth interviews or leads focus group discussions. Your participants get exposure to your concept and provide their impressions. You can find out:
- Does your concept solve a real problem?
- Is your concept worth paying for?
- What questions does a potential buyer have when they examine your concept?
- How would they justify buying this to their leadership?
All of these answers help you refine your concept, and determine what collateral you’ll need to really explain your concept to the market.
A series of in-depth interviews will give you the most impartial reactions because the participants aren’t biased by what other people are saying. However, it takes longer to perform eight in-depth interviews than to conduct one focus group with eight participants.
Focus groups are better when you have a need for speed or when you want participants to refine each other’s thinking. However, one focus group participant may bias the whole group, skewing the results.
A qualitative approach allows you to get exploratory insights that can support B2B messaging frameworks and value propositions. If you jump straight to quant first, you might not even know the right questions to ask much less who to ask them to. Hence, we strongly recommend organizations begin with qualitative research (in-depth interviews, focus groups, etc.). Then move onto quantitative research (surveys, etc.).
You have the added benefit of gathering deep contextual insights during in-depth interviews or focus groups. The answers provided using these research methods are more rich. Participants are able to explain why they think a certain way. If you ask the same question in a survey, you’re limited to multiple choice answers or Likert scales to show the magnitude of a sentiment.
Leadership Likes to Count
Quantitative methodologies give you a chance to survey hundreds of people at once to reach a statistically significant consensus. And the output of quantitative research is charts and graphs, which are easy to understand. It also solves the “My leadership only believes numbers” problem.
One of the biggest hazards of quantitative concept testing methodologies is the potential for misinterpretation. If anyone in the test group has questions or needs clarification on the concept, they must use their best judgment because they can’t ask a moderator.
“It’s easier to defend quantitative research because it isn’t just what 20 people had to say; it’s maybe what 400 people had to say. However, they can’t ask any questions. If they don’t understand exactly what the survey is showing them, they might select very strange answers. ”—Scott Swigart, co-founder and Chief Research Officer, Cascade Insights.
For example, if you ask participants, “Does this concept address your top challenges with hyper-converged infrastructure,” do all the participants even know what hyper-converged infrastructure means? Or if you’re concept testing an “AI-powered blockchain for IT disintermediation,” you can be confident that no one will have any idea how to answer your questions.
Another hurdle with quantitative research is finding enough people to survey. B2B tech is pretty niche, so there are only so many people to reach out to. You also have to be very careful in how you qualify them to ensure that they’d really be buyers or influencers for the solution you need to test.
Mind the Gap
By relying on one method alone, you could easily be misled into thinking your strategy is sound. Your concept testing results could overlook the full picture. You may miss the mark during your product/service launch—a costly mistake.
Therefore, to get the best value out of concept testing, you’ll probably want to do a mix of qualitative and quantitative research. This way, you can still defend your results using a large data set. But also, you can provide context for the charts and graphs with rich contextual insights that highlight why people feel the way they do.
Both concept testing methodologies come together to provide a well-informed decision. Protect your investment, avoid wasted development time, and ultimately produce a product or service that will attract prospective customers.
With 15 years of experience in the B2B technology sector, Cascade Insights has helped companies concept test their messaging, value propositions, market share, and more. For more information on B2B market research, visit What is B2B Market Research.
Special thanks to Cascade Insights President & CTO Scott Swigart for advising on this piece.