You Can’t Always Have Quant With Your Qual

You Can’t Always Have Quant With Your Qual

In an ideal world, all B2B research would result in a tidy heap of quantitative statistics, graphs, and charts all pointing to a clear course of action. Company leaders would glimpse the dramatic numbers and approve strategy adjustments right away.

But, as we know, fantasy is rarely reality.

This holds true with B2B market research. Many B2B tech companies would love to commission sweeping surveys that yield conclusive insights about the market.

Unfortunately, given that B2B tech is a rather niche field, it is often extremely difficult to get an appropriate sample for mathematically responsible conclusions.


B2B Revealed – ON: iTunes, Stitcher, and Google Play

Let’s walk through some of the unique challenges of conducting quantitative research for B2B tech.

B2B ≠ B2C

The B2B Buyer’s Journey Is More Complex than B2C

Market research for B2B companies is very different from B2C.

B2B deals are more complex. A CEB study found that an average of 5.4 buyers had to formally approve each B2B purchase. Whereas with B2C, there are usually far fewer buyers involved.

Conventional wisdom says that about five people are necessary to make a wise B2B purchase, as it’s usually a significant spend for the business. Hence, there is much more wooing involved to push through a B2B sale than a B2C one. Often, months or even years are spent cultivating relationships before that big B2B buy.

Small B2B Target Markets Make for Poor Survey Samples.

Be careful with sample sizes for quantitative B2B research.In many cases, B2B companies target much narrower markets than B2C companies do. A bottle of ketchup can be marketed and sold all over the world. Whereas B2B solutions, from our experience, may have a total addressable market of 100,000 companies or far less. This is a huge difference from a B2C product that could legitimately be sold to any US consumer who might walk into a grocery store.

Smaller target markets are one of the main reasons why quantitative research isn’t always the best method for understanding B2B business problems.

To prove this out, let’s just consider the math. Say your client gives you a list of 5,000 people they’d like to hear from. According to one sample calculator, you’d need 537 responses for a confidence level of 95 percent and a margin of error of 4 percent.

Response rates being what they are, you’re not likely to achieve that number.

B2B experts know to anticipate low responses. For instance, Kapost wrote of a survey effort,

“First, we were ashamed of our response rate. I’m talking a tail-between-the-legs, oh-goodness-that’s-bad kind of reaction. But after speaking with others in our industry, we now know that 1.1%-2.6% is actually quite good.”

Sure, your response rate could be higher than that, but it would have to be MUCH higher to get an appropriate sample.

So, back to our example. Let’s say you get a 2.7 percent response rate. That means you only get to talk to 135 of the target 537 people from the initial 5,000 person list.

It gets worse. Even if you got a 10 percent response rate, you’d still only get to talk to 500 people. Unfortunately, that’s still shy of a reasonable sample for the study.

Always evaluate whether you can get a mathematically responsible sample when considering survey methods and research design.

Surveys Are Limited Tools For Understanding the B2B Buyers’ Journey.

Typically, B2B stakeholders have a lot of questions about the buyers that filled out their surveys. However, they may not realize that surveys aren’t always the best way to understand the B2B buyer’s journey.

Remember, the average B2B sale involves about five buyers. To understand how they reached the decision to purchase, you’d have to hear from each of those five buyers and learn their role in that choice.

Back to our example. Let’s say you get a 2.7 percent response rate on your 5,000 person sample. That means you have 135 responses. Then, say that responses are somehow magically divided equally into five uniform roles in the purchase decision. That means you have roughly 27 responses from each persona. That’s just too small a number for solid analysis.

How To Get It Right For B2B

There are some ways to use quantitative surveys effectively in B2B. They’re not without compromise though.

Up The Sample Size.

First, you could get a larger sample and then generate a larger number of responses.

Frankly, this isn’t always possible. Perhaps the individuals the study targets are in an extremely niche market. Or, the research focuses on a certain country where the solution in question isn’t well known.

There are a million reasons why it could be difficult to get an appropriately sized sample for decent quantitative B2B research.

Ask The Client To Give You a List of People To Survey.

While this option could take some of the leg work off market research firms, there is an inherent challenge.

Any list a client provides is likely to be biased, especially if it’s based on a mix of marketing and sales leads. These leads are already predisposed to consider or buy a solution from the client. These folks alone won’t give an accurate portrayal of the client’s position in the marketplace. For that, you’d have to also hear from competitor customers and customers that decided not to buy at all.

Concentrate On Getting More Responses.

You could also put a ton of effort into initial and follow-up outreach to survey respondents to increase the response rate.

While these efforts may get you more responses from a small list of targets, it doesn’t change the fact that the list was small to begin with.

Improve The Quality Of Your Sample.

Take a page from qualitative B2B research.

Accept the fact that you’re not going to get more responses, and get to know more about who responded to the survey.

This doesn’t allow you to project your findings across a broad population in every way you might like to. However, you can have more confidence in your findings.

Stop Hitting Divide.

Finally, you might have to limit your quantitative goals.

For example, for those 5,000 targets in our example, if you adjust the acceptable margin of error to 5 percent, you only need 357 responses.

You also might have to stop hitting divide on your calculator. Say a stakeholder wants to project the research findings onto the UK, France, Germany, Russia, and the US. If you don’t have enough responses to meaningfully do that, don’t try. Seriously, don’t try to take 357 responses from 10 different countries and make projections for each individual country. That’s just bad math.

Also, you may have to limit the level of analysis you do on a persona, title, or industry basis. For example, if you’re doing research on key buying criteria, you might be able to figure out how an organization buys and who’s part of a typical buying committee. But, you won’t be able to drill down into how each individual buyer buys or is influenced.

Don’t Ship an Illusion.

Make every effort to avoid presenting faulty data as a firm conclusion.

If the sample for your survey can’t provide an accurate representation of the populations you’re trying to study, you may have to make the tough choice to ditch quant altogether. Otherwise, you’ll be making a bunch of bad decisions on flawed statistics.

Ditching quant can be challenging to propose because senior stakeholders often want information that they know can’t be argued with. They want insights that come with lots of numbers, graphs, and charts.

However, you’re not helping that senior stakeholder if you design a survey that can’t possibly get a sufficient number of responses for good analysis. That’s worse than not giving them quant in the first place.

Luckily, ditching quantitative doesn’t mean you can’t study the problem. Perhaps you just need to use a qualitative method instead.

Qualitative Research To The Rescue!

Qualitative B2B research can often answer B2B questions much more effectively than a survey.Qualitative research can often answer B2B questions much more effectively than a survey. It’s a good compromise. While you won’t have the ability to project the findings from a qualitative effort to a broader population, you will be able to answer many questions and have confidence in the results.

You’ll be able to deeply investigate key themes, drivers, and motivations.

In addition, you won’t need nearly as large a target list as you would for a quantitative study. For many of our B2B qualitative research studies, 12-20 research participants adequately cover a homogenous population.

Also, recruiting success rates are typically higher for qualitative studies. In our experience, as high as 30 percent of the population under review responds. This means that not only do you need fewer respondents in the study, but you can recruit more people from the same list than if you were doing a quant study. Further, your initial list of targets gets to be much smaller as well.

For example, you could tackle five different buyer personas in a study that only involves 50 interviews with a target list of about 500 people. That’s 10 interviews per persona.

It is important to keep in mind, though, that these qualitative approaches are not going to allow you to project your results onto a broader population.

They will give you a great deal more discovery, insight, and investigation than you already have on-hand.

Better yet, it keeps you from using quant in ways that are not only inappropriate but dangerous. You don’t want bad math informing business decisions.

In sum, the next time a stakeholder tries to take quant study beyond its design limits, just say “no.” Chances are, qualitative will get you results both you and your stakeholders can use to inform smart decisions.


This blog post and podcast episode are brought to you by Cascade Insights.

Sean Campbell

Sean Campbell

CEO
Sean Campbell
Sean Campbell

Sean Campbell (All Posts)

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.