(503) 898-0004 | hello@cascadeinsights.com
Market Research Recruiting: How To Get A Superb Sample

Market Research Recruiting: How To Get A Superb Sample

/0 Comments/in , , , , , , , , , , , , , , , , /by

When it comes to B2B studies, having the right people is just as important as asking the right questions.

The efficacy of a market research study hangs on the knowledge of its survey respondents, interview subjects, and focus group participants. Clearly, research teams should invest in recruiting study respondents that are qualified to bring insight to the business problems in question. It’s only logical. Otherwise, why bother with a research project at all?

This article is based off a B2B Revealed episode.

You can listen to the episode or read the article below.

B2B Revealed – ON: iTunes, Stitcher, and Google Play

Sloppy market research recruiting is a recipe for disaster. It fails to snare respondents capable of answering the study questions. Unfortunately, rushed recruiting is the industry standard. Especially for panel firms.

B2B tech companies typically have a vague sense of who they’d like their market research teams to talk to. They tend to have an idea of who should be in a focus group, receive a survey, or have an interview. This does not go far enough. Vague guidelines are not sufficient for an effective respondent recruitment process. Speed should not be the top priority here.

Finesse and focus are required.

In order to find first-rate respondents for a B2B study, you need specialized knowledge of the industry. Much more so than you would for a B2C consumer study. Context is crucial. To use a skiing analogy, B2B market research is not a bunny hill. It’s a black diamond.

As a firm that does most of our market research recruiting for studies in-house, we know that the process can be challenging. However, our specialized studies bring extraordinarily powerful new perspectives to our clients. Those strategy-shifting results make the extra attention we place on market research recruiting well worth the effort.

Here is how we approach finding and enticing outstanding respondents to our B2B market research projects.

Don’t Just Take Anyone.

Narrow it down.

Not everyone is qualified to be a good study respondent. At Cascade Insights, we spend a great deal of time figuring out exactly who we need to hear from to illuminate gaps in our clients’ knowledge. Market Research Recruiting: How To Get A Superb Sample

Many teams we work with aren’t as meticulous. Like the skier afraid of anything with a steep slope, these teams have fallen into the habit of asking for only a limited number of requirements for market research recruiting.

Panel Firms Get It Wrong.

Sadly, panel firms are a frequent go-to for tech companies when they want to shove off their recruiting efforts. This is not a good practice. For niche audiences like the ones B2B tech needs to hear from, panel firms typically lack industry knowledge and expertise. Panel firms simply don’t have enough context to identify and recruit respondents who can provide valuable insight.

Some of our new clients are startled at the emphasis we place on a typical market research recruiting effort. I recently had a client from a Fortune 500 tell me, “I want to talk to developers.”

“And?” I replied.

“What do you mean?” they said, sounding surprised.

“Do you want to talk to them about DevOps or the Internet of Things? Do you want to talk to development managers, architects, corporate developers, developers working for startups, developers building SaaS apps, developers who work with AWS or Azure?” I asked. I could have gone on and on.

You need to drill down like this right at the start of a B2B market research study. Without this kind of specificity, you won’t get the right people.

Get reallllllllyyyyyy specific.

I’ll share another example to illustrate how important it is to get specific about qualified respondents. Say we want to talk to competitor customers for a qualitative research project. We would never stop at simply “competitor customers” as a parameter for market research recruiting. Instead, we would narrow our focus to customers that,

  • Bought from a specific competitor company.
  • Purchased a certain product or service from that competitor.
  • Belonged to a vertical or company size range that was of interest to our client.
  • Fit the buyer personas most relevant to our client.

Having the answers to these questions makes for a much more exciting and useful study. It’s also great for the client to have this type of strategic clarity and focus.

Get beyond broad phrases like “competitor customers.” You can do better.

Think more along the lines of, “Slack users who work for enterprise organizations in the manufacturing sector where the Slack deployment is sizable and has been in use for less than 12 months.” Now that’s a good respondent profile. You’re ready to start recruiting!

Catch a Quality Candidate.

Look for them in their social media habitats.

The best place to start looking for excellent study respondents is LinkedIn. Market Research Recruiting: How To Get A Superb Sample

A few years back, people would get very wary when I told them that we trust certain types of data from LinkedIn. They would stop me to say, “You can’t do that. My sister’s cousin’s best friend doesn’t believe in LinkedIn and he never updates his profile. That’s proof that it’s a faulty data source.” That’s what we call reasoning from a specific. Not a quality you want in a research firm.

LinkedIn is a goldmine.

Take a step back. Microsoft recently invested $26.2 billion in buying LinkedIn. That’s a huge investment to make in a platform with crummy data. It’s safe to assume Microsoft found some value (billions of dollars worth!) in LinkedIn’s data. Don’t let your sister’s cousin’s best friend arbitrarily discredit a great resource for your study.

When we approach market research recruiting via LinkedIn, we hone in on study candidates’ titles, companies, and locations. Just those three pieces of information are quite valuable for identifying qualified respondents. Also, a subscription to LinkedIn Premium is much cheaper than hiring a panel firm or buying a third-party list to get the same information.

One caveat: take the job description section (not the title) with a grain of salt. LinkedIn users supply this information themselves and may play up or omit certain things. Other times, they may have neglected to refresh a dated description.

In our recruitment process, we take the job description section into consideration but we don’t value it as highly as the title, the company, and location. Before you start fretting about dated, obsolete profiles, remember that you’re not conducting a census, you’re drawing a sample. Every profile you look at doesn’t need to be perfect.

Company, title, and location = powerful data.

For example, let’s say you need 30 knowledgeable people to interview for a qualitative study about your rival’s sales process. Search filters for company, title and location may give you a pool of, say, 2,000 profiles to consider. That means that it’s perfectly acceptable- even necessary- to rule out a majority of the candidates. You’ll still end up with more than enough stellar potential respondents. You don’t have to worry that some of those initial 2,000 people may have since changed jobs and not updated their profiles. That’s what screening is for.

From our experience, the title, company, and location data for potential B2B tech study respondents is rarely wrong. It happens maybe 5 percent or less of the time in our recruiting efforts.

There is an obvious reason for this. Lying on your LinkedIn profile is like lying on your resume. Someone is going to find out. When they do, that lie damages your professional standing and employment viability. Most people implicitly understand this. Sure, just like on resumes, many people probably inflate the truth a bit, but no data set is perfect. This is why we take the job description with a grain of salt. It would be much odder to lie about who you work for, where you are and your specific job title. All of that information is rather easy to prove or disprove.

Basically, don’t worry about the occasional rotten apple amid the millions of LinkedIn profiles, there is the whole rest of the orchard! For B2B market research study recruitment, LinkedIn’s data is very high quality. The network has 467 million members, and two or more new members join each second.

A little social savvy goes a long way.

Though LinkedIn is our favorite respondent hunting ground, we also sometimes leverage Twitter to great advantage.

For a recent project, we had to reach individuals who were quite familiar with the product Tableau. To find them, we searched for folks who claimed to be Tableau Zen Masters, a designation given to active members of the Tableau community. Searching for “Tableau Zen Master” in Twitter surfaced nearly a hundred viable candidates for our study.

Another time, we had to identify companies working out of the startup incubators funded by AWS and Google. These incubators provided startups with computing power, cloud services, and office space. The challenge with finding these startups was that the founders and development team members were unlikely to mention anything about their relationship with AWS or Google on their LinkedIn profiles. Further, unlike other providers, Google and Amazon don’t publish a list of the companies currently working out of their incubators.

However, both AWS and Google have Twitter accounts that market their incubators to new startups. With that knowledge, we started looking for Twitter users who were startup founders and were actively following one of the Twitter accounts aligned with an AWS or Google incubator account.  We then checked on LinkedIn to make sure these individuals were still with the startup listed on their Twitter profile. After confirming that the startups were residents of AWS or Google incubators, we just had to reach out to enough of these folks to get a good study sample. Of course, not everyone we reached out to was working out of one of the incubators, but a surprisingly large number were.

You can find them if you know where to look.

We aren’t shy about getting creative to recruit the perfect study respondents. In addition to leveraging social media, we might look at speaker lists for conferences that align with our market research study question. If we’re looking for competitor customers, we may take a gander at the competitor’s case studies. We might even look at websites like SlideShare or individual forums and communities online to mine participants from these locations.

However, that creativity requires extensive industry knowledge of the B2B tech sector. You won’t find that in a panel firm.

Form Letters Have No Game.

Make ‘em feel special.

Let’s talk about outreach. After you identify your dream respondent profile and find a sufficient sample of them, you’ve got to convince them to actually participate in the study.  Market Research Recruiting: How To Get A Superb Sample

Here’s another area where panel firms are likely to make a mess of things. Often, third-party firms specialize in mass recruiting. Their outreach efforts tend to be very templated and rely heavily on form letters. This is no way to woo a busy specialist working in a unique niche of B2B tech. These respondents need to be interested and intrigued before they are willing to take the time to engage. A form letter isn’t going to entice them. Outreach for these prized respondents needs to be more individualized and specific.

In one recent study, we were focused on looking for individuals who could meaningfully comment on app stores. To do this, we looked for professionals who had spoken at major conferences where app stores were the topic of conversation. We also searched for influencers who were regularly sharing content on the subject via blogs and social media.

Once we had identified a pool of qualified respondents, we crafted highly personalized outreach.  We demonstrated that we were specifically interested in speaking to each candidate based on the public information we found about them online.

We say why we want them.

In most of our outreach, we include context for why we’re interested in each potential study participant. We’ll include relevant details from their public LinkedIn or Twitter profiles or information that is readily available via web search. This automatically makes our outreach personalized. It gives the impression that the person we are contacting is not just a fill-in-the-blank on a form letter but an individual specifically desired for their expertise.

For example, let’s say we’re looking for companies that have deployed a certain type of product. We might start by looking for individuals who mentioned that they have experience with that product on their LinkedIn profile. We’ll scan their job descriptions to see if they’ve claimed to have led a business initiative that leveraged that product.

In the outreach, instead of saying something generic like, “Do you have experience with GIS solutions?” we’ll say, “Given you’re responsible for running the GIS department for the city of Denver and you’ve got a team of 12 all focused on GIS concerns, we would love to talk to you about the products you use today to meet business needs.”

Understandably, this kind of outreach is going to get a much better response rate than a generic form letter. To summarize, don’t go the form letter route.

Don’t Rush The Recruit.

The wrong people can’t give the right answers. Market Research Recruiting: How To Get A Superb Sample

In sum, the next time you start a research project, ask yourself whether you know,

  • Who can best provide insight into the business problem.
  • How to find a sufficient sample of qualified respondents.
  • What information to include in the outreach to entice participation in the study.

If the answer is “no” to any of these questions, don’t worry. Send us an email or give us a call.

This podcast is brought to you by Cascade Insights. We specialize in market research and competitive intelligence for B2B technology companies. Our focus allows us to deliver detailed insights that generalist firms simply can’t match. Got a B2B tech sector question? We can help.

Know Thyself: B2B Usability Testing

/0 Comments/in , , , , , , , , , /by

You can’t be unbiased when comparing your company’s offering to your rivals’. We get it, you’re proud of the work you do and the innovations of your team. Of course, your offering is No. 1 in your eyes.

But… you do have to know where you rank. Building effective strategy depends on it. It’s just too important a matter to leave up to wishful thinking.

Luckily, B2B usability testing can give you an objective view of your pain points and competitiveness.

However, designing B2B usability studies requires finesse. You have to have the proper technology, access and, importantly, knowledgeable research participants.  In this episode of B2B Revealed, we brought in MeasuringU Principal Jeff Sauro to explain how usability testing differs in a B2B context.

Know Thyself: B2B Usability Testing

In this episode of B2B Revealed, we brought in MeasuringU Principal Jeff Sauro to explain how usability testing differs in a B2B context.

Listen to the episode…

Or you can read the edited transcript that follows.

Cascade Insights CEO Sean Campbell

Today I have Jeff Sauro with me on the podcast. Jeff is a six-sigma trained statistical analyst and a pioneer in quantifying the user experience. He has authored and co-authored five books including Customer Analytics for Dummies, and Quantifying the User Experience: Practical Statistics for User Research. He’s also the founding principal of MeasuringU. Jeff, welcome to the show.

MeasuringU Principal Jeff Sauro

Thanks for having me, Sean.

Sean

So, Jeff, I reached out to you because you currently own the first spot in Google when someone searches “B2B usability testing.”  The reason for that is that your team put together a blog post on doing usability testing for B2B that is pretty popular.

Before we get into the particulars of B2B usability testing, let’s define some terms.

Walk me through some of the key elements of a usability research effort and the goals of a typical usability study.

Why Study Usability

Jeff 

Like an in-depth interview, a usability test has a one-on-one focus. However, with usability testing, the emphasis is more on observation than on a dialogue. You’re examining attitudes and actions with usability studies. The idea is to see what users do with the system when you’re not around to help.

With this approach, you can discover pain points. You may learn that users have trouble with forms, entry, or general navigation. (These are common issues across B2B enterprise applications.)

Collecting metrics is another important component.  As we all know, if you can’t measure it, you can’t manage it.

In sum, usability tests probe:

  • Usability issues.
  • The current usability of the system.
  • After fixing issues, whether the usability has improved.
  • How usability compares in competing systems.

The Evolution of Usability Tests

Sean

Your website has a really nice infographic on a timeline of usability history. How has usability testing evolved over the last 20 years or so?

Jeff

I think usability really got its prominence in the mid-80s. In that time there was this idea that the usability lab was an extension of Stanley Milgram’s psychological experiments. You had this vision of white lab coats and PHDs conducting studies in antiseptic rooms with really expensive equipment. The perception was that only the IBMs of the world could afford to do usability studies. They didn’t seem as accessible to the emerging enterprise market.

It did start out that way. Usability tests used to require expensive equipment, highly-trained facilitators on location, the proper technology, and the physical space. Also, the participant had to be present.

As we moved toward screen sharing capabilities, things changed.

You could have participants share their screen as if they were in the room with to you. With that, many of the barriers for getting test participants were erased. We were able to take a step away from some of the complex systems previously necessary for conducting usability studies.

Another huge evolution is that usability tests no longer have to be moderated. We gained the ability to give participants a set of tasks as software records their screen. At the same time, we can track how long it takes for them to complete the tasks.

Today, you can collect data from hundreds of participants in just a matter of days. The same data set would have required a huge budget and a lot of time back in the 80s and 90s.

Today, you can collect data from hundreds of participants in just a matter of days. The same data set would have required a huge budget and a lot of time back in the 80s and 90s.

Though over the years, the method of execution has changed somewhat, we still focus on the same core objectives. We watch what people do, hear what their sentiments are, and gain a greater understanding of their attitudes and actions.

Assessing the Competitor

Sean

When you talk about the tools that allow you to crowdsource participants, you’re talking about stuff like usertesting.com, right? I know there are tools that let you inject some JavaScript into your website to record behavior behind the scenes. Others will randomly website visitors whether they’d like to participate in a survey.

Jeff

That’s right, User Testing is an example, as is Userzoom.

With new technology, we are now able to assess the usability of websites of competitors. In other words, you can test websites that you don’t have control over. You can also easily assess applications that are being used on another user’s desktop.

Sean

Let’s talk about assessing applications offered by competing companies.  What are some tools you could recommend for analyzing a competitor’s SaaS product, such as a CRM or an ERP solution?

Jeff

There are a couple of things I can recommend.

Previously, I worked for Oracle. At that time, you had client server apps which were much more complicated to analyze from a usability perspective. Now with the cloud and web-based applications, it’s much easier to record what people are doing. It’s easier now to watch participants complete tasks end-to-end.

The biggest barrier to assessing a competitor SaaS product is just getting through the authentication process. Luckily, there are several ways to do this. You can recruit participants who have accounts or who have the ability to test the product, say if they’re in the market for a CRM solution. They have to login to these sites and/or go through authentication steps, but you’re actually still able to record the screens as people go through and create a newsletter, set up sales contact information, etc.

With usability testing, it’s common to ask people to complete a core task on say three products: yours and two competitors. (It can be the same person or different people completing the task on different products.) Each time, you measure how long it takes to complete the task and the level of difficulty they report through standardized measures. In many cases, you’re able to record the screen that’s in the browser. With that, you have a record of what the participants are looking at.

Basically, this type of usability testing gives you a competitor benchmark.

Sean

You mentioned measuring how long it takes a user to complete a task. I imagine it was easier to measure that when participants were flown out to complete the tasks in a usability lab. How is that measurement taken these days, when usability testing happens remotely?

I understand how the qualitative perspective would be taken; you’re recording the screen and you’re watching their challenges. But if they’re using a competitor’s software, are you just qualitatively tabulating the amount of time between the tasks?

Usability Metrics

Jeff

That comes from looking at efficiency metrics.  That said, there are three key metrics to keep in mind. The three core metrics are:

  • Effectiveness, the ability to complete the core tasks.
  • Efficiency, how long it takes to complete the core tasks.
  • Satisfaction, how users feel about the experience of using the product or service.

You need to have some objective success criteria when you talk about effectiveness. Often, we look for an artifact of whether the task was completed properly. For example, a confirmation screen may have a unique URL. There may be software that can see when users successfully reach this URL. In other cases, we simply ask them: “What was the confirmation message you got after completing this task?”. Again, usability tests are often unmoderated, in which case, we wouldn’t actually be watching them. So one measure of effectiveness is having the participant providing evidence that they have completed the task.

As for efficiency, as soon as we give them a task, and they tell us they’re starting, it’s really easy to start collecting time automatically. From that, we get a sense of how long it takes them to complete the task. You can get the duration per task and the length of time they spend completing the set of tasks.

To measure satisfaction, immediately after they say they’ve finished a task, we ask them how confident they were that they had completed it. For this, we have a standardized measure called the “single ease question.” We ask how easy or difficult it was to complete that task. We then add the response to a database and compare it to other user experiences.

Usability testing really gets some great insights. You may look at a user experience and see that it only took them 30 seconds to complete a task but their attitude about that experience was below average in relation to hundreds of other tasks collected in the database.

Info for the database is collected from survey responses, duration of tasks, and confirmations of having completed the task. Some of this data obviously has to be verified before it is added to the database. In other cases, automating that data collection is appropriate.

Finding Participants for B2B Usability Studies

Sean

Tell me about the recruitment process, because it’s obviously different for a B2B audience than for a B2C audience.

For B2C, you have a very large population to pull from and the ability to quickly build up samples that give you a lot of statistical significance.

That’s not the case with B2B, especially in niche markets. I imagine that recruiting is a little more challenging for B2B.

Jeff

Yeah, absolutely. A lot of successful recruiting for B2B usability studies depends on finding the right partners who are able to provide and source those participants for you.

For example, right now, we’re doing a combination usability market research study for a large B2B tech company. We need to talk to the people who make decisions about backing up data. This is definitely not a general population. To reach these ideal participants, we have to be precise and creative about how we recruit.

Sean

Good points. If you look at some testing platforms, say where they ask you to do a 5-second test on a webpage, your unemployed college student would fit the bill perfectly.

…with complex B2B focused software or services, you need someone who actually has industry insight to participate in the study.

However, with complex B2B focused software or services, you need someone who actually has industry insight to participate in the study. You can’t test effectively if your participants wouldn’t actually use the system or do the tasks you’re studying. B2B companies don’t typically target novice, industry outsiders. Their target users are knowledgeable.

Jeff

Right. Exactly.

What To Watch Out For

Sean

What are some challenges of conducting usability studies?

Jeff

Getting the technology right is a big one.

It’s important that your study participants are able to log in successfully to whatever application you’re asking them to use.  You would be surprised how often this is an issue. You’re probably going to have to talk to someone in the provisioning department and make sure that is set up.

The account also needs to be properly set up after they’ve logged in. You don’t want to mar your study with an atypical experience that requires bizarre setup screens, etc.

Getting the accounts provisioned and properly set up is very important for usability studies, especially when examining a competitive space. Measuring the user experience of a competitor product or service requires the participant to have access to it. You’ll have to think out whether the participant is accessing the product or service through a free trial, a temporary subscription, physical access, etc.

It’s also important to make sure that you are asking participants to complete realistic tasks.

One particular challenge for B2B is that software platforms and services are often heavily customized by each enterprise customer to fit into a particular workflow. Payroll may be set up on one platform, meanwhile HR uses the same platform but has customized it so that it looks very different from what payroll is using. Move over to sales, and they’ve integrated the platform into Salesforce and other accounting software. In each situation, the platform is used differently. Bring all of these people into the lab and, if you’re not careful, you could be asking them to use software that they’re completely unfamiliar with. That doesn’t help your study at all.

Make sure the task you are giving participants is something they normally would do in their role. Otherwise, their user experience is rather irrelevant.

Make sure the task you are giving participants is something they normally would do in their role. Otherwise, their user experience is rather irrelevant.

Usually, you only have an hour or so with each participant. That means that the participants have to adapt to the conditions of the study quite quickly. They have to swiftly get comfortable with the fact that their screens are being watched. You may be across the country or physically sitting in a lab with an eye tracker on, and/or with cameras going.

Participants have to get used to being observed. That has to be factored into the data too. That participants may be taking time to adjust to the study conditions rather than diving right into the task at hand.

What’s next for usability testing?

Sean

What do you think is in store for the future of usability testing? What techniques or methods are on the cusp of being accepted into new common practice?

Jeff

I think we’re going to continue to see more and better refined technological approaches for B2B usability studies. So much of software is now browser-based and can be accessed remotely. It’s going to be easier and easier to observe people using highly specialized systems without physically being there with them, without having to set up some customized tower in our lab that has the software built into it. That’s going to make testing easier. The methods are going to continue to get better.

Right now, B2B is still very much in a time-warp.

The B2B purchaser and user are often separated by a big space and time. The purchase of a B2B solution may be decided on by a team years out, before it gets configured and used. As a consequence, there are frequent basic usability issues that don’t get addressed until much later. That’s why we typically see 2-10 times as many usability issues in B2B as you see for a consumer-focused website or piece of software. You can see why usability testing is really important for B2B.

How Easy Is It To Use You?

Sean

Despite inevitable technological improvements, it’s still important for usability studies to focus on the basics:

  • Who are the users?
  • What are they trying to do?

Then watch them do it. Watch simple things like form fields and how they’re aligned and set up. Watch the labels and navigation. Find out how easy it is to use your product or service, where your pain points are, and how your usability compares to the competition. Time and technology won’t change those basic questions of usability studies or their importance to B2B companies.

This transcript was modified for clarity and readability by Isabel Gautschi.

This interview is brought to you by Cascade Insights. We specialize in market research and competitive intelligence for B2B technology companies. Our focus allows us to deliver detailed insights that generalist firms simply can’t match. Got a B2B tech sector question? We can help.

Image used courtesy of enotmaks/Fotolia

In-Depth Interviews

In-Depth Interviews: The Answer To ‘None of The Above’

/0 Comments/in , , , , , , , , , , , , , , , /by

When it comes to B2B market research and competitive intelligence, there is no better tool than In-Depth Interviews (IDIs) for gathering insight.

In-Depth Interviews: The Answer To ‘None of The Above’

This article is based on an episode of the B2B Market Research Podcast. The audio version is available here.

Surveys have long been a popular tool for corporate research, but they have a significant limitation: they can only reinforce or dispel assumptions you already have.

In-Depth Interviews can uncover context that you weren’t even aware of. As such, IDIs are key to qualitative research. They are even essential to building effective surveys, as they illuminate the best questions to ask.

In-Depth Interviews can bring insight to business decisions from every angle, including:

  • The decision making process for selecting a product or service.
  • The stakeholders involved in the decision.
  • The factors considered before making the purchase.
  • What could have changed the decision.
  • Whether the customer or company is happy with the product or service they chose.

In-Depth Interviews: The Basics

To kick things off, let’s talk about what I mean by “in-depth interviews.”

Length

IDIs are typically 30-60 minute conversations with a business stakeholder.

In-Person Vs. Remote

Historically, IDIs, especially in B2C studies, were usually conducted face-to-face. These days, telephone IDIs are incredibly common.

At Cascade Insights, we even prefer telephone interviews as they tend to allow for more time with the respondent. It’s much easier to grab time on someone’s calendar if you’re just asking for a phone call rather than something that requires a face-to-face meeting.

Respondents

Interview respondents need to be able to provide insight into complex B2B business decisions. Often, studies will require respondents with a certain job title or highly specialized knowledge. To uncover powerful insight, you can’t just talk to anyone. Effective studies require talking to very specific people.

Incentives

To ensure response from the right people, it’s common practice to offer an incentive. Our incentives usually range from $100-200 in value.

Recruit

Finding good interview subjects can be done in a variety of ways. We frequently turn to LinkedIn, industry forums, and social media networks to identify knowledgeable respondents to reach out to.

The Interview

Discussions guides form the foundation of the interviews. These guides are not just a list of questions that are recited verbatim. A typical guide includes key focus areas and specific drill-down questions but leaves room for discovery and investigation during each conversation.

Clients may even influence the course of each discussion by communicating via a back channel with the researcher who is conducting the interview. As the researcher conducts the interview, the client may be in contact with the researcher through an instant messaging platform like Slack, Skype or Google Talk.

Why In-Depth Interviews?

IDIs are useful because they are uniquely able to get at the heart of customers’ decision making process. For B2B technology companies, business decisions tend to be complex and involve many different stakeholders.

To illustrate how useful IDIs are for market research and competitive intelligence, I’ll share some examples of how IDIs have uncovered key information for Cascade Insights’ clients.

The Upstart Competitor

Our client was focused on selling directly to the enterprise and had enjoyed being relatively unchallenged in that market for a period of time.

However, a competitor was moving away from selling to small and medium businesses and started targeting enterprise-sized companies. Back when the competitor had focused on SMBs, they had posed no threat to our client even though their technology was somewhat superior. As the competitor started moving up market, they posed a much greater threat.

Our client needed to know two key things:

  1. Was this competitor truly in a position to sell to the enterprise?
  2. If so, could the competitor be stopped?

To complete this project, we conducted in-depth interviews with:

Each one was able to provide a unique perspective.

Customers were able to tell us how the product was being tailored to meet enterprise needs. They could also shed light on how well the competitor’s sales team was able to engage with enterprise sized organizations.

The competitor’s former sales reps were able to tell us how much of a push there was to target enterprise accounts. They could also clarify which tactics were most often used when targeting enterprise accounts. Importantly, they could also tell us how often they were competing with our client for enterprise deals.

Talking to partners was beneficial because they were able to describe how much of the competitor’s business was pivoting towards the enterprise. Typically, partners have a unique perspective on the markets that their vendor partner is targeting. If your competitor is attempting to move up market, that will probably be reflected in requests to their partners to reshape their sales and marketing efforts. An ambitious competitor may even reward partners who have been successful at changing their strategies to attract enterprise deals.

Key Buying Criteria and In-Depth Interviews

IDIs are also incredibly useful for understanding growing industry segments such as the cybersecurity market.

For example, lately, we’ve had a lot of work researching solutions that protect organizations from viruses, malware, and the like. We’ve spent a lot of time interviewing cybersecurity professionals about the features that matter most in endpoint protection programs.

In such a crowded marketplace, knowing exactly what features cybersecurity professionals want, which ones they aren’t interested in paying for, and what features will matter five years from now are all critically important.

In researching these issues for our clients, we conducted IDIs with network security specialists, network security architects, and cybersecurity thought leaders. We reached the people that could personally answer these questions with the authority of having worked with the solutions in question day in and day out.

Uncensored Customer Feedback

Another big advantage of IDIs is that significant client engagement is possible. All throughout the research effort, clients get to listen in on the interviews and hear how the research process is progressing.

There’s nothing like hearing the voice of your competitor’s customers as they tout the virtues of your opponent’s products and lay out 10 or more reasons why they believe they made a smart decision in choosing your rival.

It’s often the case that companies’ tribal knowledge of their brand’s image is very different from how they are currently perceived. There is nothing like a direct quote from a lost customer when it comes to battling the bias within your own company about the competition.

I can personally attest to scores of times during IDIs when a client was dismayed by the strength of a customer’s advocacy of a competitor.

Meeting Stakeholder Needs and In-Depth Interviews

IDIs don’t just bring hard truths to light. They can also help various stakeholder teams accomplish their goals more effectively.

B2B Sales

Sales leaders often commission win-loss studies. Win-loss studies provide insights on what sales teams are doing most effectively, what they need to stop doing, and what the competitor does better.

Win-loss studies can also help identify the types of buyers that your sales team struggles to engage with and those they excel at selling to.

These studies frequently involve 15-60 minute IDIs with the client’s current customers and with their competitors’ customers.

Sales leaders also appreciate key buying criteria research. This type of research is typically based on IDIs with current customers, competitor customers, and partners.

B2B Marketing

IDIs can also bring insight key to the strategic development of content marketing assets. There are so many “how to do content marketing” resources out there and a plethora of blogs, articles, podcasts, video streams, social campaigns and more from your competitors and partners. With so much out there, it’s hard to know which type of content marketing to invest in.

To create content that has a real impact, you need to conduct real-world research. Certainly, you can look at statistics from your website, blog, and social media efforts, but it’s also important to actually ask current and potential customers what type of content drives them to buy.

For example, one very well-known tech company asked us to study which content drove developers to engage with their cloud service and which content helped educate them on the service’s features. Armed with the knowledge of which content initiatives were resonating the most with customers, our client was able to strategically concentrate their marketing efforts to drive even more engagement with their product.

In-Depth Interviews are also key to building great buyer personas.

One of our clients hired us to give them a greater understanding of the different types of buyers of advanced analytics. The aim of the project was to determine archetypes of different types of buyers from business leaders to tech specialists.

For this project, we conducted roughly 10 IDIs with each type of buyer. Through the IDIs, we learned what motivated each archetype to buy, the type of relationship they preferred with suppliers, the key buying criteria that drives them, and many other factors in their decision to purchase advanced analytics solutions. Obviously, all of this insight greatly improved our client’s marketing and sales efforts.

Product Development

IDIs with customers, potential customers, competitors’ customers and aspirational customers can reveal information that is critical to product development. You may learn:

  • Which features customers wish you had.
  • Which features drove customers to buy a solution; yours or your competitors’.
  • Why customers use a particular feature.

Today, most companies have a lot of data on what their customers are using but they don’t always understand the context around that use. IDIs can clear that up.

IDIs are also great for digging into how customers rate companies on the “itys” of tech: security, manageability, reliability, scalability, interoperability, etc.

Challenges with In-Depth Interviews

Now that we’ve talked at length about the benefits of IDIs, I’ll acknowledge some of the risks of using this methodology.

Mistaken Stakeholders

Stakeholders sometimes push for quantitative measures when the research demands qualitative data.

Many stakeholders know that they can obtain a quantitative view of their own customer base quite easily. With direct access to their customer base, they often assume it will be easy to get a good-sized sample.

However, things change when you need to hear from competitors’ customers. Finding enough of these folks to have a meaningful quant study of their views is less likely, definitely not easy, and most likely expensive. A better first step is to go with a qualitative research effort that includes IDIs with competitor customers.

Needlessly Infinite In-Depth Interviews

Another related challenge is that stakeholders often want more IDIs than are necessary.

Frankly, you don’t need that many IDIs to have an effective study.

If you think about a study that focuses on an average B2B sales process, the sample you need to target is already pretty darn small. Just some basic criteria shrinks the pool down a lot: customers who bought a certain solution from a particular competitive landscape (example: 4-6 vendors) for the purpose of completing a specific job, for instance.

Further, clients typically commission a study to understand a certain type of buyer persona such as a technical or business leader.

Once a study is dialed in this tightly, it’s usually better to conduct a set of 10 -20 IDIs with stellar respondents than to launch an enormous (and expensive!) quant study.

Stakeholders also sometimes fail to recognize the difficulty that exists in targeting the customer base of small startup companies with a quant survey. For example, at least 10 times a year, clients call on us to research a company with a customer base that only numbers in the hundreds. For studies like these, it would be unwise to go with a quant survey. With such a small customer base, and factoring in typical response rates, you won’t be able to get meaningful insights about a startup’s customer base with the number of surveys that would actually get completed.

Luckily, 10-20 IDIs can give you a ton of insight about a customer base that numbers in the 100s.

Recalcitrant Respondents

Interviewees, particularly for B2B studies, often need to be pushed.

For B2B tech market research and competitive intelligence studies, we’re not asking people about sensitive medical conditions, their kids, or their failed dreams. We’re just talking to them about business and technical issues. Hence, it’s fine to push for a meaningful answer that has a strong industry context.

For example, if someone tells one of our analysts that they decided to go with a competitor solely based on price, we are not just going to drop it there. We will ask them to justify that response, maybe even a couple of times, because business decisions are rarely that easy. In B2B tech, there are multiple stakeholders that interact as part of a very complex decision tree. It’s important to get at those secondary and additional factors that drove the decision. Perhaps price played into the decision, but it was really a particular integration or support offering that landed the competitor the deal.

Interviewers That Don’t Get the Jargon

For effective IDIs, the interviewer needs to be nearly as knowledgeable as the research participant.

How often have you been at a party and struggled to explain your day job to someone who isn’t in the same line of work? Instantly, you start dumbing things down so that they can understand what you do day to day. In casual conversation, that’s just fine. However, for B2B market research, the respondent dumbing things down is the last thing you want. Ideally, the respondent should speak to the interviewer like they would speak to a colleague. Only then do you know you’re getting in-depth answers that have real industry context.

The Drive-Thru Interview

Another challenge with IDIs is that you can’t control the interviewee’s environment. If the interviewee is ordering a cheeseburger, they aren’t really paying attention. The same is true if they’re focused on their email as much as they are the phone call you’re having with them.

A skilled interviewer versed in the technical and business aspects of the industry tends to solve this problem as well. As soon as interviewees realize that the person interviewing them really gets the challenges in their industry, they tend to quickly focus on the phone conversation. The interviewer just has to be knowledgeable and personable enough to be more interesting than the subject’s email or cheeseburger.

…Why Do In-Depth Interviews

There is just no better way to solve the puzzle of B2B business decisions and find out what customers really, really want.

This podcast is brought to you by Cascade Insights. We specialize in market research and competitive intelligence for B2B technology companies. Our focus allows us to deliver detailed insights that generalist firms simply can’t match. Got a B2B tech sector question? We can help.

Image used courtesy of AldanNa/Fotolia.

 

Find Your Competitor’s Achilles’ Heel: Post-Sale Questions for Customers

/in , , , , , , , , , , , , , /by

Competitors’ customers are treasure troves of insight. Not only can they tell you how satisfied they are with the competitor’s product or service, they can also clue you in as to why they picked the competitor in the first place.

In other words, you may learn why you missed out and how you can have the winning solution the next time around. Here are a few good questions to ask customers who have been using the competitor’s products for a period of time.

Episode 122: Post-Sale Questions for Competitor CustomersThis article is based on a “B2B Market Research” podcast episode. If you would rather listen to the podcast episode. You can do so below.

How often was the user or organization engaged with after initial sign-up?

The answer to this question shows you whether or not the competing company was high-touch. It can also show you whether the customer felt they were engaged with as much as they would have expected.

Was the transition from sales to implementation smooth?

Some of your competitors are going to be very good at selling and not so good at implementing. This is a potential win-back opportunity for your firm.

Did the implementation go well?

What were the pain points? What went smoother than expected? Again, this can highlight whether your competition is good at selling but not so good at implementing.

Now that the product has been in use for some time, which features are being used more than expected and which are being used less than expected?

The features the customer is enamored with may not be the ones that you expect.

qa

Dig into this question to figure out which features drove customer engagement and which features have grabbed and held the customer’s attention after deployment.

Did third parties influence the customer’s decision to go with the competitor?

Were there partners involved while the application or the service was being deployed and integrations between the new product or service and existing solutions were being established?

How fundamental were those partners to the success of the deployment of your competitor’s product?

How was education handled for the product or service?

Buying is one thing, but having a product or service that you can learn well enough to actually use in a real-world setting is another.

How much support did the competitor offer?

What type of support did the competitor provide? Was it predominantly focused on community support or were paid support vehicles available? This can help you decide what types of support assets you need to invest in.

Which type of support did the customer utilize the most?

Which type of support did the customer find the most useful? You may realize you should consider webinars, “how-to” style content, reference architecture guides, or something completely different.

Was the customer satisfied with the mechanism of payment?

Did the customer feel that they were paying for the product in the way they wanted to? Were they forced to sign a yearly agreement when they would have preferred a monthly agreement? Were there certain tiers of pricing that they didn’t necessarily agree with? Were there areas in which they wished there were more options available?

Did the actual cost of the solution measure up with expectations?

Ask whether the solution began to cost the organization more and more over time. This could happen due to increasing support costs, challenges with integrating the new purchase with existing systems, or even the cost of educating users on how to use the new product or service effectively.

Does the customer foresee a time where they will outgrow the solution now that they’ve had it in production?

What could cause the company to outgrow the solution? The answer to this question could help you predict a win-back opportunity. Even if you have lost that company as a potential customer for the time being, you might have the opportunity to win them back months or years down the road. Factors such as vertical, company size, number of users, etc. could be indicators that it’s time to get your customer back.

Find The Competitor’s Achilles’ Heel

All of these questions are designed to bring insight to:

  • How the customer came to the competitor.
  • How satisfied they are with the competitor’s solution.
  • Whether and when they may be looking elsewhere.

Obviously, having that sort of knowledge of your competitor will give you an edge.

Any one of these questions are guaranteed to generate insight in your next research study. Go out and ask them.

This podcast is brought to you by Cascade Insights. We specialize in market research and competitive intelligence for B2B technology companies. Our focus allows us to deliver detailed insights that generalist firms simply can’t match. Check out our podcast for more episodes and articles on B2B Market Research method.

Image Courtesy of Florin Deloiu/Shutterstock.com

How to Talk To a Software Developer: 23 Questions B2B Market Researchers Should Ask

How to Talk To a Software Developer: 23 Questions B2B Market Researchers Should Ask

/0 Comments/in , , , , , , , /by

B2B Market Research Podcast: How to Talk To a Software Developer: 23 Questions Researchers Should Ask

Did you enjoy this podcast? Let us know! Review the B2B Market Research podcast on iTunes, Stitcher, or TuneIn.

In this podcast episode, Cascade Insights CEO Sean Campbell describes a few of the questions market researchers need to ask when interviewing software developers.

Sean covers:

  • Figuring out how code is managed
  • Determining how new product functionality is released
  • Learning how the development team solves problems

When talking to developers, it’s important to ask the right questions.

In this podcast, we’ll talk about what market researchers and competitive intelligence teams should ask about when analyzing a modern software product engineering effort.

First off, software developers in engineering teams are their own unique breed. It’s their business to be on the cutting edge with the software and technology they create. When you talk with the developers, you need to ask questions that demonstrate you’re informed about what their day-to-day life is like.

So how can you crack the code on how developers and engineering teams do their jobs?

Here are 23 questions you need to ask.

Find out how code is managed

How to Talk To a Software Developer: 23 Questions B2B Market Researchers Should Ask

First off, there are a host of code management questions to dig into. These include:

  • What set of programming tools is the team using to create their product or service?
  • What development methodologies are currently in vogue across the team?
  • How are teams organized?
  • How are project management and cross dependencies handled?
  • What is the process for writing code and building applications?
  • How are related documents like specs and design elements distributed and maintained?
  • What do code reviews entail? Are they formal or informal?
  • In the software development process, what is automated and what requires human intervention?
  • How are code and applications deployed? Who handles deployment?

Determine the process of releasing new product functionality

You’ll also want to figure out how software features are prioritized and the steps for determining which features make it into the final product or service.

Questions could include:

  • How is product functionality tested before it is released?
  • Are teams free to prototype or is the process more rigidly controlled?
  • When it comes to releasing and shipping new code, do smaller teams have a lot of latitude? Or is it a highly coordinated effort?
  • How is design handled? Is it top down or more organic?
  • Does the team use beta test builds, interim builds, etc.?
  • Do they make early, pre-production-ready versions externally available to people?

Learn how the team solves problems

It’s important to know how the software development team solves problems. You can learn a lot from asking:

  • How does the team respond to site outages, product issues, etc.?
  • How does the team deal with technical debt?
  • How do they handle major rewrites?
  • What’s their approach for preventing problems – i.e. SWCC (“Stop Writing Crummy Code”)?
  • How is functionality or services deprecated?
  • How is software monitored in production?
  • What are the best processes for capturing and analyzing customer usage data? This is a critical question, since the user monitoring allowed by services these days is unprecedented.
  • When a feature or lack of a feature is confused with a bug, how does the team handle it? (For example, was the lack of copy / paste in early iOS releases a feature waiting to be implemented or a bug?)

For all of the subjects (Infrastructure, SaaS, Cloud, Cybersecurity, DevOps, etc.) we’ve covered over the years we know that it’s critical that our analysts have the ability to engage with each interviewee in a meaningful way that leads to great insights.

Having the right questions in hand goes along way toward doing just that.

Thanks for listening, and don’t forget to rate the B2B Market Research podcast on iTunes, Stitcher, or TuneIn. If you’d like to suggest a topic or guest for an upcoming episode, send me an email at sean@cascadeinsights.com.

This podcast is brought to you by Cascade Insights. Cascade Insights specializes in market research and competitive intelligence for B2B technology companies. Our specialization allows us to deliver detailed insights that generalist firms simply can’t match. We offer an array of free resources including a blog, ebook, podcast archive and newsletter.