(503) 898-0004 | hello@cascadeinsights.com

Posts

Your Favorite B2B Podcasts of 2017

/0 Comments/in , , , , , , , , , /by

This year our podcast went through a major change. The B2B Market Research Podcast became B2B Revealed. Our focus broadened to a myriad of issues that impact our clients in the B2B technology sector. The result? More interviews, more thought leaders, more insight, and an even better B2B Podcast.

As the end of the year approaches, we decided to take a look back at your favorite episodes of the year.

Your Favorite B2B Market Research Podcasts of 2017

1. Customer Insights: You Need More Than Market Segmentation Data

For this episode, B2B Market Research/B2B Revealed Host Sean Campbell reviewed Clayton Christensen’s “Competing Against Luck: The Story of Innovation and Customer Choice.”

Campbell, also the CEO of Cascade Insights, explains the book’s special relevance to product teams and corporate researchers.

Understanding “jobs to be done” allows companies to build the solutions their target customers actually need and to identify unmet market needs.

Ignore target customers’ “jobs to be done” at your peril. If a rival meets that need before or better than you, you’re ripe for disruption. Market research can uncover your customers’ “jobs to be done” and spot disruption red flags.

2. Big Data Ethics: Math Responsibility

Big data is just a bunch of numbers without analysis. But, that analysis has to have a basis on good math. Unfortunately, many a big, impactful decision in business, technology, and government have a basis on flawed math and bad analysis… Worse, these false arguments give the appearance of being backed by big data. Needless to say, it has never been more important to math responsibly.

In this episode, Campbell discusses Cathy O’Neil’s excellent book “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy” and how it applies to the B2B technology field.

3. The Pre-Flight Checklist for Research Presentations

No one wants to sit through a boring presentation. To capture and hold your audience’s attention, you need to transform yourself into a master storyteller.

Luckily, there is an easy way to make sure your presentation is interesting, effectively communicating your story, and highlighting critical information. Campbell shares a pre-flight checklist to keep your business presentation from crashing and burning. Make sure your audience’s key takeaways are the ones you intended.

4. Market Research Recruiting: How To Get A Superb Sample

Unfortunately, sloppy market research recruiting is the industry standard. But why waste your research budget on a panel firm when they interview respondents that aren’t qualified to answer the study questions?

Campbell explains Cascade Insights’ unique approach to finding study respondents that can truly answer clients’ questions in context.

5. Sales Needs A Say In Strategy

Through interactions with buyers, your sales team is essentially doing qualitative research on target customers every day. So why do so many B2B tech companies overlook their sales team’s experiences as a valuable source of data?

Campbell describes the unfortunate tendency for B2B tech companies to isolate their sales teams. He goes on to explain why it’s in companies’ best interest to give sales a seat at the table when determining strategy.

Your Favorite B2B Revealed Podcasts of 2017

1. Why You Suck at Virtual Presentations

This episode features Roger Courville, the “Michael Jordan of online presentations and virtual classes.” Courville and Campbell discuss how to effectively teach via webinar, captivating audiences, and generating engagement.

2. Creating The Category

Campbell talks mid-market marketing with Matt Ipri, vice president of marketing and business development at Decision Lens. The two discuss how to create a new category when the product is first-of-its-kind.

3. Is Cold Email a Legal Evil?

Campbell takes a hard look at cold email legislation, market research loopholes, and international law. Spoiler alert: the legality of cold email varies from country to country.

He also shares tips for B2B tech companies to keep their cold emails legal, and less evil than the rest of the junk mail flooding your inbox.

4. Sell Like The Avengers

In this episode, Campbell discusses evolutions in B2B sales with Nic Read, founder of SalesLabs, co-founder of RGI and EdX, researcher, author, consultant, and teacher. Together they discuss how old sales habits need to change, and new approaches to consider.

5. The Spy Who Wasn’t – Clarifying Competitive Intelligence

No, it is not corporate espionage. Campbell interviews Competitive Intelligence Expert August Jackson on the strategy, systemization, and ethics of CI. Learn why a strategic pricing analysis program can get you more information than even the competitor’s price book.

Thanks For Listening!

We’ve enjoyed bringing B2B tech topics and thought leaders to our listeners. Here’s to another great year of B2B!

From all of us at Cascade Insights, we wish a happy holiday season and a fantastic 2018 to all of our listeners.

Subscribe to B2B Revealed on iTunes, Soundcloud, Google Play, or Stitcher.

B2B Market Research

AI: More R2-D2 Than General Grievous

/0 Comments/in , , , , /by

What are the most cutting-edge applications of AI for B2B market research? How much busywork can analysts delegate to a bot? Will automated systems replace the need for human analysis?

We (virtually) sent our analysts to the 2017 QRCA Mini-Conference on Artificial Intelligence to find out.

Here are their key takeaways.

Synchronizing AI With Meaningful B2B Market Research

 By Philippe Boutros, Senior Research Analyst

 

NLP + Open-Ended Survey Responses = (Slightly) Less Frustration. 

The “Other (please specify)” box is the bane of market researchers.

The sheer volume of responses can overwhelm even the most dedicated analysts. Unfortunately, valuable insights get lost along the way.

B2B Market Research

AI has a practical application here. It can segment, group, and summarize different types of responses. This saves researchers a lot of time and energy.

However, artificial intelligence can’t understand a study’s corpus of text with the same context as a (human) trained researcher.

Dear Chatbot, Today I Meaningfully Participated In A Study…

Diaries are valuable ways for qualitative researchers to learn how their users think.

Historically, diaries have had their issues. Problems such as user churn, low-value participation, and researchers getting over-involved have been obstacles.

Luckily, chatbots powered by simplistic AIs can remove much of the burden from researchers. These chatbots ask the right questions at the right time and keep users engaged throughout.

AI Gets A Higher EQ 

By Hercules Randolph, Research Analyst 

 

How Am I Feeling, AI? 

Obviously, qualitative research requires an interpretation of interviewees’ feelings. But, with this interpretation comes bias. Consequently, stakeholders are asking for more and more supplemental quantitative data to provide unbiased context.

B2B Market Research

AI may help eliminate human bias in sentiment analysis. (Though it’s debatable whether machine learning has made AI immune to developers’ biases.)

Rising AI platforms can identify patterns in language including emotions, trends, and also positive vs. negative framing. Further, AI can make inferences from passive language, pronouns, and aggressive language.

When considering the application of AI sentiment analysis to B2B market research, it’s important to note that you will get what you put in. Meaning, faulty data leads to bad analysis. Be careful not to stray far from the discussion guide, use poor recordings, etc.

AI Helps Rather Than Replaces Human Research Analysts

By Colleen Clancy, Senior Research Analyst

 

Ramp Up Mass Qual With Chatbots. 

Chatbots are increasingly effective at managing outreach, scheduling, and participant diaries. This makes it easier to process a high volume of respondents. It also frees up your time to focus on research elements that require human analysis.

Bots & Analysts Can Be Market Research Friends. 

B2B Market Research

AI currently supports but does not replace the need for human analysis in B2B market research.

The tools described in this conference have specific applications that could prove very useful to market research analysts.

For example, moderators can now use AI platforms to inform the direction of in-depth interviews in real-time. AI isn’t yet ready to conduct an in-depth interview on its own though.

Eventually, AI might drive these surveys and the analysis that follows much further. However, the day that B2B market research doesn’t rely on human analysis is still a long way out.

This blog post is brought to you by Cascade Insights.

Welcome to the B2B Revealed Podcast

/0 Comments/in , , , , , , /by

Having what amounts to a collective PhD in B2B comes with the responsibility to share some of our knowledge with the world.  We’ve done that for years through The B2B Market Research podcast.

But what if we had more to share?  What if we had more insights about B2B sales, marketing, and product development that were relevant and timely? How could we best share those tips, tactics, and strategies?

Welcome to the B2B Revealed podcast.

Welcome to B2B Revealed

B2B Revealed will:

  • Deal with the real world challenges that B2B sellers face.
  • Help B2B marketers build great marketing strategies and tactics.
  • Bring product leaders to solutions that B2B customers will love.
  • Include more interviews with thought leaders across the B2B landscape
  • Show B2B researchers how to effectively answer key business questions.

Listen to the B2B Revealed podcast on iTunes, Stitcher, Google Play, and Soundcloud.

You Can’t Always Have Quant With Your Qual

You Can’t Always Have Quant With Your Qual

/0 Comments/in , , , , , , , , , /by

In an ideal world, all B2B research would result in a tidy heap of statistics, graphs, and charts all pointing to a clear course of action. Company leaders would glimpse the dramatic numbers and approve strategy adjustments right away.


B2B Revealed – ON: iTunes, Stitcher, and Google Play

But, as we know, fantasy is rarely reality.

This holds true with B2B research. Many B2B tech companies would love to commission sweeping surveys that yield conclusive insights about the market.

Unfortunately, given that B2B tech is a rather niche field, it is often extremely difficult to get an appropriate sample for mathematically responsible conclusions.

Let’s walk through some of the unique challenges of conducting quantitative research for B2B tech.

B2B ≠ B2C

The B2B Buyer’s Journey Is More Complex than B2C

Market research for B2B companies is very different from B2C.

B2B deals are more complex. A CEB study found that an average of 5.4 buyers had to formally approve each B2B purchase. Whereas with B2C, there are usually far fewer buyers involved.

Conventional wisdom says that about five people are necessary to make a wise B2B purchase, as it’s usually a significant spend for the business. Hence, there is much more wooing involved to push through a B2B sale than a B2C one. Often, months or even years are spent cultivating relationships before that big B2B buy.

Small B2B Target Markets Make for Poor Survey Samples.

 You Can't Always Have Quant With Your Qual - B2B ResearchIn many cases, B2B companies target much narrower markets than B2C companies do. A bottle of ketchup can be marketed and sold all over the world. Whereas B2B solutions, from our experience, may have a total addressable market of 10,000 or 100,000 companies or less. This is a huge difference from a B2C product that could legitimately be sold to any US consumer who might walk into a grocery store.

Smaller target markets are one of the main reasons why quantitative studies aren’t always the best approach for understanding B2B business problems.

To prove this out, let’s just consider the math. Say your client gives you a list of 5,000 people they’d like to hear from. According to one sample calculator, you’d need 537 responses for a confidence level of 95 percent and a margin of error of 4 percent.

Response rates being what they are, you’re not likely to achieve that number.

B2B experts know to anticipate low responses. For instance, Kapost wrote of a survey effort,

First, we were ashamed of our response rate. I’m talking a tail-between-the-legs, oh-goodness-that’s-bad kind of reaction. But after speaking with others in our industry, we now know that 1.1%-2.6% is actually quite good.  

Sure, your response rate could be higher than that, but it would have to be MUCH higher to get an appropriate sample.

So, back to our example. Let’s say you get a 2.7 percent response rate. That means you only get to talk to 135 of the target 537 people from the initial 5,000 person list.

It gets worse. Even if you got a 10 percent response rate, you’d still only get to talk to 500 people. Unfortunately, that’s still shy of a reasonable sample for the study.

Surveys Are Limited Tools For Understanding the B2B Buyers’ Journey.

Typically, B2B stakeholders have a lot of questions about the buyers that filled out their surveys. However, they may not realize that surveys aren’t always the best way to understand the B2B buyer’s journey.

Remember, the average B2B sale involves about five buyers. To understand how they reached the decision to purchase, you’d have to hear from each of those five buyers and learn their role in that choice.

Back to our example. Let’s say you get a 2.7 percent response rate on your 5,000 person sample. That means you have 135 responses. Then, say that responses are somehow magically divided equally into five uniform roles in the purchase decision. That means you have roughly 27 responses from each persona. That’s just too small a number for solid analysis.

How To Get It Right For B2B

There are some ways to use quantitative surveys effectively in B2B. They’re not without compromise though.

Up The Sample Size.

First, you could get a larger sample and then generate a larger number of responses.

Frankly, this isn’t always possible. Perhaps the individuals the study targets are in an extremely niche market. Or, the research focuses on a certain country where the solution in question isn’t well known.

There are a million reasons why it could be difficult to get an appropriately sized sample for a decent quantitative study.

Ask The Client To Give You a List of People To Survey.

 You Can't Always Have Quant With Your Qual - B2B ResearchWhile this option could take some of the leg work off market research firms, there is an inherent challenge.

Any list a client provides is likely to be biased, especially if it’s based on a mix of marketing and sales leads. These leads are already predisposed to consider or buy a solution from the client. These folks alone won’t give an accurate portrayal of the client’s position in the marketplace. For that, you’d have to also hear from competitor customers and customers that decided not to buy at all.

Concentrate On Getting More Responses.

You could also put a ton of effort into initial and follow-up outreach to survey respondents to increase the response rate.

While these efforts may get you more responses from a small list of targets, it doesn’t change the fact that the list was small, to begin with.

Improve The Quality Of Your Sample.

Take a page from qualitative research.

Accept the fact that you’re not going to get more responses, and get to know more about who responded to the survey.

This doesn’t allow you to project your findings across a broad population in every way you might like to. However, you can have more confidence in your findings.

Stop Hitting Divide.

Finally, you might have to limit your quant goals.

For example, for those 5,000 targets in our example, if you adjust the acceptable margin of error to 5 percent, you only need 357 responses.

You also might have to stop hitting divide on your calculator. Say a stakeholder wants to project the research findings onto the UK, France, Germany, Russia, and the US. If you don’t have enough responses to meaningfully do that, don’t try. Seriously, don’t try to take 357 responses from 10 different countries and make projections for each individual country. That’s just bad math.

Also, you may have to limit the level of analysis you do on a persona, title, or industry basis. For example, if you’re doing research on key buying criteria, you might be able to figure out how an organization buys and who’s part of a typical buying committee. But, you won’t be able to drill down into how each individual buyer buys or is influenced.

Don’t Ship an Illusion.

Make every effort to avoid presenting faulty data as a firm conclusion.

If the sample for your survey can’t provide an accurate representation of the populations you’re trying to study, you may have to make the tough choice to ditch quant altogether. Otherwise, you’ll be making a bunch of bad decisions on flawed statistics.

Ditching quant can be challenging to propose because senior stakeholders often want information that they know can’t be argued with. They want insights that come with lots of numbers, graphs, and charts.

However, you’re not helping that senior stakeholder if you design a survey that can’t possibly get a sufficient number of responses for good analysis. That’s worse than not giving them quant in the first place.

Luckily, ditching quant doesn’t mean you can’t study the problem. Perhaps you just need to use a qualitative method instead.

Qual To The Rescue!

You Can't Always Have Quant With Your Qual - B2B ResearchQualitative research can often answer B2B questions much more effectively than a survey. It’s a good compromise.While you won’t have the ability to project the findings from a qualitative effort to a broader population, you will be able to answer many questions and have confidence in the results.

You’ll be able deeply investigate key themes, drivers, and motivations.

In addition, you won’t need nearly as large a target list as you would for a quantitative study. For many of our qualitative studies, 12-20 research participants adequately cover a homogenous population.

Also, recruiting success rates are typically higher for qualitative studies. In our experience, as high as 30 percent of the population under review responds. This means that not only do you need fewer respondents in the study, but you can recruit more people from the same list than if you were doing a quant study. Further, your initial list of targets gets to be much smaller as well.

For example, you could tackle five different buyer personas in a study that only involves 50 interviews with a target list of about 500 people. That’s 10 interviews per persona.

It is important to keep in mind, though, that these qualitative approaches are not going to allow you to project your results onto a broader population.

They will give you a great deal more discovery, insight, and investigation than you already have on-hand.

Better yet, it keeps you from using quant in ways that are not only inappropriate but dangerous. You don’t want bad math informing business decisions.

In sum, the next time a stakeholder tries to take quant study beyond its design limits, just say “no.” Chances are, qualitative will get you results both you and your stakeholders can use to inform smart decisions.

This podcast is brought to you by Cascade Insights.

Sales Needs A Say In Strategy

Sales Needs A Say In Strategy

/3 Comments/in , , , , , , , , , , , , , , /by

Your sales team is conducting qualitative research all the time. Talking to them is perhaps the lowest cost qualitative research project you could possibly do. It’s also your best chance to achieve a strong sales strategy in sync with the goals of your organization.

This article is based off a B2B Revealed episode.

You can listen to the episode or read the article below.

B2B Revealed – ON: iTunes, Stitcher, and Google Play

Unfortunately, many companies undervalue their sales professionals. This results in scores of missed opportunities and the alienation of the sales team.

Companies often make two big mistakes:

  1. Companies’ market research teams typically don’t have a great perception of their own sales team.
  2. Sales teams aren’t often invited to participate as major stakeholders in market research studies.

Having worked with many Fortune 1000 clients, I’ve seen this happen a lot. I know who gets invited to the table. Usually, representatives from marketing and product development are invited right away. If the project is important enough, a vice president or C-level executive gets an invite too. Sales leadership is skipped over far too often. That is a huge mistake.

Sellers and market researchers typically have very different personalities.

Sales and market research demand very different approaches to be effective at each job. Sellers and market researchers typically have very different personalities.

Think about how each role would behave in a meeting to discuss a research project. You’re probably picturing the following:

  • The salesperson will talk a lot and the market researcher will ask a lot of questions.
  • The salesperson will push the group to make a decision and the market researcher will counsel patience.
  • The market researcher will rely on data and the salesperson will rely on instinct.

Frankly, there may be an element of truth to these stereotypes. No problem.

Different perspectives are not a bad thing. The whole room will be pushed to consider possibilities they wouldn’t have otherwise.

I’ll give an example of the sales-as-an-afterthought syndrome. On a recent project for a Fortune 200 company, we received a list of stakeholders as the project kicked off. Since this project was large, the stakeholder list was understandably large as well. There were 15 names from all over the business. Product development, the executive team, marketing and, of course, market research team leads, were all represented. Oddly, not a single salesperson, sales manager, director of sales or even a sales VP were on the list.

To make matters worse, this was a project to figure out which opportunities the client should pursue in a market crowded with competitors. It’s hard to imagine that sales leadership wouldn’t have something worthwhile to contribute to the subject of sales strategy. They may have already had the answers to some of the questions that the researchers were after. At the very least, sales leaders undoubtedly had highly relevant questions on the topic and should have been given the opportunity to ask. Further, the outcome of the study was directly pertinent to crafting an effective sales strategy. It made zero sense to have no one to represent sales on the stakeholder list.

There is a false perception that products sell themselves.

This is another big reason why I think invites to sales leaders get “lost in the mail.”

Sellers and market researchers typically have very different personalities.

You see this bias crop up in many Silicon Valley companies. They might say, “We don’t worry about what the competition is doing. We just build great products.” They may proudly boast of their focus on “doing” over selling.

I’ve even seen companies advertise that they don’t have salespeople. They do, but they make up a title like “customer success representative” to call them instead. I don’t buy it.

I guarantee you that every company has someone (or several someones) in sales even if they don’t title them like they are.

I’m sorry, products don’t just sell themselves. Someone has to tell the narrative that connects the product to a set of business problems or opportunities.  Someone has to develop and execute on a sales strategy.

Regrettably, judging from the overwhelming majority of stakeholder lists for studies, it looks as though many-a-market research project completely overlooks this fundamental truth.

The experience of sales leaders is unduly discounted as a valuable source of qualitative data.

The biggest reason why sales teams are not invited to the stakeholder team is that people tend to think that the data salespeople can provide is inherently biased. The experience of sales leaders is unduly discounted as a valuable source of qualitative data.

ALL data is inherently biased.

Download numbers, SaaS service trial counts, website page visits, feature usage, NPS scores… All that data has an agenda too. It’s collected for a specific purpose and intended to measure a specific thing. It doesn’t necessarily bring context along with it. (That’s what market researchers are for.)

In many ways, B2B salespeople are conducting qualitative research every single day that they work for your company.

Your first reaction to this statement may be, “No way. They don’t record the conversations they have. They don’t transcribe them. They certainly don’t code the responses, and they don’t build findings decks.” You’re right. They don’t.

They do have conversations with real and potential customers every single day. With each interaction, salespeople use their personal experience to influence the behavior of buyers. That makes their personal experience an incredibly important factor for sales strategy.

Conversations between salespeople and buyers drive:

  • Customer engagement.
  • Future conversations with potential customers.
  • The actual sales of your company’s product.

B2B sales cycles last months or even years. Sellers engage with a buying team of roughly 5-10 people through what might amount to tens or even hundreds of online, phone, and in-person interactions.

Basically, your sales team has a ton of untapped data on your customers and potential customers.

Each B2B sale is a mini qualitative research project.

What are some ways you can engage with sales to mine this kind of intelligence? How can you infuse your next research effort with sales insights? How can you learn more about your company’s sales strategy? Ask your sales team these three questions.

Can I see a demo?

You haven’t properly researched the product unless you’ve seen it demonstrated.

With B2C software solutions and products, this is usually an easy process. You can go down to the mall and see or buy the product, or read public reviews.

With B2B solutions, it’s different. These products cost tens or hundreds of thousands of dollars, so you can’t just buy them yourself.

For some solutions, like technology products in B2B, you can’t be expected to fully understand the product unless you’re trained on it. Just like the people who are going to use this solution, you can’t be expected to just pick it up and run with it. A demo is the easiest way to get up to speed on a complex product. Simply ask for one.

Salespeople demo the product for real customers every single day. Asking for an hour of their time is a small price to pay to conduct effective research on the product or solution and its competitors.

Who do you sell to and who do you avoid?

So often, market research targets personas that have been in play for so long that they are no longer questioned.

The sales team knows what’s really working. They know the personas who bite and the personas who don’t.

Ask them who they go for first at the start of a sales cycle by:

  • Role.
  • Title.
  • Type of company.

This will tell you at least one of the buyer personas you need to target in your research efforts.

Then ask them the same questions about the big sales. The ones that take longer and require more effort but would provide more long-term benefits. You’ll surface additional buyer personas or even whole market segments that need to be considered.

Make sure to also ask the crucial question of who they never try to sell to by role, title and type of organization. You’ll get a sense of which customers are actively buying competing solutions or aren’t buying any solutions.

Pitch Me

The third thing you want to do is ask your sales team to share their pitch.

In complex B2B sales, there’s no script that a seller reads. It’s not a standardized process.

Your best salespeople hone their pitch using the positive and negative feedback they receive from talking to prospective and current customers. If they highlight certain features, you can guarantee there’s a data-driven reason for that. If they talk up pricing in a certain way, there’s a data-driven reason for that too. If they specifically comment on competitors, there is a reason why.

Inviting sales to the stakeholder team should be common practice.  It’s in your company’s best interest to make it mandatory. You’re guaranteed to get insights that will help shape your study in positive ways. You’re also likely to make all the other stakeholders in marketing, product development, and executive teams happier with the results. Not mention sharpen your sales strategy.

This podcast is brought to you by Cascade Insights. We specialize in market research and competitive intelligence for B2B technology companies. Our focus allows us to deliver detailed insights that generalist firms simply can’t match. Got a B2B tech sector question? We can help.

Qual: There's an App For That

Qual: There’s An App For That

/1 Comment/in , , , , , , , , , /by

Will smartphones transform qualitative research methodology? Get an inside view from an expert who knows qual via smartphone inside and out.

In this episode of the B2B Market Research Podcast, Cascade Insights CEO Sean Campbell interviews Over The Shoulder Co-Founder Ross McLean. You can listen to the podcast or read an edited transcript below.

This article is based off a B2B Revealed episode.

You can listen to the episode or read the article below.

B2B Revealed – ON: iTunes, Stitcher, and Google Play

The following transcript has been modified for clarity and readability.

Sean

On this episode of the B2B market research podcast, I have with me Ross McLean, co-founder and president of Over The Shoulder.

Qual Via Smartphone

Sean

Ross, welcome to the podcast. Why don’t you give the listeners a summary of yourself and your company?

Ross

Sure. At Over The Shoulder, we do smartphone-based qualitative research.

Here is a quick background on the company. We were a bunch of qualitative researchers, innovation strategists, and ethnographers who did a lot of traditional qualitative research.

About nine years ago, we got really excited about smartphones.

In doing traditional qualitative, we had always suspected that speaking to people after a decision was made would not lead to the best insights. People don’t always do a great job of remembering the factors that led them to make a certain decision. They may not recall how they felt at a certain point in their pathway to purchase.

With smartphones, we saw an opportunity to reach people before their decisions get made.

We knew that smartphones would spend a lot of time in the pockets and purses of the people that we wanted to understand better. We said, “Gosh, that’s going to allow us to be there in the moments we’re missing out on right now.”

However, nine years ago, there just wasn’t any good software solutions to interact with participants in a project through their smartphones. Luckily, we realized we were just techy enough, we’d researched enough user experience, and we had enough software design experience to build such a solution ourselves.

We built a platform and did, I think, more than 400 qualitative research projects through smartphones over about three and a half years.

We’ve learned a lot and are very pleased with the results. We bumped into every sharp object out there and learned how to work around them from both an operational and software perspective. We gained a lot of nuance on how to ask questions and build refinements to do qualitative on the smartphone. Everything we learned helped us to build a seamless platform for qualitative research via smartphone.

About three years ago, as a group, the broad base of market research practitioners started to really take interest in smartphone based qualitative. We said, “OK, we’ve got a terrific software platform to enable this. We’ve got people who are great at designing projects in this space. We’ve got a help desk, community management, and operations stuff that we’ve been doing this for. Let’s stop doing direct, for-client projects. Instead, let’s make this a company that allows all qualitative practitioners and insight seekers to add smartphone qualitative to their tool kit. Let’s enable them to do that really well.”

Anyone who has strategic and qualitative skills can now work with us and execute smartphone-based projects really easily. Also, with us, they’ll have best-in-industry standards the first time out of the gate.

Participant Experience

Sean

What does the technology stack or infrastructure look like? What does a research participant see?

Ross

In terms of technology, it’s a native app for Android and iPhone.

Some of the most important things we consider for a positive participant experience are:

  • Ease of use.
  • Visual appeal.
  • Affirmation of answers.
  • Participant management.

You have to make it easy and even fun for people to pop out their smartphone and tell you what they’re experiencing right then and there.

If participants have a great user experience, you’re going to get great stuff.

We’ve spent nearly eight years trying to make the participant experience as smooth as possible.

This is how it works:

  • The app is easy to download.
  • Participants log in one time.
  • Participants are given a briefing on how to use the app and what is expected of them during the project.
  • Instructions are given for photo, audio and/or video capture. These instructions are designed to make it very easy for participants to use the app. We’ve got it to where if we want people capturing video, they do it within the app using their native video recorder. With the Over the Shoulder app, they literally just hit a button. They see the question that they’re being asked in the viewfinder of their camera or video recorder so that it’s there for them to reference and remember. They just hit a record button to answer.
  • When they are finished with that assignment, they hit “I’m done.” All of the uploads are managed in the background for them. They don’t have to leave the app to go and take a photo using their smartphone’s camera and then come back to the home screen, then back to the app, etc. We took all of that out, to make it smooth and intuitive for participants.

Researcher Experience

The other area we put a lot of focus on is the analyst experience.

You get a ton of material when you’re asking people to pull out their smartphone and do an assignment which may involve audios, visuals, and photos, a choice every time they interact with a piece of software, or a response when they have a moment of frustration with a system or tool.

This generates a lot of data. We had to make it extremely efficient to consume, transcribe or translate, and tag it up so that the researcher can find stuff that matters quickly.

What we’ve learned is that for analysts to have a positive experience, the app needs to present information in a way that is quick and efficient. We also had to provide tools that allow researchers to approach their analysis the way that the qualitative mind approaches it.

Since we already were qual practitioners and spent a lot of time talking to other qual practitioners, we had a good understanding of the nuances of how qualitative analysis gets done.

We wanted to build the digital version of the way people were already doing qualitative analysis and give it super powers.

It has always been important to keep the app intuitive and simple for anyone who knows qualitative insight and strategy.

Sean

What does the experience look like on the backend for qual researchers?

Looking through that much video, audio, and multimedia must be time-consuming not only for the researcher but, I imagine, for the client as well. They’re obviously not going to sit and watch 10-20 hours of video.

Ross

Absolutely not. And sometimes even expecting them to spend an hour on it is generous. Instead, researchers often need the five-minute video capture that will convey the story and the insights that will get people interested to hear more. That’s the level of distillation you need to be able to get down to.

To do proper qualitative research, you’ve got to have someone who’s reviewing. A lot of the insights come from things that are non-verbal, so it was important to us to have someone looking at everything.

A participant may show you through their body language, “Hey, this piece of machinery is interrupting my workflow because I’m trying to operate it with my left hand and my right hand has to be doing something else.” That would indicate that the machine needs to be designed in a different way. You need to be able to see that, not just read a transcript of it.

It was hugely important for us to make the digestion of information through Over The Shoulder as efficient as possible.

The reviewer can theme, tag, and rate the qual data in real-time when watching or listening. That way, you can easily point to specific instances if asked.

This allows for several layers of analysis. Here are a few examples. You could only show videos that:

  • Have been rated 3-4 stars for quality.
  • Deal with themes related to usability.
  • Have been tagged as of particular interest to a certain client.

Also, it’s easy to share this information quickly. You can download it to include in a report, or edit it into a video presentation, etc.

When you’re dealing with smartphones, everything is coming back in different formats and different codecs depending on each participant’s device. We have to re-encode it and, obviously, secure and encrypt it while it’s in transfer because a lot of this stuff is highly confidential. However, we also make it readily accessible, in standardized formats, and easy to edit.

The B2B Benefit

Sean

Tell me about how Over The Shoulder is used for B2B research. What are some of the more interesting examples you’ve seen of how the solution could be used in a B2B environment?

Ross

So roughly one in five of the practitioners we enable to do smartphone-based qual is using it in a B2B context.

Most researchers use our platform to better understand:

  • How big decisions get made in the moment.
  • The pain points of software in daily use.

The studies may have to be designed a little differently, but we have people looking into things like how to use mobile apps in a particular context, such as time or project management. These are all fantastic things to be able to do with smartphone-based qualitative, because those are things, that, by definition, happen in the moment.

Say you’re working on a mapping system that will be used by a sales organization or a bunch of technicians who will use certain mobile software to connect to that mapping system. To give two quick examples of the insights Over The Shoulder could bring to this scenario, our app could show:

  • When and how people use maps on their smartphones.
  • Frustrations users have with using different mapping solutions.

Over The Shoulder allows this feedback in the moment. Whereas with an interview, you’d have to ask people to remember the stuff they did earlier in the week or even further back. No one has perfect recall, so information derived from memory alone will always be slightly flawed.

With our platform, you can say, “Hit this button every time you are interacting with a piece of mapping software, tell us what you were trying to do, tell us what this piece of software did really well, and tell us the things you wish it could be doing.”

I’ll give another example of the benefit of smartphone-based qualitative research in a B2B scenario. Say you work in logistics in a shipping company. Your customer is interacting with some of your software or your competitor’s software, but they’re doing it seven times a day at totally random times. Over The Shoulder lets you tell them, “Every time you need to ship something, we want you to hit this button on your smartphone. Tell us the situation that you’re facing, the solution that you’re using, why you have picked this solution over the alternatives, how your chosen solution is working out for you, and how it could be improved.”

Privacy & The Platform

Sean

Let’s talk about the compliance and privacy concerns in a B2B context. That’s a problem that sometimes crops up even when you’re dealing with in-depth interviews. You have to promise anonymity to the respondents. Potential interview subjects will often say, “Well, I have some important concerns here about compliance and privacy, so I can’t have a conversation on that subject, etc.”

I imagine the minute you start asking to take video in certain environments like healthcare, or inside a data center, or something like that, you can end up with people just not willing to participate due to privacy concerns.

How often have you seen that become an issue? How do you mitigate those concerns?

Ross

Well, we help our practitioner clients do a ton of pharma work and that sort of sets the bar for personally identifiable standards and security.

We had to build a lot of encryption, monitoring and privacy protection into our system to be able to do that work. Obviously, that’s very helpful when you’re dealing with strategies and systems that are proprietary and confidential.

With data centers or manufacturing procedures, on the factory floor, you’re typically not allowed to be recording visual images. However, there are ways for people to interact that don’t capture video and photo. The person can describe the pain points of the process that they’re using with a recording of their voice, a text response, or just a multiple choice response.

Sometimes, we have to design very carefully to actively ensure not to capture stuff that is a breach of any kind of privacy protocol. There are ways you can design studies to make sure that doesn’t happen.

You can also have it so that all the information collected is reviewed for quality and any potential PII or security breaches by somebody who knows what is and isn’t out-of-bounds before it is submitted. That way, say if a doctor or nurse inadvertently takes a picture that captures a medical record with personally identifiable information on it, that image can be hard deleted out of the system before anyone else is going to see it.

You can take extreme security measures, but more than anything it’s just being transparent with the people who are participating. Tell them what you want them to capture and what you want them not to capture.  Make sure they understand that they are not being asked to operate in restricted areas or share information that is outside of their privacy policies.

You can say, “We want you to acknowledge that we do not want you to share this type of information. Sign your name here.” They can add a signature to make sure this is understood right within the app as they are doing the tutorial.

Clarity is important, but you can also make it very hard for them to capture confidential information that you’d have to worry about in any way.

Does Big Data Replace The Need For Qual?

Sean

All fair points. It’s great to see you have a bunch of ways to gather information in sensitive environments.

big-data

You had a recent blog post about how big data misses the “why.” I completely agree with the post. Why don’t you summarize the message and your motivation for posting it on the Over The Shoulder website?

Ross

Several years ago, we wondered if big data and live capture behavioral data would be a threat to qualitative research- smartphone qual in particular. We thought that big data may have had the potential to answer some of the same questions. However, as we started to interact with companies that had a lot of behavioral information and big data at their disposal, what we found was really just the opposite.

Big data does an incredible job of capturing real behavior. It’s great for identifying whether there is something to pay attention to in terms of website interaction, social media topics of relevance, etc. Big data points you to the places you need to look into. It’s saying something’s going on here. There’s a quantifiable behavioral thing happening in this specific area.

Big data can point out what’s happening, but it doesn’t necessarily explain why it is happening.

Big data was much more complimentary to smartphone qualitative research than we had feared. In a way, big data points out qualitative objectives for you really well.

Smartphone qual then lets you understand with pictures, audio, and video what the human experience of this observable, behavioral phenomenon is. That’s what points people to understanding how to solve pain points.

Sean

I completely agree.

One of our larger clients had an employee who gave a talk at an industry conference. This particular client of ours owns a lot of very well known SaaS solutions. The individual giving the talk, a director of project management, I believe, was telling the audience, “I have all the telemetry in the world. I have way more telemetry than I had when the solution was on premise. I have all this data on who clicked what part of the interface, how long they spent on a given part of the interface, how much time they spent with the solution all up, etc..” His next slide was, “But we have no idea why they spent the time they did.”

Ross

Exactly. Problems come up when you don’t have the context around a decision.

For example, say we build into our website something that prompts you to answer a specific type of question. People click on it once and then they never come back. What did we get wrong in this scenario? Those are things that big data can’t necessarily answer. Big data does an amazing job of helping us realize, “Here’s something that’s not working the way we thought it would at all. We need to go and understand that.” Then smartphone qual can step in to explain the reasons behind the behaviors.

Sean

Thank you for being on the podcast and best of luck to you guys with the solution. It sounds really well crafted. If folks want to check it out, they can visit your website at overtheshoulder.com.

This podcast is brought to you by Cascade Insights. We specialize in market research and competitive intelligence for B2B technology companies. Our focus allows us to deliver detailed insights that generalist firms simply can’t match. Got a B2B tech sector question? We can help.

Riveting Research: It’s Not a Mission Impossible

/0 Comments/in , , , , , , , /by

There is no excuse for a dry market research readout.

Documentary film-making can teach qualitative researchers about the art of crafting a great story, leveraging in-depth interviews for all they’re worth, and building findings that make clients act.

Of course, not every documentary is great. For every Ken Burns there are literally hundreds of one-star documentaries on Amazon. Don’t worry, we’re not asking you to take tips from duds.

We’re going more for the edge-of-your-seat, riveting variety like Man On Wire, Harlan County, Audrie & Daisy, Hoop Dreams, The War Room, etc.

Riveting Research: It’s Not a Mission Impossible
This article is based off a B2B Revealed episode. You can listen to the episode or read the article below.


B2B Revealed Podcast – ON: iTunes, Stitcher, and Google Play

We want to emulate the type of documentary that keeps you fascinated until the very end, leaves you re-evaluating your view of the world, and inspires actions to make it better. Wouldn’t it be great if your business stakeholders had that reaction to your research?

Documentaries and qualitative research have more in common than you may think. Great documentary films are typically based on scores of interviews, just like a good qualitative research study. With B2B, in-depth interviews are often the most effective means of gathering worthwhile research data.

Documentary filmmakers basically have the same goals as qualitative researchers:

  • To report on what they see and observe.
  • Inspire actions based on those observations.

There is a lot we can learn from great documentary filmmakers about creating a compelling narrative.

It’s All In The Name

Take, for instance, the title. The average qualitative research presentation doesn’t have as evocative a name as the “The War Room,” “Man On Wire,” or “Hoop Dreams.”

They’re more likely to be named after the proposal the vendor originally created, or the name of an RFP, or, worse yet, the name of a Purchase Order. Sad, but true.

Next time you’re getting ready to title a presentation, focus on the tension in your story and turn it into a title.  Here are a few examples.

  • “[Company Name]: An Upstart to Watch”: for when your study reveals your company or client is losing to an emerging competitor.
  • “Missing Features That Matter”: for when your research reveals that product development should shift their focus.
  • “Post-Purchase Remorse”: for when interview subjects discuss implementation issues or dissatisfaction.

Title aside, there is a lot that corporate researchers can learn from great documentary filmmakers.

Advice From Great Documentarians

First, we’ll turn to Michael Moore. Michael Moore has produced tons of documentaries over the years including Roger & Me and Sicko.

In a recent interview, he mentioned 13 rules for great documentaries. I’ll share a few of his suggestions and how to apply them to research.

Say Something New

“Don’t tell me shit I already know,” Michael Moore writes. He goes on,

I don’t go to those kinds of documentaries, the ones that think I’m ignorant. Don’t tell me that nuclear power is bad. I know it’s bad. I’m not going to give up two hours of my life to have you tell me it’s bad. All right? Seriously, “I don’t want to hear anything I already know. I don’t like watching a movie where the filmmakers obviously think they’re the first people to discover something might be wrong with genetically modified foods. You think that you’re the only one who knows that? Your failure to trust that there are quite a few smart people out there is the reason people are not going to come see your documentary. Oh, I see — you made the movie because there are so many people who DON’T know about genetically modified foods. And you’re right. There are. And they just can’t wait to give up their Saturday to learn about it.

This rings so true for market research presentations too. How often have you listened to a readout and realized that the vendor isn’t telling you anything you didn’t already know?  Or that a researcher doesn’t really understand the customers your company is targeting?

A research presentation that is just a regurgitation of established facts is cringe worthy. Don’t go there.

Tell the people in the room something they don’t already know.

It’s more likely that you’ll be able to do this if you have picked a focus: an industry or method that you work in exclusively. That way, over time, you’ll get better and better at crafting data-driven stories that resonate.

We’ve chosen to focus our research exclusively on B2B technology companies. For that reason, we know the space better than any generalist out there.  We know what is already held as common knowledge, so we’re not going to suggest the obvious. Instead, we’re able to share something that’s truly insightful.

We’re delighted to frequently get feedback from our clients that we have given them an insight they haven’t heard before or that we thought to ask a question of an interviewee that wouldn’t have occurred to our stakeholders, etc.

These new questions and new insights should hold places of honor and focus in the presentation.

Entertainment Is Not A *&!% Word

OK, so maybe your day-to-day life as a researcher isn’t filled with laugh tracks and emotional cut scenes. But your presentation sure should be.

Riveting Market Research Can Be Entertaining

“Entertainment” should not be a forbidden word in the world of corporate research. © Scanrail / Fotolia

Moore writes,

Yes, repeat after me, they want to be entertained! If you can’t accept that you are an entertainer with your truth, then please get out of the business.

I couldn’t agree with him more.

Personally, I’m a huge fan of nonfiction books. Sure, some nonfiction books are boring. However, there are some fantastic nonfiction books with all the makings of ground-breaking, imagination capturing, must-see TV.

The obvious difference between the two is narrative. One is just data, the other is a story.

You can make research findings interesting regardless of the subject matter.

The key point here to remember, “you are an entertainer with your truth.”

There isn’t anything wrong with being entertaining so long as the entertainment doesn’t detract or alter the truth. Some of the best comedy is truth telling in disguise. Does this mean you should include a laugh track in your presentation? No. But it does mean that you can frame your findings in a way that is interesting enough to inspire insights and actions.

Talk To People Who Don’t Like You

I especially like Moore’s advice to,

As much as possible, try to film only the people who disagree with you.

This should ring true for most market researchers.  Some of the most valuable market research insights come from customers who love your competitor.

So often our clients say in a surprised tone, “Those competitor customers sound rational as they described their decision to go with a competitor.” Well, yes, those competitor customers were rational. Unfortunately, you’d never know that unless you talk to them.

This is also a reason why organizations focused solely on customer satisfaction scores often miss out on what’s really driving the market. They never hear a contrary voice. They won’t hear from a customer who is super happy and satisfied with a competitor. They won’t know if potential customers think that no solution currently in the marketplace is adequate. Only talking to sympathetic voices makes you miss a lot.

Who Gets The Spotlight?

We’ll take our next tips from a Filmmaker Magazine article titled “Expert Tips For First-time Documentary Filmmakers”. This one comes from Nick Berardini, the director of “Killing Them Safely.”

Understand the point of view you want to represent. It’s a bit different than saying ‘understand your story’ because the point of view you’re representing is coming directly from your primary character/characters. More than just what story do you want to tell, whose story do you want to tell, and how does the way you interpret their story filter who your characters really are? You should be an expert in all your characters’ voices, but really have a clear understanding about the way your main character sees things, and don’t neglect them. Everyone else is reactionary to that voice.

Your customers aren't the only ones that deserve a spotlight in the study. © pixelalex / Fotolia

Your customers aren’t the only ones that deserve a spotlight in the study. © pixelalex / Fotolia

So often, researchers consider that their only job is to get the information right. They do need to get the information right, but that’s just a starting point. You need to build toward a bigger story that can motivate a company to act.

It’s also important to get all the perspectives and voices you need into the research study. Don’t just focus on current customers. Also include:

After gathering sufficient perspectives, you can put your main character, your client, into full context.

Invariably, our clients find these other perspectives equally or even more interesting than the conversations with their own customers. At that point, we can return to describing how our client fits within the broader landscape of insights.

The biggest reason why research findings presentations fail to motivate companies to act is the lack of complete relevant industry context.

A Presentation is More Than Its Slides

Next, we’re going to turn to some tips from Ken Burns based on an interview he did for Screencraft.

Burns writes,

1 + 1 = 3.

Over his career, Burns has been able to take dry historical subjects and turn them into more than the sum of their parts.   Some examples include Ken’s series on the National Parks or his documentary on the Dust Bowl.

For market researchers, we need to arrive at something that is more than just raw data. Take all those interviews or survey responses and build a story. Provide a set of actions for your clients to take based on the narrative you’ve given them.

Make It a Nail-Biter

Burns says,

Stories Can Wake the Dead.

I think Burns has managed to achieve this for several topics such as baseball and the Civil War.

Great market research can do the same for companies. It can rattle leadership out of outdated thinking. It can warn of a new threat. It can illuminate a new opportunity. But only if the information driving these insights is presented in a way that’s worth listening to.

In closing, I want to share this quote from Eugene Jarecki, an award-winning filmmaker. He said, “One of the greatest things a person can do is change his mind.”

We need to be doing this with our research. Our presentations should introduce our clients to new perspectives and offer a path forward.

This podcast is brought to you by Cascade Insights. We specialize in market research and competitive intelligence for B2B technology companies. Our focus allows us to deliver detailed insights that generalist firms simply can’t match. Got a B2B tech sector question? We can help.

Header image used courtesy of  doomu / Fotolia.