How to Get Insight from Consumer Panel Surveys: 10 Tips

Alongside focus groups and ad-hoc questionnaires, consumer panel surveys are a popular tool for businesses looking to gain insight into what their audiences – and consumers at large – are thinking.

Using pre-recruited and pre-screened groups (or panels) of respondents, consumer panel surveys have specific advantages over alternative consumer research methods. You can control your audience, for example, and return to the panel over time to collect data repeatedly from the same set of people. This lets you dig deep into highly targeted audiences – and, crucially, compare changes in attitude over time.

Yet, as with all research methods, to get real value from consumer panel surveys, you need to refine your technique. That’s why, in this article, we’re sharing 10 tips and best practices to get the most from your consumer panel surveys. We’re talking how to shrewdly recruit and screen respondents, how to ask questions that elicit authentic opinion, and how to use the right analytical methods to yield deep insight.

Note: Don’t forget, Pollfish has created a new consumer research methodology that works a little differently. One that fulfills the promise of consumer panel surveys, while keeping bad data to a minimum – and making access to deep insights easier and more efficient. 

Where to Start with Consumer Panel Surveys?

Consumer research can deliver highly valuable insight for your business. Whether you want to test how an ad campaign will land, or get feedback on a new product line, opinion from authentic consumers – and potential customers – can be priceless.

Yet, you can’t just go and fire questions around at random. This won’t tell you anything of value. Rather, thinking through who you are asking and why is crucial before you start. 

Here are three tips on how to get that done.

#1: Gain Clarity on Your Research Problem and Goal

We’re starting with the basics, yes. But a true understanding of what you want from your research is an essential element that too often goes overlooked, particularly by businesses conducting research themselves. If you want insights that have value for you, you need absolute clarity on your larger goals.

Do you want to boost sales, refine a specific product, or determine brand positioning? A robust research objective will help you sculpt questions that deliver sharper insights – and save you time asking questions that don’t deliver value.

#2: Refine Your Research’s Scope and Scale

Once armed with a research objective, you now need to assess the desired scale of your research. This can get a little complex – but it ultimately shapes the reliability of your research.

Of course, the ideal survey would consult 100% of your target population – whether that’s marketing graduates in Scotland or cyclists among your company staff. While this scale would yield data on which you can surely rely, this is usually not possible in practice. Even if your target population is very small.

However, the further from 100% of the population, the greater the likelihood of inaccuracy. What you need to decide as a business is what margin of error you can tolerate. If you are making high-stakes decisions, you want data you can trust – and the larger the sample size the better.

#3 Source and Speak to the Right People

For authentic insights that deliver value for you, speak to the right audience. You can’t take this for granted. Before you conduct any survey, you need to recruit respondents that suit your target demographic.

This is what screening questions are for. These filter out respondents who aren’t right for you. However, as our resident expert, Jim Theodoropoulos, notes, using screening questions can be difficult to get right. These need to be combined with demographic targeting to narrow in on your audience.

Using Pollfish, for example, if you are seeking mothers who are runners, you can use the “gender” and “number of children” filters, before asking a screening question about preferred exercise. Like this, you can access deeper insights into specific audiences – and keep the professional panelists to a minimum.

Asking Questions that Deliver Real Value

The way you ask questions – and the form in which you receive answers – affects the quality of your data. And avoiding bias and panel fatigue, for example, can make conducting consumer panel surveys a bit of a minefield.

This is why many businesses use options like Pollfish to help design the questions. But if you’re going it alone, below are some tips on how to ask good survey questions and get answers you can trust. 

Note: For more on this, check out our article on how to ask good survey questions.

#4 Ensure Neutrality

Bias is a bad look in consumer research. For surveys to yield data that is actually valuable, they need to tap into the authentic opinion of respondents. Not reflect the prejudices of the researchers. 

That means avoiding leading questions, those questions that direct respondents to specific answers. For example, “How would you rate our award-winning customer service?” has the potential to skew responses by including the qualifier “award-winning”. 

However, even “what problems do you see in our customer service?” distorts the insights you will receive – as it assumes problems, whereas none may have been seen by the respondent.

This also means providing multiple-choice responses that reflect the range of opinion. If “how would you rate our customer service?” is your neutral question, the possible responses need to achieve similar neutrality. “Extremely helpful / Helpful / Neither helpful or unhelpful / Unhelpful / Extremely unhelpful”, for example, keeps the balance between positive and negative options while keeping framing consistent.

#5 Use Strategies to Mitigate Panel Fatigue

During surveys, and particularly during panel surveys, it’s not unlikely that respondents will become bored or disengaged – even though they sign up to participate.

When respondents are asked too many questions, we call this panel fatigue. It’s one of the biggest downsides to consumer panel surveys. Respondents might lean on “don’t know” or they might start “straightlining” – where they choose the same response for every question.

To avoid this, change up your question formats and shuffle questions around. It boosts engagement. It ensures more trustworthy responses. And it ups your completion rate – making data more reliable overall.

#6 Be Specific – and Remember Your Research Goal

Every question in your survey should provide an insight into your audience’s opinions, attitudes, and behaviors. Those that are irrelevant or too vague are not going to deliver the insights you need to make effective business decisions.

In this way, if you are a gaming app, an open question like “what do you like to do in your downtime?” is not focused enough – and it doesn’t direct respondents’ thoughts to gaming. As such, it won’t provide data that will be of use to your research goal.

Understanding Insights from Consumer Panel Research

You’ve asked the questions. Now, you need to understand the responses. 

This will first mean collating recurring answers, identifying patterns, and highlighting anomalies – a process that can be hugely time-consuming if you are doing it alone. That’s why Pollfish, by the way, delivers automated analytics in real-time. More on that later.

Once you’ve arranged the data, you need to analyze it. Here’s how to get started.

#7 Compare Your Data to Larger Trends

Numbers, let’s remember, are all relative. And getting valuable insights from them is difficult when they are viewed in isolation.

Your survey tells you 80% of respondents said that they were happy with your customer service. But for this to be valuable to you, this needs to be put in context. How does this compare, for example, to other businesses of your size in your industry? Or to your results from last year?

Larger trends are your friends – and they help make sense of your customer panel survey. If this is your first survey of its kind, let these results be the benchmark from which you understand future developments. If you have numbers from previous surveys, use these to assess your progress.

#8 Cross-Tabulate – and Dig Deep

Similarly, when you are reading your results, the big picture isn’t always the most useful view.

Say that 65% of respondents in a general population survey said they found your new ad campaign funny or very funny. That might feel like a win. However, these numbers could be disguising insights of more specific value.

That’s what cross-tabulation is about. Using crosstabs can let you see how different data fields overlap – and this is an indispensable tool for analyzing your data. For example, maybe the ad landed much less effectively with women. Or the joke was completely lost on 21-30-year olds. 

This matters. There may be an insight you need to address. You just need to dig deeper to find it.

#9 Keep on Researching

In a world in which business competition is sharp, a single survey won’t do. As a result, there’s one tip that needs to be stressed: keep returning to your audience to do more research.

The unique value of consumer panel research is that going back and asking questions is easy. Your panelists are there, pre-recruited, ready, and willing to respond. Make use of them. There is really no such thing as too much data.

#10 Use Pollfish to Make Deeper Insights Easier

At Pollfish, we’ve moved beyond the traditional consumer panel survey. We’ve developed a research methodology that makes gathering insights into consumer opinion easier. It makes recruiting panelists and understanding insights faster, more cost-effective, and less labor-intensive, too. 

For example, our consumer research platform sources insights from over half a billion engaged consumers, connected via our network of 120,000 app providers. We screen every participant based on pre-collected demographic, biographic, and behavioral data – as well as your screening questions – to take the stress out of finding the right audience.

Pollfish makes setting research objectives and framing questions easier too. We give you some of the most common research goals and pre-crafted question templates to choose from. To keep bias to a minimum and ensure participant engagement, all you have to do is select.

Finally, analyzing data is easier with Pollfish, too. Our advanced data tools let you filter your findings in real-time. All admin is taken care of, so that you can dedicate yourself to visualizing your insights – and ultimately putting them to work.