All About Panel Surveys: Using this Method for Market Research

All About Panel Surveys: Using this Method for Market Research

Panel surveys (not to be confused with survey panels) are used in survey research as part of longitudinal studies. The purpose of this kind of study is to reap continual observations on different variables on the same sample pool over a period of time. 

The variables include consumer feelings, attitudes and opinions regarding a number of matters. As such, the surveys are conducted across the same sample pool, i.e., the survey panel.

This way, the opinions of the same panelists can be monitored, as opposed to bringing in new respondents (even if they are part of the same target market). Panel surveys are conducted in waves (sequences of surveys) to measure changes in consumer thoughts and behaviors. 

There are many key aspects of panel surveys; this article will cover all of them to help you decide on whether your business needs to use the panel survey method.

How Panel Surveys are Conducted

First off, panel surveys are but one type of longitudinal studies, which also include retrospective studies and record linkages.

As their name indicates, panel surveys use surveying as the data collection method. This data is mainly collected through the aforementioned survey panels, a method in which a pre-recruited pool of respondents agreed to take part in your survey.

However, panel surveys can also be collected through other online survey methods, mainly DIY survey platforms. These allow you to target a specific set of respondents, but unlike survey panels, they are not conducted among the same exact group of individuals. (More on this below).

Since panel surveys are longitudinal, this sampling pool stays the same and is used for repeated studies and observations.

Panel surveys are part of panel research and can take the form of either qualitative or quantitative studies for measuring consumer behavior. 

The Pros of Panel Surveys

It’s worth considering several of the advantages that panel surveys carry when considering this surveying method. You ought to then weigh them against the disadvantages to determine if this is the correct survey route for your market research needs.

  1. Easy Collection of the Sampling Pool: Whether you intend to use an online survey platform or a survey panel, panel surveys make it easy to use a qualified sampling pool. That is because there is no need to look for respondents on an individual basis; instead, you either opt for recruited participants, or the online survey platform you use prescreens respondents for qualification.

  2. Longitudinal Benefits: As it follows the longitudinal survey method, panel surveys allow you to truly access the psyche of your target market. This is because consumers and the general public can change their minds — some more regularly than others. By conducting this survey research, you can stay up to date with any and all changes in attitude, thoughts and behaviors your target market undergoes.
  3. Speed to Insights: Online survey tools allow you to quickly garner all the respondents that match your screening requirements and demographics selections. A survey panel entails that respondents have already opted into the survey. In any case, getting results will be quick.
  4. Optimized Design: Whether you use an online survey panel or organic random sampling via a survey platform, each allows you to create customized questionnaires. The degree of this will change from one survey platform to the next, but the convenience of optimizing your survey to your favor is still present.
  5. Affordability: Panel surveys are relatively affordable. With online survey panels, you should be wise about how you incentivize your panel — this can be done by paying the panelists small amounts per each survey they take, or a larger sum for a set amount of surveys. A decent online survey tool should include several payment plans, with at least one perfect for your budget. 

The Cons of Panel Surveys

While an invaluable form of primary research, panel surveys are not without setbacks. Here are a few of the disadvantages these surveys harbor. Some may appear minor, while others are more considerable.

    1. Panel Fatigue: Predominantly found in survey panels, this occurrence refers to the reduction of interest among the panelist(s) when they take part in too many survey waves. This breeds full-fledged boredom and exhaustion, leading to a decline in the quality of data. In this case, panelists may take part in flat-lining or other inaccurate answer tactics. There are 5 types of survey respondents like these to look out for.

    2. Limited Internet Traffic: A problem principally found in online survey tools, as they are under the heel of the publisher sites and apps that deploy the surveys. There may not be enough qualifying respondents in one site or app, depending on who visits it during survey distribution.
    3. Survey Attrition: Piggybacking off of panel fatigue as the most severe case of it, survey attrition alludes to the dropping out of a panel. Panelists may experience a negative UX with the panel, thereby attriting at any wave of the study. These respondents are especially difficult to replace in survey panels, as they’ve already provided some crucial data, so there will be a void when it comes to conducting further surveys based on their responses. Some panelists do not drop out permanently, as they may return to the panel at some later time. With online surveys, this is relatively non-existent, as new responders are screened in each survey wave.
    4. Inclination towards Bias: Particular to survey panels, this occurs when panel respondents have taken too many surveys, thus becoming programmed to the way your surveys are set up. As such, they become less like genuine research subjects and more like trained survey-taking professionals. They may not put forth too much effort or thought into further survey waves because of this; rather they will clamber to get out of a survey as soon as possible.
    5. Respondent Identity Fraud: Especially common in online survey tools where you cannot validate the identity of your sampling pool, this can lead to respondent fraud. Respondents may lie about their age, employment and any other type of demographic identifier. Your data will suffer as a result. Survey panels may be immune to this if you choose your panel via face-to-face interviews, or if you know any member of the panel.

Panel Surveys: Uses and Applications

Panel surveys have various applications that you can put to use for your brand. Whether you’re looking to innovate an existing product, develop a new one, understand how your target market responds to current affairs, how they respond to certain communications or virtually anything else you need data for, panel surveys are useful ind=struments to leverage. 

Here are several specific uses and applications for these kinds of surveys.

  1. Detect common customer behaviors in relation to purchasing, clicking on an ad, or interacting with your brand in any other way (digital or physical).
  2. Analyze the costs of a product, service or subscription.
  3. Predict sales for particular campaigns and seasons.

  4. Understand how to use current affairs in brand messaging (including the knowledge of which subjects and rhetoric may be too sensitive for your target market).
  5. Monitor trends in how customers buy from brands.
  6. Find how recent and distant events have affected the attitude and opinions of your target market.
  7. Segmenting your target market. 

Marrying Survey Panels and Organic Sampling 

Closing off, it is crucial to reiterate that one of the key differentiators of panel surveys is their ability to incorporate two different types of survey respondent collection methods: survey panels and organic sampling (in online survey tools).

These fall into diametric opposition with one another, as the former involves recruiting willing survey respondents (the panel), while the latter uses organic sampling, in which respondents opt into a survey in real-time when they discover it in a website or app.

Although survey panels are the chief method to conduct panel surveys, due to the repeated nature of observation, both of these survey methods have the ability to sustain the longitudinal study method of panel surveys. 

Survey panels are the primary collection method of responses, as panel surveys and other longitudinal studies involve studying the same group of respondents

Survey panels do just that. However, survey tools that use organic sampling, can also study the same group based on their demographics and conditioned screening questions. The only difference is that the responders in organic sampling won’t be the exact same people, but rather those who fit the categories.

Either way these two methods can both be applied, or even used hand in hand. We suggest using a robust online software tool that provides a wide range of features to optimize your survey research. 


How to Get Insight from Consumer Panel Surveys: 10 Tips

How to Get Insight from Consumer Panel Surveys: 10 Tips

Alongside focus groups and ad-hoc questionnaires, consumer panel surveys are a popular tool for businesses looking to gain insight into what their audiences – and consumers at large – are thinking.

Using pre-recruited and pre-screened groups (or panels) of respondents, consumer panel surveys have specific advantages over alternative consumer research methods. You can control your audience, for example, and return to the panel over time to collect data repeatedly from the same set of people. This lets you dig deep into highly targeted audiences – and, crucially, compare changes in attitude over time.

Yet, as with all research methods, to get real value from consumer panel surveys, you need to refine your technique. That’s why, in this article, we’re sharing 10 tips and best practices to get the most from your consumer panel surveys. We’re talking how to shrewdly recruit and screen respondents, how to ask questions that elicit authentic opinion, and how to use the right analytical methods to yield deep insight.

Note: Don’t forget, Pollfish has created a new consumer research methodology that works a little differently. One that fulfills the promise of consumer panel surveys, while keeping bad data to a minimum – and making access to deep insights easier and more efficient. 

Where to Start with Consumer Panel Surveys?

Consumer research can deliver highly valuable insight for your business. Whether you want to test how an ad campaign will land, or get feedback on a new product line, opinion from authentic consumers – and potential customers – can be priceless.

Yet, you can’t just go and fire questions around at random. This won’t tell you anything of value. Rather, thinking through who you are asking and why is crucial before you start. 

Here are three tips on how to get that done.

#1: Gain Clarity on Your Research Problem and Goal

We’re starting with the basics, yes. But a true understanding of what you want from your research is an essential element that too often goes overlooked, particularly by businesses conducting research themselves. If you want insights that have value for you, you need absolute clarity on your larger goals.

Do you want to boost sales, refine a specific product, or determine brand positioning? A robust research objective will help you sculpt questions that deliver sharper insights – and save you time asking questions that don’t deliver value.

#2: Refine Your Research’s Scope and Scale

Once armed with a research objective, you now need to assess the desired scale of your research. This can get a little complex – but it ultimately shapes the reliability of your research.

Of course, the ideal survey would consult 100% of your target population – whether that’s marketing graduates in Scotland or cyclists among your company staff. While this scale would yield data on which you can surely rely, this is usually not possible in practice. Even if your target population is very small.

However, the further from 100% of the population, the greater the likelihood of inaccuracy. What you need to decide as a business is what margin of error you can tolerate. If you are making high-stakes decisions, you want data you can trust – and the larger the sample size the better.

#3 Source and Speak to the Right People

For authentic insights that deliver value for you, speak to the right audience. You can’t take this for granted. Before you conduct any survey, you need to recruit respondents that suit your target demographic.

This is what screening questions are for. These filter out respondents who aren’t right for you. However, as our resident expert, Jim Theodoropoulos, notes, using screening questions can be difficult to get right. These need to be combined with demographic targeting to narrow in on your audience.

Using Pollfish, for example, if you are seeking mothers who are runners, you can use the “gender” and “number of children” filters, before asking a screening question about preferred exercise. Like this, you can access deeper insights into specific audiences – and keep the professional panelists to a minimum.

Asking Questions that Deliver Real Value

The way you ask questions – and the form in which you receive answers – affects the quality of your data. And avoiding bias and panel fatigue, for example, can make conducting consumer panel surveys a bit of a minefield.

This is why many businesses use options like Pollfish to help design the questions. But if you’re going it alone, below are some tips on how to ask good survey questions and get answers you can trust. 

Note: For more on this, check out our article on how to ask good survey questions.

#4 Ensure Neutrality

Bias is a bad look in consumer research. For surveys to yield data that is actually valuable, they need to tap into the authentic opinion of respondents. Not reflect the prejudices of the researchers. 

That means avoiding leading questions, those questions that direct respondents to specific answers. For example, “How would you rate our award-winning customer service?” has the potential to skew responses by including the qualifier “award-winning”. 

However, even “what problems do you see in our customer service?” distorts the insights you will receive – as it assumes problems, whereas none may have been seen by the respondent.

This also means providing multiple-choice responses that reflect the range of opinion. If “how would you rate our customer service?” is your neutral question, the possible responses need to achieve similar neutrality. “Extremely helpful / Helpful / Neither helpful or unhelpful / Unhelpful / Extremely unhelpful”, for example, keeps the balance between positive and negative options while keeping framing consistent.

#5 Use Strategies to Mitigate Panel Fatigue

During surveys, and particularly during panel surveys, it’s not unlikely that respondents will become bored or disengaged – even though they sign up to participate.

When respondents are asked too many questions, we call this panel fatigue. It’s one of the biggest downsides to consumer panel surveys. Respondents might lean on “don’t know” or they might start “straightlining” – where they choose the same response for every question.

To avoid this, change up your question formats and shuffle questions around. It boosts engagement. It ensures more trustworthy responses. And it ups your completion rate – making data more reliable overall.

#6 Be Specific – and Remember Your Research Goal

Every question in your survey should provide an insight into your audience’s opinions, attitudes, and behaviors. Those that are irrelevant or too vague are not going to deliver the insights you need to make effective business decisions.

In this way, if you are a gaming app, an open question like “what do you like to do in your downtime?” is not focused enough – and it doesn’t direct respondents’ thoughts to gaming. As such, it won’t provide data that will be of use to your research goal.

Understanding Insights from Consumer Panel Research

You’ve asked the questions. Now, you need to understand the responses. 

This will first mean collating recurring answers, identifying patterns, and highlighting anomalies – a process that can be hugely time-consuming if you are doing it alone. That’s why Pollfish, by the way, delivers automated analytics in real-time. More on that later.

Once you’ve arranged the data, you need to analyze it. Here’s how to get started.

#7 Compare Your Data to Larger Trends

Numbers, let’s remember, are all relative. And getting valuable insights from them is difficult when they are viewed in isolation.

Your survey tells you 80% of respondents said that they were happy with your customer service. But for this to be valuable to you, this needs to be put in context. How does this compare, for example, to other businesses of your size in your industry? Or to your results from last year?

Larger trends are your friends – and they help make sense of your customer panel survey. If this is your first survey of its kind, let these results be the benchmark from which you understand future developments. If you have numbers from previous surveys, use these to assess your progress.

#8 Cross-Tabulate – and Dig Deep

Similarly, when you are reading your results, the big picture isn’t always the most useful view.

Say that 65% of respondents in a general population survey said they found your new ad campaign funny or very funny. That might feel like a win. However, these numbers could be disguising insights of more specific value.

That’s what cross-tabulation is about. Using crosstabs can let you see how different data fields overlap – and this is an indispensable tool for analyzing your data. For example, maybe the ad landed much less effectively with women. Or the joke was completely lost on 21-30-year olds. 

This matters. There may be an insight you need to address. You just need to dig deeper to find it.

#9 Keep on Researching

In a world in which business competition is sharp, a single survey won’t do. As a result, there’s one tip that needs to be stressed: keep returning to your audience to do more research.

The unique value of consumer panel research is that going back and asking questions is easy. Your panelists are there, pre-recruited, ready, and willing to respond. Make use of them. There is really no such thing as too much data.

#10 Use Pollfish to Make Deeper Insights Easier

At Pollfish, we’ve moved beyond the traditional consumer panel survey. We’ve developed a research methodology that makes gathering insights into consumer opinion easier. It makes recruiting panelists and understanding insights faster, more cost-effective, and less labor-intensive, too. 

For example, our consumer research platform sources insights from over half a billion engaged consumers, connected via our network of 120,000 app providers. We screen every participant based on pre-collected demographic, biographic, and behavioral data – as well as your screening questions – to take the stress out of finding the right audience.

Pollfish makes setting research objectives and framing questions easier too. We give you some of the most common research goals and pre-crafted question templates to choose from. To keep bias to a minimum and ensure participant engagement, all you have to do is select.

Finally, analyzing data is easier with Pollfish, too. Our advanced data tools let you filter your findings in real-time. All admin is taken care of, so that you can dedicate yourself to visualizing your insights – and ultimately putting them to work.

Frequently asked questions

What is a consumer panel survey?

A consumer panel survey is a type of survey that is conducted at set intervals over a period of time with the same group of respondents. The time period can range from days to years.

Why are consumer panel surveys used?

With interviews conducted over a period of time, consumer panel surveys are ideal for understanding how the thoughts, feelings, or behaviors of people evolve over time.

What are the benefits of conducting a consumer panel survey?

One of the primary benefits of consumer panel surveys is that they can be used to gain deep insights about a highly targeted audience. In addition, individual surveys are easy to deploy because the respondents have already been identified.

How are screening questions used?

Screening questions are used to identify the appropriate audience or panel for a survey. Combined with demographic targeting, these questions can help identify the best audience for a survey.

What is panel fatigue?

Panel fatigue occurs when survey respondents become bored or disengaged from the survey process, resulting in mediocre or inaccurate data.


Online Survey Panels: How to Access Deep Insights Efficiently

Online Survey Panels: How to Access Deep Insights Efficiently

When it comes to gathering data on consumer attitudes and opinions, there are a number of options available to businesses. You can go out into the world with a pencil and paper and ask questions directly to the public. You can organize a focus group for detailed, highly targeted responses. 

Or you can try online survey panels.

Online survey panels can offer deep and efficient access to what your potential audiences are thinking. Yet, they have been known to sometimes be tarnished by poor data. Panelists get over-surveyed, for example, and stop providing authentic responses. Or they’re just in it for the compensation – and so speed through questions without giving them enough attention.

Yet, this isn’t always the case. In fact, new methods of conducting consumer research are taking steps beyond traditional online survey panels. 

In this article, we’ll share what we at Pollfish are doing to ensure that you can gain deep, authentic insights from your consumers – in ways that are truly efficient in terms of cost and time. But first, what exactly is an online survey panel? And why would you use one in the first place?

What is an Online Survey Panel?

Survey panels are a research method in which data is collected from a group of pre-recruited and pre-screened respondents who have agreed to complete your survey. In a nutshell, that’s it.

That said, survey panels are often used for a more specific purpose – namely, to collect data repeatedly from the same set of people. In this way, a group of panelists responds to surveys in a number of research sessions over a given period stretching from weeks up to a year. This gives panel surveys some specific advantages over ad-hoc, one-off questionnaires, for example.

Meanwhile, while a focus group has only a handful of participants, from whom researchers can take very detailed data, survey panels use a much larger sample size – well into the hundreds and thousands. This way, they can give businesses access to a much broader section of the population, or provide deep insight into highly targeted demographics.

And whether you want to test a product, or see how audiences respond to an ad campaign, this knowledge of your consumers is indispensable.

What Insights Can You Gain from an Online Consumer Research Panel?

There are a number of reasons why you choose to conduct an online survey panel over other forms of data collection. Fundamentally, though, businesses tend to opt for panels for the specific kinds of insights that they provide.

In-Depth Data from Targeted Audiences

What survey panels offer better than most other methods of research is a targeted audience at scale. Participants in online research panels are profiled and screened – meaning that you can be sure that the panelists to whom you are asking the questions truly reflect your desired audience.

We don’t just mean this in terms of demographics, age, or gender. Pre-screening questions allow for targeted research in very narrow consumer segments. You may not just want to see how a product or ad will go down with the general population. You might be interested in how it will go down with men in the north of England who like jazz music and drink craft beer. Obviously, this is a little more specific.

Opinion Analysis over Time

By returning to the same group of respondents, online survey panels can provide data on changing opinion through time, within your identified audience. This lets you analyze trends within your target demographic, or within those with a certain behavior, to understand how your audience develops.

This continuous approach can offer powerful insights – and it can set up a great baseline to which you can compare one-off survey data too.

A Broader Selection of Data

Ad-hoc surveys provide limited snapshots of consumer opinion. Online panel surveys rather provide a much richer field of data. Alongside the simple responses to the surveys themselves, the initial screening questions, demographic information, and longitudinal data promise a much more comprehensive profile of your audience. And this can better inform key business decisions.

How Online Panel Surveys Can Boost Business Efficiency

Consumer panel surveys offer businesses with authentic, in-depth insights into their audiences. Yet, importantly, they do this at a relatively low cost – while offering quick, efficient results. It’s the combination of these factors that make them one of the most popular research methods.

Here are some of the efficiencies you can expect from online panels.

Repeat Participants Reduce Processing and Costs

With online survey panels, respondents opt-in and provide information about themselves in the process. This keeps start-up costs and time expenditure low. If you had to prepare screening questions and find your specific target audience for every survey you wanted to ask, your work would be never-ending.

The good thing about panels is that once set up, the panel remains there, pre-recruited and pre-profiled. This means that you can move fast whenever you have a question to ask. Changes to your brand can be tested quickly – allowing you to move forward on them in a more favorable timescale.

Quality Respondents Available at an Instant

Similarly, as participants sign up for the panel, you can be much more sure that they are willing to participate in your research. This gives you more data faster – as it tends to boost response rates, while keeping participants free from bots.

All that needs to be ensured is that the quality of the panel remains high. Yet, this saves serious energy and money in comparison is starting from scratch every time.

Efficient, Automated Data Collection and Analysis

Traditional panels are one thing; online survey panels are quite another. Modern technologies have powered efficiencies not only in the ways that you ask questions but in the ways that you understand the answers too.

This inevitably makes the analysis of patterns in data easier and more sophisticated. And it cuts down on the costs associated with labor time. In some cases, insights can be delivered in real-time as they come in. 

Yet, There’s a Problem with Online Survey Panels

Consumer research always has its imperfections. And this is also true for online survey panels. No matter how efficient they may be, there are issues that have plagued this research method since its inception – and are no less significant since the move online.  

Panel fatigue. A common issue, panel fatigue happens when respondents are over-surveyed. While they may sign up in good faith, participants can simply become bored. As a result, they rush through or skip questions, or lean heavily on the ‘don’t know’ option – which is not very helpful for your data.

Professional panelists. Panel participation has often been encouraged through cash-based incentives. But “professional panelists” – respondents who complete as many surveys as possible, to maximize their compensation – have proliferated as a result. These favour speed over quality answers – and are known to manipulate screening questions to ensure access to as many surveys as possible. Again, this affects the data.

Non-organic environment. A fundamental challenge for the accuracy of survey panels is that data collection does not occur within a consumer’s natural setting. Survey environments tend to be artificial, and this affects participants’ thought processes – meaning that quality, authentic data is not guaranteed.

What Pollfish Does Better with Online Surveys

At Pollfish, we’ve moved away from simple online research panels due to exactly these problems. And our consumer research methodology tackles these issues head-on – while assuring cost-effective and efficient insights for your business.

Our access to over half a billion engaged consumers makes this possible. These come through our network of 120,000+ app providers, which deliver surveys to their users in-app in exchange for specific, in-app incentives. This draws a line under professional panelists, who cannot accumulate rewards while assuring that surveys are completed by engaged participants in their organic environment.

Part of our method means that we also remove 30% of our responses. Why? With so many reasons why poor data accumulates – from professionals to fatigued panelists, to bots and fraudsters – we’ve implemented radical measures to ensure that quality remains high. This means that you get more authentic consumer insights and that every data point you receive is more reliable – and better value for your money, too.

Finally, our platform does all the screening. We have pre-collected data on participant demographics, and even mobile usage – so that you don’t have to spend the time doing this yourself. As a result, all you have to do is ask the questions.

How You Can Access Consumer Insights Efficiently with Pollfish

We make it easier to gain deep consumer insights. With the largest audience in the world, living in over 160 countries, we use a rolling profiling model to ensure that the data we hold is always up to date. This keeps your work to a minimum.

  • Define your audience demographics. Tell us who your audience is based on age, gender, education, and more. Then focus the specific consumer segment using our predefined criteria – on behavioral attributes, psychographics, and other biographic information.
  • Refine your target with screening questions. These get you even closer to the audience you want. For example, you may want to include only those who are aware of certain events or who hold certain opinions.
  • Select from question types and research objectives. We have collected common research objectives and pre-crafted question types so that all you have to do is select.
  • Analyze data with ease. Our advanced data tools allow you to filter your findings even further, by location or employment status. Survey results are available to you in real-time too, so you don’t need to wait for all the data to come in.

At Pollfish, we’ve gone beyond simple online survey panels, to make accessing customers insights even more efficient – and less prone to bad data. We do the heavy lifting so that you can dedicate your energies to putting your consumer insights to work.

Frequently asked questions

What is a survey panel?

A type of research method, survey panels are used to collect data from a sample of individuals who have agreed to complete the panel survey. In most cases, survey panels are performed over a period of time to collect data from participants several times.

What type of insights are survey panels used for?

Survey panels are particularly useful for providing insightful data from a targeted audience, understanding how opinions change over time, and providing a more comprehensive view of the target audience.

What are the benefits of survey panels?

A main benefit of survey panels is that you already have an established, pre-qualified group of respondents available to answer your questions. This results in a more cost-effective process and allows business owners to collect information quickly when needed.

What are the disadvantages of survey panels?

Survey panels are subject to some flaws, including panel fatigue and panel conditioning. Both of these occur when the respondents alter their survey responses or change their behavior as a result of their participation in the panel.

How can “professional panelists” affect the results of a survey?

Professional panelists are motivated to join the survey panel to gain the reward that is offered. These individuals may participate in as many surveys as possible, resulting in much lower quality responses.