research panel

What is a Research Panel and is it Necessary for Market Research

What is a Research Panel and is it Necessary for Market Research

research panel

A research panel is a frequently used means for conducting research, including market research (the study of your customers). This method involves studying the same group of opted-in participants through various methods and stages that are developed as part of a research campaign.

The technique that underpins a research panel counters organic sampling, which seeks out research participants, particularly survey respondents, in their natural digital habitats. As such, a research panel is an alternative to random sampling and has various differentiations. 

You ought to know all the differentiators of research panels, how they stray from organic sampling, as well as what makes a research panel tick.

With this key information at hand, it will make your research endeavors simpler; it will also allow you to choose the best research method. This is a must, considering that there is a wide range of market research techniques. Panels are just one of many.

You may be wondering if panel research is a viable research method for your business needs or research campaign. Or, you may consider using it in tandem with another research technique or tool. 

do you need a research panel

Luckily, we’ve got you covered on this topic. 

This article explains the concept of the research panel in full depth, which can serve as a possible avenue market researchers can explore within the vast array of market research techniques. 

What is a Research Panel and is it Necessary for Market Research? Table of Contents

  1. Defining the Research Panel
  2. The Role of a Research Panel in the Market Research Process
  3. When to Use a Research Panel</a
  4. The Pros and Cons of a Research Panel
    1. The Pros
    2. The Cons
  5. Research Panel Examples
  6. Why Online Polling Software is Better
    1. Organic Sampling and RDE
    2. Greater Privacy
    3. Greater Reach to Research Participants
    4. Upfront Incentives
    5. Less Time Consuming
    6. Less Room for Attrition, Boredom and Bias
  7. The Online Survey Tool: A Stronger Alternative
  8. What a Machine-Learning and AI-Powered Survey Tool Does for Your Market Research Campaign
    1. How else do research panels compare to an online survey platform?
  9. The Ultimate Verdict on the Research Panel for Market Research

Defining the Research Panel

A market research panel is a pre-recruited and pre-screened group of research participants who have opted in to take part as the studied subjects of a market research campaign

This kind of research method can involve studying its members repeatedly. In this case, the particular study is called a panel study

It is also referred to as a longitudinal study, although longitudinal studies don’t necessarily need to involve panels, as there can be longitudinal surveys completed by non-panelists. 

As such, it is a way of describing those who have agreed to take surveys on an ongoing basis, which, in market research, are typically members of your target market

You can use a research panel for a wide range of subject matters. The members of the research panel can include a wide range of people across multiple sets of populations

what is a research panel

Whether you seek to study the workforce of a company or a major constituent of a national population, the term research panel can apply to all such groups. 

The key is to use participants who represent members of your target market and most importantly your target audience also referred to as your survey target audience. That’s because a research panel is a recruitment method used to get respondents to take your survey.  

In market research, the participants in a research panel are usually the people who belong to a business’s consumer base. 

Moreover, they belong to a particular audience, known specifically as a survey target audience in survey studies. This label can also apply to research panelists, as they too can be asked to take surveys. 

The members making up a research panel must share several traits, such as demographics, psychographics, geographic location and more. A market researcher may also study various segments that make up a target market.

There are various methods researchers can employ in their research campaigns, in which a research panel provides insights. These include:

  1. Interviews
  2. Focus groups
  3. Surveys
  4. In-home usage tests
  5. Experimentations
  6. Test marketing

The Role of a Research Panel in the Market Research Process

A research panel is but one process within the encompassing practice of market research. Some businesses may decide to extract data from a research panel alone, while others may use it alongside probability sampling.

Also called organic sampling, this method involves reaching out to all the individuals who fall under the qualifications of your subjects of study. As such, it allows more individuals to take part in the sample.

Unlike many of the sub-methods of organic sampling, a research panel is not anonymous, in the sense that the panelists’ identities will not be hidden from the researcher. 

research panel in the market research process

They are still kept anonymous when it comes to sharing the findings with the public, as you wouldn’t reveal the panelists’ identities. 

This allows researchers to study the members at a greater depth, in that researchers can match answers with the respondents themselves. This is due to the nature of pre-recruiting participants; when you do so, you’re going to need to collect information on each panelist, some of which may be personal.

This method will allow you to understand if they’re qualified to partake in your studies. As such, you’re effectively putting names and faces with data, essentially identifying each member. Additionally, this allows you to build a profile on each participant, adding bulk by applying multiple studies. 

Forming profiles gives you a glimpse into the presence of personas in your target market. A research panel is the starting point in building a persona. 

When you’ve profiled panelists through various means (interviews, focus groups, etc.), you have several kinds of data, from which you can form an analysis and draw conclusions.

You can test the prevalence of these conclusions by surveying other members of your target market, i.e., those who are not in the research panel. 

Various survey sampling methods will not only complement your research panel but also give it validity and statistical relevance. After all, there are only so many panelists you can interview or meet with.

Even if you study your research panel via surveys, it is not practical to spend a lofty amount of time vetting people to ensure they fit your research campaign. Thus, a research panel may not be the strongest of the various market research techniques.

When to Use a Research Panel

While businesses and market researchers can use a research panel liberally, it is not always in their best interest. This can be due to the size of a business, a limited budget, the objectives of a research campaign and the length of the research study. There are also times when it makes sense to engage in research yourself and other times in which it may be beneficial to work through a professional market research agency such as IntoTheMinds.

when to use a research panel

With this in mind, there are particular times in which companies and researchers alike can benefit from using a research panel. These include the following instances:

  1. Obtaining a constant, in-depth read of a certain group of participants.
  2. Conducting a more intimate study on a particular group of people.
  3. Running continuous studies on the same people, ie, for longitudinal studies
  4. Gathering data on subjects with scant studies due to rarities. Ex: people aged 100+
  5. Large research projects that will involve multiple modes of data collection
  6. When you are performing market segmentation.
  7. When you are building research or customer personas.
  8. To fulfill the preference of conducting research in a group setting.
  9. To gain insights on a topic that you may not have considered from your list of questions/concerns.
    1. These insights typically arise in conversations, as participants bring up points and considerations that you may not have originally thought of when forming your research plan.
  10. To assist or act as a helping agent in conjunction with another form of research, such as survey studies

The Pros and Cons of a Research Panel

The research panel tactic offers advantages and disadvantages that all market researchers should be privy to. Like other research techniques, it is not perfect and for some, the disadvantages may outweigh the benefits, while to other researchers, the opposite may be true.

You should mull over both the advantages and disadvantages that come with this form of research. 

The following lists the advantages and drawbacks of using a research panel.

The Pros

  1. Panel members have a more advanced understanding of the research topic since they can be recruited through a longer vetting process.
  2. It can be used multiple times on the same survey, to study change within a particular group that represents segments of your target market.
  3. It’s easier to conduct in recurring intervals, given that you have all the panelists’ information and don’t need to screen them as you would with a new set of participants.
  4. Deeper reads and longer researcher/panelist interactions are suitable for the 3 main types of survey research methods
  5. It is much easier to follow up with panelists, should you need more research, as you already have their contact information.

The Cons

  1. Lack of privacy: face-to-face interviews, along with phone interviews in which researchers know the identities of panelists can be intimidating.
    1. Even a panel study lacks privacy, which can lead to intimidation or fear of answering honestly. 
  2. Acquiescence bias: along with other biases, this issue can take shape, as respondents may feel pressured to answer in a particular way, leading to forced or inaccurate responses.
  3. Panel attrition: Due to re-interviewing, research panels are susceptible to fatigue, loss of interest, or pressure (Points 1, 2), making them easy candidates for attrition.
  4. Ingenuine change of attitude/ opinion: Interviewing and reinterviewing can change attitudes, in ways that are not always genuine, due to re-interviewing.
  5. Expensive: Whether you hire an in-house panel or use an external one, it is often an expensive affair, as you will need to pay each panelist. Since this is an ongoing study, you may have to pay them for each session.
  6. Poor data quality: This is especially true when a panel member is a participant in multiple panel companies. 
    1. The quality of the data may be compromised when a respondent is a member of two or more panels. 
    2. This is because the respondent may partake in the same survey.
    3. If they answer the same way, you will have duplicate data, but if they answer differently, there might be bias. At any rate, you’re getting data from the same person twice, which doesn’t improve the trustworthiness of research findings.
  7. Missing out on a larger survey pool: This relates to the aforementioned lack of privacy. Not everyone in your target audience will want to give away their contact info, let alone have their answers be tied to their identity. 
    1. As such, you may not get enough participants for the specific quantity required for your survey sampling size.  

Research Panel Examples

A research panel can be applied to all kinds of scenarios and has various use cases. Remember, they can be applied to both long and short-term research, despite being associated with the former more often.

They can be used in market research, which is for business purposes and is centered on customers. Or, they can be used for a wide range of other research types, such as medical, scientific, social, behavioral and educational research.

To help you better understand research panels, the following list includes seven examples of them across different areas of study:

  1. A business studying the customer buying behavior of three of its customer market segments. 
    1. This is especially useful to compare segments with high and low consumer loyalty
  2. A university research group studying the effects of sleep deprivation among students over a semester or year.
  3. An enterprise company seeking to release the most resonating ad campaigns by comparing how it's received across the world.
  4. A condiment manufacturer who is interested in comparing flavor and texture preferences across different parts of the country.
  5. A business that is intent on following its target market’s shopping habits and how they compare to their competitors.
    1. This will need to involve research on competitors. That means you’ll need to inquire about them in the panel, as well as perform secondary research to complement the study.
  6. A healthcare company seeking to find the relationship between device usage and obesity.
  7. A government program that tracks the success of a new social program for certain populations.

Why Online Polling Software is Better

Online polling software trumps research panels for a variety of objective reasons. There are also various subjective and preference-based justifications for leveraging an online survey tool instead of a research panel. 

polling software

Organic Sampling and RDE

First off, online survey platforms allow you to run random organic sampling, which allows you to reach non-professional survey takers and gain a far larger reach than you otherwise would have.

This is because organic sampling involves what’s known as Random Device Engagement (RDE), a kind of polling that relies on advertising networks and other portals on devices, to engage random people where they are, voluntarily.

Additionally, in Random Device Engagement, the surveys are delivered to users in their natural digital environments, capturing them where they prefer to be. They were not pre-recruited and thus do not face the same pressures and conditioning that they would in a research panel.

As such, they are more likely to answer questions truthfully, as they have no one to answer to, are kept anonymous and have nothing to lose

Greater Privacy

With far more privacy afforded to them, respondents of organic sampling surveys are also less vulnerable to acquiescence bias and all the other biases that involve the respondents’ reputation. 

On the other hand, there’s polling software. This method, as aforesaid, provides respondents with the most privacy, as they are not pre-recruited or pre-screened. In many cases, polling software reaches respondents organically, which affords respondents the most amount of privacy.

Some survey platforms (such as Pollfish), allow you to send surveys to specific individuals instead of simply across a vast network of online platforms; in this case, the study won’t be as private. However, it is another deployment option to expand how you run your survey study.  

Greater Reach to Research Participants

It also has a far greater reach to respondents. This, however, will depend on the online survey platform you use. We suggest one that allows you to conduct global surveys with the same ease as you would with local surveys. 

Upfront Incentives

When you use an online survey platform, survey incentives are usually mentioned upfront. This is typically the case with a survey platform that partners with gaming sites and other digital platforms that offer in-app awards, which can be either monetary or non-monetary.

With incentives being offered (or at least mentioned) at the fore, all kinds of customers will be more willing to participate in the survey study.

Less Time Consuming 

Moreover, an online polling platform isn’t as time-consuming for respondents. This is because such a platform does not simply conduct longitudinal studies — and even when it does, it can target random people who fit into certain customer profiles and customer personas.  

It is also far less time-consuming for researchers. That’s because they don’t need to conduct interviews or other actions to recruit participants; the polling software does it for them.  As such, it’s a win-win for all the people involved in the study: the respondents and the researchers. 

Less Room for Attrition, Boredom and Bias

As such, it isn’t reliant on using the same people repeatedly to take part in a study. In this way, it cuts survey attrition. This is because some panel members may feel exhausted, burned out or simply frustrated with having to continuously be part of a study, especially if it covers the same subject. 

As such, using polling software grants you the opportunity to ward off boredom from your respondents, as well as gain accurate responses. As mentioned earlier, panelists are far more prone to acquiescence bias and other biases. 

Respondents of a polling platform offering organic sampling are at a far lower risk of being biased or getting bored. The latter is especially true in a platform that offers a mobile-first environment. After all, mobile dominates online web traffic, as over half of web traffic comes from mobile devices.

Thus, a good survey design, especially one built for the mobile space creates a pleasant survey experience, one that intrigues respondents to take a survey in the first place, and most importantly, complete it.  

Aside from these advantages that online polling offers over research panels, there are many more. The other pros deal largely with the survey tool itself as opposed to its distribution and high-level polling aspects.

The Online Survey Tool: A Stronger Alternative

While a research panel has several benefits and use cases, online survey tools present a stronger alternative. First off, they have even more use cases and can be applied to all with greater ease.

This is because the survey tool itself does all the recruiting and screening for you. As a researcher, marketer, or business owner, you don’t have to worry about whether your survey respondents fit your target survey audience’s qualifications.

Identifying and acquiring respondents are both taken care of by an online survey platform, that is if you choose a potent one. This means you don’t need to have a pre-study interview to vet potential participants. Instead, everything is automated.

A strong online survey platform offers machine learning and artificial intelligence software to run all of its functions and mechanisms. This means, there is little to no manual labor required on your part. 

AI survey platform

All you need to do in your survey campaign with a strong online survey tool is:

  • Set your screener so that your survey targets the correct populations
  • Create your questionnaire
  • Analyze the survey

Those are the three steps involved in the Pollfish platform. If you’d like to learn how to make your own survey in just 3 easy steps, read the article in the hyperlink.

The online survey platform should handle all the rest. When it comes to running a high-quality market research campaign, there is a lot that goes into staving off poor-quality data and ensuring accurate results.

The following lays out what an AI-powered survey platform can do for your survey campaign:

What a Machine-Learning and AI-Powered Survey Tool Does for YOur Market Research Campaign

A lot is going on behind the scenes of an online survey platform. Luckily, you won’t have to worry about nearly all of them. Regardless, it is crucial to understand the depth of survey SaaS that runs on machine learning and artificial intelligence.

Here’s what to expect from an AI-based survey platform:

  1. A strong adherence to targeting
    1. No respondent partially matches the demographic and psychographic screening that the researcher inputs into the platform.
    2. All survey participants must match 100% of the respondent qualifications. If not, they are disqualified from taking the survey, no matter how close to filling all slots of the criteria they get.

      market research survey
      The Audience section on the organic sampling survey platform Pollfish has a rigid adherence to granular respondent targeting.


  2. Respondent verification
    1. This mechanism checks respondents for duplicated IDs to ensure each survey completed is done by a unique person, as opposed to one person taking a survey more than once.
    2. The platform checks IP and MAC addresses, Google Advertising and mobile device identifiers.
    3. In addition, the platform works with vetted publishers to send unique IDs as an added layer of protection against survey fraud
  3. A layer of security in the questions themselves
    1. In-survey questions are designed as yet another layer of security against survey fraud.
    2. For example, a question can request respondents to answer a simple math problem.
    3. Or the survey would include identical questions with the response options re-ordered to verify answer consistency.
  4. Antibot Policy
    1. Bots are no match for an AI-powered platform that is designed to disqualify them from taking a survey.
  5. Zero tolerance for VPNs
    1. Most businesses and research campaigns put qualifications based on geographies. 
    2. A respondent on a VPN would tarnish any study with filters on who gets to partake in the survey based on location.
    3. The Pollfish zero-tolerance approach to VPNs ensures the veracity of respondents’ location.   
  6. Removal of incomplete surveys
    1. This speaks for itself, as surveys are meant to be fully completed. A partially complete survey would provide insufficient data.
    2. Incomplete surveys are especially problematic in surveys with follow-up questions to past questions, or those seeking more depth to a certain issue. 
  7. Removal of surveys with suspicious activities 
    1. Surveys with any questionable behaviors are rejected.
    2. This includes the removal of the following:
      1. Answering open-ended questions with nonsense 
      2. Attempting to sign in from multiple countries/devices at once.
      3. Taking an inappropriate amount of time on the survey.
  8. Multiple layers of quality checks
    1. The survey platform uses a technical layer to perform other quality checks.
    2. This process includes our technical experts continuously working to avoid survey fraud. 
    3. There are several layers that we use to maintain good data quality. These include checks on the following:
      1. Hasty answers Check: catches respondents who answer faster than the average time needed to read the questions.
      2. Reset ID Check: Activates when the respondent answered the same survey previously, but with a different device to avoid the same respondent from taking the survey more than once.
      3. Gibberish Check: Checks for answers contain nonsensical text. This is the kind of text without real words, such as “jnfjv vdf gre.”
        gibberish survey answersAvoid receiving gibberish answers thanks to the Pollfish AI-powered survey platform
      4. Same IP Participation: Checks if a survey has been completed before within a certain time from the same IP address of the respondent’s device.
      5. Carrier Consistency: Assures that the carrier of the respondent’s internet service exists in the targeting market.
  9. Specialized questions to identify those not paying attention 
    1. Aside from a layer of security in all questions, we offer specialized questions that detect poor data quality.
    2. These include the following question types:
      1. Red herring questions: Asks questions with odd answers to assure respondents are paying attention.
      2. Trap questions: Finds who is paying attention to a command, usually one that asks to select a negative response. Responders who choose positive responses will be caught.
      3. Quality Questions: Similar to red herring questions, they check if respondents read and understand what’s being asked.
  10. Constant iteration until all quotas are met
    1. With the agile research approach, the platform doesn’t merely provide speedy insights.
    2. Instead, it creates constant iterations until all the quotas and the desired amount of completed surveys are met.
    3. As such, the platform doesn’t cease, or pause (unless you set this command on your dashboard).
    4. It allows you to gain the proper amount for your sampling pool.
    5. With this, no survey pool is too large (relative to the necessary sampling size).

How else do research panels compare to an online survey platform?

Respondents can rest assured that they do not need to give away their data. To add to this, they can still be incentivized to take part in a survey study. 

An online survey platform does all the heavy lifting in terms of retrieving responses, while in a research panel, the researcher has to make sure that all the participants respond adequately. This is to say that the researchers themselves must check for gibberish answers, questions left unanswered and much more.

This is especially more difficult in focus groups and one-on-one interviews, in which a researcher has to make sure everyone participates in the former, and that the panelists are willing to truthfully answer all the questions in the latter.

An online survey tool also effectively eliminates the need to worry about survey response rates, as it keeps iterating until the preset requirements are met (including the number of respondents).

As such, researchers have plenty to gain for their research needs from using an online survey tool in tandem with a research panel, or even as a replacement for a research panel.

The Ultimate Verdict on the Research Panel for Market Research 

A research panel is a useful method for conducting market research, particularly for studying the same group of participants to monitor their opinions and behaviors and changes thereof.

However, a productive market research campaign will rely on using more diverse methods to extract data. This involves using random organic sampling, which forgoes the conditioning and pressures of a research panel.

As such, you should opt for a survey platform that offers RDE, or Random Device Engagement, which, as mentioned earlier on, distributes surveys randomly, across a wide network of digital properties. This includes websites, mobile sites and mobile apps. 

With this survey function, the platform does all of the work when it comes to identifying respondents and covering all quotas. That means you don’t need to do anything in this regard, as the platform performs these tasks.

But there’s more.

To piggyback off of the section on the role of the research panel, online surveys and research panels do have some beneficial similarities. For example, they’re both ideal for creating and validating personas. 

A research panel can identify a persona over several rounds of interviews/ surveys/ etc., while an online survey tool can conduct further research to find whether those personas are statistically significant.

Thus, these methods work well hand-in-hand when it comes to conducting market research. A strong online survey platform will ensure a synergistic relationship between random sampling surveys and research panels.

It should allow you to survey specific people, such as via email, or whichever digital channel you seek to use. Luckily, there’s the Distribution Link feature, which enables you to do just that. 

Frequently asked questions

What is a research panel?

A market research panel includes participants who have willingly opted to participate in a research group regarding a specific subject. These members are pre-screened and pre-recruited.

Why are research panels important in the market research process?

Research panels are essential because they are not anonymous, unlike the respondents who take the surveys incognito. This is important because it allows researchers to find out everything about the members to match answers with the respondents themselves, ruling out chances of inaccuracy.

When should businesses use a research panel?

Businesses should set up a panel to facilitate in-depth research of audiences and their behavioral patterns or conduct a detailed, intimate study on customers. Large research projects that require multiple modes of data collection or market segmentation also work well with a dedicated research panel. You can also use them when building customer profiles.

What are some pros of using a research panel?

Panel members usually have a more advanced understanding of a research topic. Research panels are also easy to conduct in recurring intervals, and researchers do not need to screen information as they would with a new set of participants. In this way, it is faster and more efficient.

What are some cons of using a research panel?

With a panel, participants do not have privacy. They may have to participate in interviews which can be intimidating. This may pressurize a respondent to answer in a particular way, leading to an incorrect response. Research panels are also prone to fatigue, loss of interest, and panel attrition. Also, hiring a research panel is usually costlier as you may have to pay the participants for every session.

survey panel

All About Panel Surveys: 5 Panel Survey Examples and More

All About Panel Surveys: 5 Panel Survey Examples and More

survey panel

A survey panel also referred to as a panel survey, is a common research method used to gain insights from the same group of participants over a certain period

This technique relies on examining research participants who join a market research study voluntarily. Aside from this, the participants are pre-recruited by researchers, to ensure they fit the respondent criteria they set for a particular research project.

This kind of research technique may appear to be a buzzword, given its prevalence in the market research space. However, whether it is useful and effective for your market research is entirely up to your research preferences and campaign needs. 

Nonetheless, it is crucial to understand what a panel survey is before you can decide on whether you ought to implement it into your research strategy or not. Or, perhaps you want to use it in conjunction with another research method.

Either way, you should have a holistic view of everything this research method entails. This includes comparing it with random organic sampling and online surveys. Luckily, this guide provides all of this information and more.

This guide explores panel surveys, explaining them in full depth, along with their pros and cons, 5 examples of them and a comparative analysis of random sampling and survey panels. 

Note: If you’re looking to do market research without the drawbacks of conducting panel surveys, try Pollfish. Our unique methodology guarantees authentic high-quality data, and it’s a fast and reliable way to survey targeted (yet randomized) people while they’re already engaged on their devices.

All About Panel Surveys: Table of Contents

  1. Defining Panel Surveys
  2. Panel Surveys: A Kind of Longitudinal Study
  3. Using Panelists in Your Target Market
  4. What to Study with a Panel Survey
  5. The Importance of Panel Surveys
  6. Panel Survey Examples: What Do Researchers Use Panels For?
    1. Example #1: Tracking Changes Over Time
    2. Example #2: User Experience Research
    3. Example #3: Customer Satisfaction Surveys
    4. Example #4: Employee Engagement
    5. Example #5: Customer Loyalty
  7. Pros & Cons: The Advantages and Drawbacks of Panel Surveys
  8. Organic Sampling: A Better Alternative to Panel Surveys

Defining Panel Surveys

Also called survey panels, panel surveys are a distinct form of market research that relies solely on panels to carry out the research study, as its name suggests.

Thus, it’s key to understand what the term “panel” means in the context of market research. A panel is a selection of participants who help researchers conduct market research by providing them with feedback on various studies. 

The panelists do not join the study randomly, rather they are pre-recruited and pre-screened, based on the researcher’s panel needs and criteria. 

A kind of research panel, a survey panel operates in the same way, except that it is strictly used within survey campaigns. As such, this panel is used for survey research alone, as opposed to other methods of panel usage.

Panels can be used in various research settings, including the following:

  1. One-on-one interviews
  2. Focus groups 
  3. Survey studies
  4. All 6 main types of research, which include
    1. Exploratory research
    2. Correlational research
    3. Causal research
    4. Descriptive research
    5. Experimental research
    6. Explanatory research
  5. Field research

In the context of this guide, we’re going to focus on panels used in survey research alone. Hence, they are called survey panels, or panel surveys.

One of many market research techniques, this kind of research process uses a consistent group of participants, with the researchers returning to them repeatedly, either for a single survey campaign, which can require using multiple surveys or for multiple campaigns. 

Panel Surveys: A Kind of Longitudinal Study

It is important to know that when referred to as a panel survey, this kind of research method is known as a longitudinal study. As such, this process is paired with longitudinal surveys.

This type of research study is often associated with correlational research.

As a type of longitudinal study, panel surveys involve correlational research because they examine the relationship between variables over an extended period. The period itself can vary.

As such, this kind of research method can be conducted in the following ways, occurring throughout:

  1. Weeks 
  2. Months
  3. Years 
  4. Decades

Regardless of the time dedicated to performing this research, it always involves the same survey target audience. By surveying the panelists at different points in time, researchers can glean a wide variety of changes and consistencies within their research subjects, aka the panelists. 

Using Panelists in Your Target Market

While the panel remains the same, the topic(s) that can be scrutinized are virtually limitless

For market research, the topics deal with a business’s target market. As such, all members of the panel survey must also have the qualities of those in your target market.

You can perform panel survey studies by zeroing in on your target market’s various market segments and customer personas. These are more distinct and defined groups of people who fall under your customer base.

survey panelists

This is the ideal way to conduct this kind of research and many others, as your target market can include a wide range of subgroups, whether they differ based on the following categories:

  1. Geographical locations
  2. Demographics 
    1. This involves groupings based on gender, age, income level, ethnicity, employment type, education, etc.
  3. Psychographics
    1. This involves their attitudes, lifestyles, aspirations, values and other psychological criteria.
    2. It also involves whether they engage in a particular customer behavior. 
  4. Firmographic
    1. This label is particular to studying a panel of business personnel, which you may do in B2B research.
    2. As such, it requires running B2B surveys

Whichever categories you intend to group your panelists into, remember that they must fit into the qualities of your target market, as this group is the population of people most likely to purchase from your business. 

What to Study with a Panel Survey

As aforementioned, you can explore a wide variety of topics in your panel survey. The following list enumerates some of these topics. Note that this is not a comprehensive list, as various other topics of concern may crop up, depending on your business, your customers and your research needs.

Here’s what this kind of research method allows you to study:

  1. Changing attitudes, opinions and sentiments
  2. Changes in customer buying behavior 
  3. Changes in customer lifestyles
  4. Changes to consumer preferences 
  5. Developments in your target market needs
  6. How your target market views your competitors
  7. The willingness of your customers to make repeat purchases
  8. Gaining a more in-depth glance at your market segments and customer personas
  9. Performing time-based research, such as longitudinal studies, retrospective studies and prospective studies 
  10. Acquiring a snapshot of the present through cross-sectional studies

In this post, we’ll look at five-panel survey examples, which delve deeper into some of the topics on this list. But first, we’ll explain why this research method is important for researchers and their market research campaigns.

The Importance of Panel Surveys

Panel surveys are important for a variety of reasons. Some of these reasons may hold more weight than others, depending on your research needs and preferences.

Firstly, these panels are ideal for longitudinal research, as the same group of participants agreed to partake in your study. You would make it known that they would be used in continuous studies.

Aside from facilitating longitudinal studies, panel surveys are also ideal for another time-based research campaign: retrospective studies. As such, these panels aid researchers in completing their retrospective surveys.

This kind of study is used to examine changes over a long period, except the survey itself is only administered once. That’s because retrospective studies invoke occurrences from the participants’ past, whether they are experiences, inclinations, feelings, or attitudes. 

The study may ask participants to compare past phenomena with how they feel in the present. The purpose is to study historical data and compare it with the present. 

Panel surveys are also necessary for prospective studies, which also can last for years and study changes and developments. However, unlike retrospective studies, they involve participants who do not have a certain outcome. Instead, prospective studies seek to study people before they develop a certain outcome, such as a disease or a new shopping habit.

panel surveys

Aside from enabling researchers to study changes, these panels also make it possible to conduct cross-sectional studies, which are different from the prior three kinds of research studies —  longitudinal, retrospective and prospective studies. Nonetheless, cross-sectional studies are still one of the key survey research methods.

As opposed to studying survey subjects at multiple points in time, a cross-sectional study entails gathering research about a particular population at one fixed point in time. Because of this, this type of research method is often referred to as a snapshot of a studied population.

This kind of study is conducted by using cross-sectional surveys, which are designed specifically for the cross-sectional model and can be sent to your panelists. Although cross-sectional studies don’t require a panel survey, given that they focus on a single point in time instead of several to examine changes, they can still be administered via these panels. 

That’s because you can recruit panels for virtually any study, whether it includes partaking in a survey, an in-person interview, field research, or a focus group. Thus, panel surveys are not limited to any form of study, making them multifunctional. 

This grants plenty of convenience to researchers, especially those bogged down by various other tasks they have to uphold, instead of putting together a group of respondents. 

Panel surveys are especially important, as they provide an ongoing sampling pool of participants, which researchers can turn to whenever they seek to conduct more research. This can be to jump-start a new research campaign or conduct further research for an ongoing campaign. 

This constant access to a sampling pool can also assist researchers who initially thought they had enough data, only to later discover they need more information. After all, if the panelists have already opted in, there is no need to seek out participants any further.

Panel surveys are also useful due to the quality of the panelists. Although identifying and vetting panel members may take some time, the result is a quality survey panel, one that ticks off all the panelist criteria you have. With this, you can mitigate fraud and rest assured you’re studying the correct populations.

All in all, panel surveys are useful for a variety of market research campaigns and come with a good share of benefits, despite some of their shortcomings.

Panel Survey Examples: What Do Researchers Use Panels For? 

Now, let’s get to the heart of these panels, by going over what they can consist of, when, where,  and how to use them. 

The best way to explain the uses of a panel survey is by providing several panel survey examples. In this section, we’ll review five examples of when panel surveys can be used to good effect. 

This will allow you to understand how panels work, along with some of their best-case scenarios for usage. 

survey panels

Example #1: Tracking Changes Over Time

Given that studying changes over time is the main purpose of leveraging a survey panel, it’s key to review this use case.

As such, one of the strongest examples of a panel survey is when researchers ask the same question(s) to a group of the same participants over a long period. 

For example, if you were conducting a sociological experiment to understand if (or how) age correlates with wealth, you can set up a survey that studies panelists on the following:

  • Income
  • Expenses
  • Assets
  • Changes in employment
  • Willingness to change jobs
  • Willingness to switch job fields 

A survey covering this topic ought to be conducted every two years, possibly spanning a decade (or more if need be). 

That way, you can track changes relative to the scope of the issue you’re researching. Given that people typically stay at a job for about two years, it is key to follow up every two years on this research subject.

However, this has changed due to the Great Resignation, which saw the largest exodus of employees quitting their jobs in recent history. 

The Great Resignation is not slowing down any time soon, with large numbers of people seeking better pay and benefits, along with the rise in job openings and remote work. This makes it much easier to switch jobs. 

As such, the length of time you devote to tracking changes will depend on the circumstances of your study.

Alternatively, if you run a business, you’ll need to acquire ongoing insights into the market – and into audience behavior. For example, if your gym wear company wants to track the fitness habits of people over the year, a panel can be questioned every three months. You can run surveys with the same questions about how long people exercise, where they exercise, and how they do so.

For any business endeavor, you can implement the brand tracking survey, which allows you to track any changes that deal with your brand. This involves studying how your business is perceived, how in-demand its offerings are, what concerns your target market has that your business can address and much more. 

Example #2: User Experience Research

Another common example of using a panel survey is for UX and UI design research. While UI refers specifically to designing web experiences, UX is far broader. 

User experience can refer to a wide range of experiences that your customers encounter when interacting with a product, digital experience, or service. As such, it covers a wide range of aspects, all of which have some impact on how users experience your business’s offerings.

ux research

You can use these aspects as key considerations for driving your UX research. The following lists the four key components of user experience, which you should use in your research:

  1. Value: Does this experience give customers value?
    1. You can ask your panel whether a certain product, service, or digital experience provides them use and how so.
    2. It is key to study how your offerings can be made more valuable.
    3. Also, you should inquire about the products/services that your customers don’t find useful.
  2. Function: Does a product or product feature work?
    1. Ask your panel whether there are any bugs or glitches in how a product, feature, or service works.
    2. You can also study whether new needs arise in terms of your product’s functionalities.
  3. Usability: Is it easy to use?
    1. While your offerings may provide value, ask your customers whether they are easy to use, regularly.
    2. Study if your offerings can be made simpler over time.
    3. Also, look into whether customers seek other ways to access your offerings. 
      1. For example: different devices, operating systems, web and app access
  4. General impression: Is it pleasant to use?
    1. Track changes to the overall sentiment towards your products, services and business at large. 
    2. Ask your customers whether your offerings delight them or upset them.
    3. Ask customers to reveal anything else they’d like to see from your business to improve their UX.

When a company is building an app or a digital product, they want to gauge the customer experience as they go

While usability testing (i.e. tracking how a person interacts with a specific feature), is often done using different methods, you can conduct qualitative research about design and experience by way of panel surveys.  

The benefit of a panel survey here is that the participants remain constant – which removes a key variable. 

Additionally, you can select panel participants based on their background or expertise, which will allow your pool to answer questions in a way that adds value to your product development process. 

Example #3: Customer Satisfaction Surveys

Customer satisfaction is central to keeping your business alive, whether you want to release new products or build a steady base of loyal customers. As such, you can use panels to keep a watchful eye on your customers' satisfaction with your business. 

Doing so will allow you to increase your customer lifetime value, a metric that measures the total monetary value a customer will bring to a business during their relationship with the business throughout their lifetime.

If you want to gauge customer satisfaction and positive/negative sentiment over the lifetime of their interactions with your offering, use a select group of long-term customers and ask them questions periodically about your value proposition. 

Customer Satisfaction Survey
Use the Pollfish Customer Satisfaction Survey template

Use a customer satisfaction survey for this campaign, which can be framed to reveal hidden sentiments, or can be explicitly focused on asking how satisfied they are with what you offer. 

For example, use the following questions/considerations to measure customer satisfaction:

  1. On a scale of 1-10, how satisfied are you with [a product, service, experience]?
    1. A rating scale question
  2. What would you change about [a product, service, experience, or your company] if you could?
    1. Open-ended question
  3. How easy was it for you to [do something] with our product?
    1. Options include a scale of extremely easy to extremely difficult
    2. This is a Likert scale question,
  4. How would you rate your experience with our new [a product, service, experience]]?
    1. This can be a Likert scale.
    2. Or, it can involve multiple choices of varying answers.

In this panel survey example, the business would be able to track patterns of customer happiness over time – for example, to see whether features continue to add value months or years after the customer signs up. And you can incentivize participation with gifts, reduced subscription fees, or extra services.

Consider discussing the following topics with your panel:

  1. What are their overall thoughts about your business?
  2. How do they receive product updates? (Poorly, well, etc.)
  3. What kind of value does your business provide and where does it lack in value?
  4. What else can you do to delight your customers?

Example #4: Employee Engagement

Another crucial panel survey example deals with studying your human capital, aka, your employees. After all, it is the employees that allow your business to function daily. 

employee engagement

As such, your business needs to track employee engagement and team morale. In essence, here, your staff are the members of the panel survey – and you might survey them every week, month, quarter, or year. In this case, you’re surveying the same people about issues that answer the following:

  • Are employees engaged at work, or do they have low morale?
  • Do employees enjoy their work?
  • Are they meeting their goals?
  • Do they understand the company's mission?
  • Do they understand the company roadmap and high-level goals for the coming weeks and quarters?
  • Are they happy with their compensation?
  • Where can the company improve?

If this is the type of panel survey you’re looking for, there are plenty of surveys that are specialized for human resources and employee engagement purposes

This includes the following surveys:

  1. The employee satisfaction survey
    1. Used specifically to measure satisfaction, this survey focuses on what customers like and dislike about their jobs and the company, 
  2. The employee feedback survey
    1. This is a general survey that should handle high-level topics. 
  3. The employee burnout survey
    1. This survey focuses on the negative aspects of employees’ jobs and your company. 
    2. You can use it to avoid burnout by asking them what they’d like to see change in their employment, workflows, job processes, schedules, etc.
  4. The employee recognition survey
    1. This is conducted to measure and understand the degree of employee recognition within a business. 
    2. It gauges the perception of recognition that employees feel they have from their managers, peers and the company at large.
  5. The eNPS survey
    1. This functions the same way as a Net Promoter Score survey but for employees.
    2. Employees are asked how likely they are to recommend their company to others, on a scale of 1-10.
  6. The employee retention survey
    1. This is used as an instrument to retain employees.
    2. It delves into the topics that business owners and HR workers need to understand, to forge ideal work  environments and cater to the needs of their employees,

Example #5: Customer Loyalty

It takes years to build solid consumer loyalty, which is a must, given that it is the core of the aforementioned customer lifetime value. By retaining your customers, you’ll spend less on attempting to acquire new ones, have a steady pool of revenue and maintain a good brand reputation

Your panel survey can help you keep track of your customer loyalty. As you gain their insights, you’ll discover how you can raise their loyalty to your brand. To do so, use a market research platform that offers a brand tracker.

This tool allows you to keep track of a variety of brand issues throughout the year and beyond. It enables you to uncover whether there are any shafts in your brand perception or reactions to your marketing campaigns.

Consider tracking the following on customer loyalty:

  1. What excites my target market about your offerings?
  2. What kind of brand news incites the most interest?
  3. Which segments of customers are most likely to make repeat purchases?
  4. Which customer groups are the most loyal to your brand?
  5. Which actions and offerings inspire loyalty to your brand?

Pros & Cons: The Advantages and Drawbacks of Panel Surveys

Now, it’s time to analyze the advantages and disadvantages of using panel surveys for your market research. 

One of the primary advantages of a panel survey is that the participants learn to trust the researchers, and therefore may be open to deeper and more truthful answers

In addition, this ongoing relationship can allow researchers to dig deeper with follow-up questions.

pros and cons of survey panels

Another advantage is that with screening having occurred at the start of the research, all following surveys are quicker to carry out than starting from scratch.

However, the disadvantages of panel surveys include:

  • Panel conditioning: When you’re surveying the same people repeatedly, previous surveys might influence their responses and/or behavior. For example, if they know a certain question triggers a longer process, they may choose an answer that offers the path of least resistance. Or if you’re asking about a particular activity, like eating donuts, they may eat less (or more) thereafter.

    Note: Furthermore, panel surveys don’t tend to occur within a “natural” setting, and therefore the artificial environment might affect thought processes and responses to questions.
  • Panel fatigue: If a person feels like a pay-off for participating in a panel survey is no longer worth the effort, the quality of their response may drop. This leads to incomplete or poor-quality survey responses, and survey “straight-lining” – i.e. answering the same to every question.
  • Declining participation: Over time, inevitably, participants will drop out of the process – either due to panel fatigue, or other reasons. This will damage the quality and depth of your data. 

And with survey panels, you run the risk of accidentally signing up “professional” panelists – especially when there are cash-based incentives on offer. Though screening should be designed to pick up these people before they enter the process, they’re adept at slipping through the net. 

Organic Sampling: A Better Alternative to Panel Surveys

With increased pressure on people’s time, survey response rates have been decreasing. It’s harder than ever to guarantee consistent high-quality data, and as conventional survey companies scramble to include as many willing participants as possible, the quality reduces yet further. 

At Pollfish, we don’t use conventional panels. Our unique methodology, called Random Device Engagement (RDE) uses the organic sampling approach for finding and procuring survey participants. 

organic sampling

A kind of organic probability sampling, RDE polling relies on advertising networks and other device portals to engage potential respondents wherever they are voluntarily. This includes a variety of digital platforms and properties, such as:

  1. Mobile sites
  2. Apps 
  3. Website 
  4. Mobile games

All respondents gathered via the RDE method have no biases, as they are not pre-recruited and therefore face less pressure to answer things in a certain way. Due to this, they are less subject to a wide host of survey biases.

This includes the following survey biases:

  1. Sampling bias: This bias occurs when the respondent selection process, specifically when a survey sample is not chosen at random.
    1. This leads to under or overrepresentation of a certain segment of your survey target population, as only certain types of respondents are going to partake, while others aren’t.
  2. Acquiesce bias: Also called agreement bias, this occurs when respondents gravitate towards positive or agreeable answers. 
    1. In this bias, respondents feel more social pressure to answer in a particular way, as their identities are known in panel surveys.

Organic sampling and Random Device Engagement obliterate these two (and other) biases. That’s because this process is completely randomized, stamping out sampling bias. Not only does it reach people randomly, but on Pollfish, this method is paired with our state-of-the-art survey platform. 

Our online survey platform reaches specific populations based on demographics, psychographics and specific screening questions you can input. This ensures you’ll reach all the required target audiences to partake in your study.

organic sampling
Our audience reaches over 250 million randomized respondents

Aside from reaching respondents organically and randomly, the respondents are completely anonymous. They don’t need to provide their names, contact information, or any identification to take a survey. Instead, they are simply reached and opted in. 

They may also have the option of gaining survey incentives. This eliminates virtually all forms of acquiescence bias, as there is no pressure to provide certain answers over others. 

Each screened participant has a unique ID, which prevents them from taking the same survey more than once, weeding out bias and survey fraud.

When working with Pollfish, you don’t need to use conventional panels – which are slow and subject to conditioning, fatigue, and declining participation. Instead, get authentic insights from our pool of over 250 million people (and growing) – who are already engaged on their devices and ready to participate. 

As such, organic sampling, coupled with a potent online survey platform like Pollfish, provides a better market research option over panel surveys. 

Frequently asked questions

What is a panel survey?

A panel survey is a type of survey method that involves repeated interviews of a group of people over a specific period of time.

What is a longitudinal study?

A longitudinal study is another term used to describe panel survey. It refers to any study that examines the same group of people over a period of time.

What are the benefits of panel surveys?

Panel participants often learn to trust the researcher and may reveal deeper, most honest insights. This type of survey is also a good way to track progress or evolution over time.

What is panel fatigue?

Panel fatigue occurs when the participant no longer feels that the reward for participating in the panel is worth it. If panel fatigue occurs, the participant may not give deep answers, but instead try to answer the questions as quickly as possible.

How can panel surveys influence participant behavior?

When interviewing the same people repeatedly, the answers to previous surveys may influence their behavior by causing them to reflect on activities that they previously gave little thought to. This influence is known as panel conditioning.

Survey Panels Vs Organic Sampling: Which is Better for Market Research?

Survey Panels Vs Organic Sampling: Which is Better for Market Research?

Survey panels and organic sampling are two of the major methods used to collect survey data. Since getting survey responses without the assistance of software is an almost impossible feat, these two methods have reigned supreme.

Both of these survey response mechanisms are unlikely to wane anytime soon, due to the prevalence of online surveys. As a matter of fact, the online survey software space has risen by 8% between 2015 and 2020.

With online surveys granting market researchers and marketers a large sweep of survey types to conduct, the point of contention becomes: which type of survey response collection data is better, survey panels or organic sampling?

This article will explore both forms of data collection so that you can objectively decide which to opt for in your survey research campaigns. 

Defining Online Survey Panels

Survey panels, also called online panels or research panels, all denote a data collection method in which responses are collected by way of pre-recruited and pre-screened respondents who agreed to take part in a survey.

This method helps businesses in that it assures them that a group of people will take their survey; usually, members of their target market are called upon to take the survey. 

There are a number of ways survey panels can be assembled. One such method involves mail-in recruitment, which has significantly declined in the digital age. Another relies on phone calls via Random-Digit Dial (RDD), a method in which respondents are chosen to take a survey from random telephone numbers. 

When it comes to recruiting survey panels digitally, they are garnered through an opt-in format such as a signup page or through an email invite that routes users to the survey page.  All of these recruitment manners have low participation, as few people opt into panels.

That is why some brands and market researchers resort to compensating their pre-screened panelists.

The Pros and Cons of Survey Panels

Now that you understand the basic methodology behind survey panels, you ought to consider their pros and cons before fully forming your opinion on whether to use them or not. 

The pros of online panels:

  1. They provide a network of respondents for continuous survey participation. This is especially useful if you need to conduct longitudinal studies. 
  2. They are inexpensive and create fast studies. 
  3. Returning to the same respondents, allows you to detect changing opinions over time, allowing you to see how your target market changes its opinions.
  4. They allow you to create informed custom polling questions based on previous research.

The cons:

  1. Repeated survey participation causes panel fatigue, a term denoting the decline in the quality of survey data, due to the boredom or exhaustion of a panelist. This creates them to inaccurately provide responses, either due to skipping questions, ticking off the “don’t know” option or rushing through a survey. 
  2. They occur in non-organic (unnatural) environments — inauthentic environments create inauthentic responses. This is because a survey’s environment can affect its’ respondents’ mindsets.
  3. Web panels gather respondents either on desktop or mobile, creating scenarios in which participants are dependent on device types. As such, the survey experience is not very adaptive. 
  4. Panel conditioning: Repeated survey participation can change respondents’ true attitudes, behaviors and knowledge. This makes it difficult to differentiate between actual changes and changes in reporting behaviors.

Random Device Engagement (RDE): A Precursor to Organic Sampling 

Before you analyze the organic sampling method, you should understand what makes it tick. That is because organic sampling is reliant on the delivery structure known as Random Device Sampling (RDE).

This framework implements intent-based behavioral targeting, typically used by advertisers, to narrow down random respondents in a digital setting, such as on websites, mobile sites and mobile apps. 

Thus, it provides a solution for randomization and capturing the correct audience. 

Random Device Sampling works by tracing the unique IDs of respondents, which are used to track them across devices. RDE, therefore, institutes a mechanism that is both random and organic. 

Organic Sampling Defined

Also called random organic sampling, this method refers to an RDE-based response collection method in which a survey is deployed randomly to users who are already in apps and other digital spaces.

Since surveys are randomly transferred, this method allows respondents to take the surveys while they are in their organic environments. These are the spaces that users spend time in organically, meaning they chose to take part in those environments, rather than being taken there via a promotion, incentive or signing up at a web panel.

Organic sampling works by giving optional invitations (or call-outs) to users in organic settings, so that they would partake in quick surveys. These invitations (along with the surveys themselves) are natively integrated within the digital environments (ex: apps) that the users are in.

This makes several benefits possible. 

The Pros and Cons of Organic Sampling

Powered by Random Device Engagement, organic sampling has many upper hands. However, like the panel survey method, organic sampling also presents certain disadvantages that you should carefully consider. You ought to weigh them against one another before deciding whether or not to use them. 

The Pros of Organic Sampling

  1. An organic environment allows you to avoid panel conditioning, allowing you to extract genuine responses from participants.
  2. It targets respondents’ unique IDs so that they can be traced even while shuffling between devices, preventing the same participants from taking the same survey twice. 
  3. Using respondents’ IDs allows you to create respondent profiles, which build an overview of the respondents’ behaviors demographics, which are critical data to possess.
  4. In turn, the respondent profile helps prevent fraud, as multiple accounts or bots won’t able to submit their surveys. 
  5. It yields high response rates (higher than RDD and survey panels).
  6. It creates a seamless UX across all device types, rather than being tied to just one with little wiggle room. 
  7. It provides vast coverage, as RDE integrates natively with a bevy of digital and mobile platforms. That entails greater accuracy. 

The Cons:

  1. It relies on non-probability sampling, wherein some of the members of a population have been excluded and this amount cannot be calculated, which limits how much you can determine about the population from the sample.
  2. It will include biases based on the populations you have preset to include in your sample, even if you assign quotas.
  3. Organic sampling and RDE are at the mercy of the websites and apps that they can be integrated with. It is possible that your targeted sample pool visits apps and other digital places that your RDE survey isn't integrated with.

Which Sample Pool Method Reigns Supreme?

While random organic sampling has made a mark in the survey realm, due to Random Device Engagement, it still faces its rivalrous counterpart: survey panels. 

Other forms of survey sampling methods have been on the decline, yet survey panels are still in use. While they may appear to have fewer advantages and more disadvantages, survey panels still provide value to researchers. 

As such, it is entirely up to you to decide which survey sampling method is best for your market research needs. Random organic sampling is our survey sampling method of choice, as it continues to rise above survey challenges, provides solid results and does so in a short span of time.

Frequently asked questions

What is an online survey panel?

Also called online panels or research panels, an online survey panel is a way of repeatedly collecting data from pre-recruited respondents via digital surveys.

What are some of the benefits of an online survey panel?

Online survey panels allow companies to quickly and inexpensively collect data from a group of people since the recruitment and screening process only needs to be performed once. Since responses are collected from the same group over time, they are able to show if and how opinions or behaviors change over.

What is panel fatigue?

Panel fatigue is a concern associated with survey panels. It occurs when panel participants become bored or tired of replying to surveys. This can result in the overall inaccuracy of the data collected from that participant.

What is organic sampling?

Organic sampling, or random organic sampling, refers to the process of distributing surveys to a random group of respondents. The respondents are typically sourced from websites or apps where they are given the option to participate in a survey.

What are the benefits of organic sampling?

Organic sampling can help prevent some of the pitfalls of survey panels including panel fatigue and panel conditioning. Organic sampling also tends to yield higher response rates and can help prevent fraud or data errors.

How to Get Insight from Consumer Panel Surveys: 10 Tips

How to Get Insight from Consumer Panel Surveys: 10 Tips

Alongside focus groups and ad-hoc questionnaires, consumer panel surveys are a popular tool for businesses looking to gain insight into what their audiences – and consumers at large – are thinking.

Using pre-recruited and pre-screened groups (or panels) of respondents, consumer panel surveys have specific advantages over alternative consumer research methods. You can control your audience, for example, and return to the panel over time to collect data repeatedly from the same set of people. This lets you dig deep into highly targeted audiences – and, crucially, compare changes in attitude over time.

Yet, as with all research methods, to get real value from consumer panel surveys, you need to refine your technique. That’s why, in this article, we’re sharing 10 tips and best practices to get the most from your consumer panel surveys. We’re talking how to shrewdly recruit and screen respondents, how to ask questions that elicit authentic opinion, and how to use the right analytical methods to yield deep insight.

Note: Don’t forget, Pollfish has created a new consumer research methodology that works a little differently. One that fulfills the promise of consumer panel surveys, while keeping bad data to a minimum – and making access to deep insights easier and more efficient. 

Where to Start with Consumer Panel Surveys?

Consumer research can deliver highly valuable insight for your business. Whether you want to test how an ad campaign will land, or get feedback on a new product line, opinion from authentic consumers – and potential customers – can be priceless.

Yet, you can’t just go and fire questions around at random. This won’t tell you anything of value. Rather, thinking through who you are asking and why is crucial before you start. 

Here are three tips on how to get that done.

#1: Gain Clarity on Your Research Problem and Goal

We’re starting with the basics, yes. But a true understanding of what you want from your research is an essential element that too often goes overlooked, particularly by businesses conducting research themselves. If you want insights that have value for you, you need absolute clarity on your larger goals.

Do you want to boost sales, refine a specific product, or determine brand positioning? A robust research objective will help you sculpt questions that deliver sharper insights – and save you time asking questions that don’t deliver value.

#2: Refine Your Research’s Scope and Scale

Once armed with a research objective, you now need to assess the desired scale of your research. This can get a little complex – but it ultimately shapes the reliability of your research.

Of course, the ideal survey would consult 100% of your target population – whether that’s marketing graduates in Scotland or cyclists among your company staff. While this scale would yield data on which you can surely rely, this is usually not possible in practice. Even if your target population is very small.

However, the further from 100% of the population, the greater the likelihood of inaccuracy. What you need to decide as a business is what margin of error you can tolerate. If you are making high-stakes decisions, you want data you can trust – and the larger the sample size the better.

#3 Source and Speak to the Right People

For authentic insights that deliver value for you, speak to the right audience. You can’t take this for granted. Before you conduct any survey, you need to recruit respondents that suit your target demographic.

This is what screening questions are for. These filter out respondents who aren’t right for you. However, as our resident expert, Jim Theodoropoulos, notes, using screening questions can be difficult to get right. These need to be combined with demographic targeting to narrow in on your audience.

Using Pollfish, for example, if you are seeking mothers who are runners, you can use the “gender” and “number of children” filters, before asking a screening question about preferred exercise. Like this, you can access deeper insights into specific audiences – and keep the professional panelists to a minimum.

Asking Questions that Deliver Real Value

The way you ask questions – and the form in which you receive answers – affects the quality of your data. And avoiding bias and panel fatigue, for example, can make conducting consumer panel surveys a bit of a minefield.

This is why many businesses use options like Pollfish to help design the questions. But if you’re going it alone, below are some tips on how to ask good survey questions and get answers you can trust. 

Note: For more on this, check out our article on how to ask good survey questions.

#4 Ensure Neutrality

Bias is a bad look in consumer research. For surveys to yield data that is actually valuable, they need to tap into the authentic opinion of respondents. Not reflect the prejudices of the researchers. 

That means avoiding leading questions, those questions that direct respondents to specific answers. For example, “How would you rate our award-winning customer service?” has the potential to skew responses by including the qualifier “award-winning”. 

However, even “what problems do you see in our customer service?” distorts the insights you will receive – as it assumes problems, whereas none may have been seen by the respondent.

This also means providing multiple-choice responses that reflect the range of opinion. If “how would you rate our customer service?” is your neutral question, the possible responses need to achieve similar neutrality. “Extremely helpful / Helpful / Neither helpful or unhelpful / Unhelpful / Extremely unhelpful”, for example, keeps the balance between positive and negative options while keeping framing consistent.

#5 Use Strategies to Mitigate Panel Fatigue

During surveys, and particularly during panel surveys, it’s not unlikely that respondents will become bored or disengaged – even though they sign up to participate.

When respondents are asked too many questions, we call this panel fatigue. It’s one of the biggest downsides to consumer panel surveys. Respondents might lean on “don’t know” or they might start “straightlining” – where they choose the same response for every question.

To avoid this, change up your question formats and shuffle questions around. It boosts engagement. It ensures more trustworthy responses. And it ups your completion rate – making data more reliable overall.

#6 Be Specific – and Remember Your Research Goal

Every question in your survey should provide an insight into your audience’s opinions, attitudes, and behaviors. Those that are irrelevant or too vague are not going to deliver the insights you need to make effective business decisions.

In this way, if you are a gaming app, an open question like “what do you like to do in your downtime?” is not focused enough – and it doesn’t direct respondents’ thoughts to gaming. As such, it won’t provide data that will be of use to your research goal.

Understanding Insights from Consumer Panel Research

You’ve asked the questions. Now, you need to understand the responses. 

This will first mean collating recurring answers, identifying patterns, and highlighting anomalies – a process that can be hugely time-consuming if you are doing it alone. That’s why Pollfish, by the way, delivers automated analytics in real-time. More on that later.

Once you’ve arranged the data, you need to analyze it. Here’s how to get started.

#7 Compare Your Data to Larger Trends

Numbers, let’s remember, are all relative. And getting valuable insights from them is difficult when they are viewed in isolation.

Your survey tells you 80% of respondents said that they were happy with your customer service. But for this to be valuable to you, this needs to be put in context. How does this compare, for example, to other businesses of your size in your industry? Or to your results from last year?

Larger trends are your friends – and they help make sense of your customer panel survey. If this is your first survey of its kind, let these results be the benchmark from which you understand future developments. If you have numbers from previous surveys, use these to assess your progress.

#8 Cross-Tabulate – and Dig Deep

Similarly, when you are reading your results, the big picture isn’t always the most useful view.

Say that 65% of respondents in a general population survey said they found your new ad campaign funny or very funny. That might feel like a win. However, these numbers could be disguising insights of more specific value.

That’s what cross-tabulation is about. Using crosstabs can let you see how different data fields overlap – and this is an indispensable tool for analyzing your data. For example, maybe the ad landed much less effectively with women. Or the joke was completely lost on 21-30-year olds. 

This matters. There may be an insight you need to address. You just need to dig deeper to find it.

#9 Keep on Researching

In a world in which business competition is sharp, a single survey won’t do. As a result, there’s one tip that needs to be stressed: keep returning to your audience to do more research.

The unique value of consumer panel research is that going back and asking questions is easy. Your panelists are there, pre-recruited, ready, and willing to respond. Make use of them. There is really no such thing as too much data.

#10 Use Pollfish to Make Deeper Insights Easier

At Pollfish, we’ve moved beyond the traditional consumer panel survey. We’ve developed a research methodology that makes gathering insights into consumer opinion easier. It makes recruiting panelists and understanding insights faster, more cost-effective, and less labor-intensive, too. 

For example, our consumer research platform sources insights from over half a billion engaged consumers, connected via our network of 120,000 app providers. We screen every participant based on pre-collected demographic, biographic, and behavioral data – as well as your screening questions – to take the stress out of finding the right audience.

Pollfish makes setting research objectives and framing questions easier too. We give you some of the most common research goals and pre-crafted question templates to choose from. To keep bias to a minimum and ensure participant engagement, all you have to do is select.

Finally, analyzing data is easier with Pollfish, too. Our advanced data tools let you filter your findings in real-time. All admin is taken care of, so that you can dedicate yourself to visualizing your insights – and ultimately putting them to work.

Frequently asked questions

What is a consumer panel survey?

A consumer panel survey is a type of survey that is conducted at set intervals over a period of time with the same group of respondents. The time period can range from days to years.

Why are consumer panel surveys used?

With interviews conducted over a period of time, consumer panel surveys are ideal for understanding how the thoughts, feelings, or behaviors of people evolve over time.

What are the benefits of conducting a consumer panel survey?

One of the primary benefits of consumer panel surveys is that they can be used to gain deep insights about a highly targeted audience. In addition, individual surveys are easy to deploy because the respondents have already been identified.

How are screening questions used?

Screening questions are used to identify the appropriate audience or panel for a survey. Combined with demographic targeting, these questions can help identify the best audience for a survey.

What is panel fatigue?

Panel fatigue occurs when survey respondents become bored or disengaged from the survey process, resulting in mediocre or inaccurate data.