All NEW Pollfish AI Builder: The Smarter Way to Create Surveys
đĄ Revolutionizing Survey Creation with AI
At Pollfish, weâre always looking for ways to make survey creation faster, easier, and smarter.
Thatâs why weâre thrilled to introduce a cutting-edge revamp to AI Builder, our next-generation AI-powered survey creation tool.
It's free to use for all of our customers. So whether youâre a seasoned researcher or just getting started, our new AI Builder can help you craft high-quality surveys with expert precision and minimal effort.
What is AI Builder?
Taking pride of place at the top of your survey dashboard, AI Builder is our conversational tool that enhances your survey creation process by offering real-time suggestions, modifications, and optimizations to your projects. With AI Builder, you can:
- Generate professional-quality surveys from a simple prompt.
- Modify, rephrase, or expand questions with expert survey guidance.
- Change the tone, spell check and translate your survey for different audiences.
- Adjust response options by adding, removing, or shuffling answers.
- Undo previous AI modifications for full control over your questionnaire.
Supporting Advanced Research Methods
AI Builder isnât just about creating questions - itâs designed to support a variety of research methodologies, including:
Plus, it supports a wide range of question types, from single and multiple selections to matrix, ranking, sliders, open-ended responses, and more.
Whatâs Next for AI Builder?
We're just getting started! Our dedicated team of Pollfish AI developers are continuously iterating the product to lead AI in the market research industry. So you'll see it's speed and capabilities increase quickly over time.
Upcoming enhancements include:
- Question Groups â Create structured sets of questions for deeper insights.
- Advanced Logic â Set up complex branching to refine survey flow.
What functionality would you like to see? Let us know over on Linkedin and we'll feed it straight back to the team...
Try AI Builder Today
Our all new AI Builder is now live in the Pollfish platform! The next time you log in to your dashboard or create a survey, look for the âAsk AIâ button to start building smarter, faster, and with greater precision than ever before.
And of course weâd love to hear your feedback...
How are you using AI Builder?
What additional features would make it even more powerful for your research needs?
Share your thoughts in the comments and help shape the future of AI-powered survey creation, together.
Ready to experience the future of survey creation?
Log in to Pollfish and try AI Builder today!
Save & Restore Surveys w/ Versioning!
đ ď¸ Save & Restore Surveys w/ Versioning!
Ever made a ton of changes to your survey and wished you could go back to a previous version? Sort of how you can in Google Docs? Now you can!
It never fails⌠your heads down making changes to your survey and realize you went a bit too far⌠ever wish you could view that previous version?! Why oh why do I have to start all over?! Good news - with Pollfishâs new Survey Versioning tool you don't have to.
With Pollfishâs Survey Versioning, you can save, restore, and manage multiple versions of your survey seamlessly. You can save up to 5 snapshots of your survey while itâs in draft status and restore any of them in seconds. Plus, weâve got temporary autosaves happening every 3 minutes to make sure nothing gets lost.
How Survey Versioning Works:
See your current version: From the questionnaire builder page, simply click on the "rewind clock" icon in the top right corner. A popup will display on the right showing your current version and any past autosaves. In the case of a brand new survey, only the current version will show.
Find your autosaved versions: Once you've made a change, you see your first autosaved version appear, and every 3 minutes, another one will appear. Autosaves max out at 5 total, however you can manually save as many versions as you'd like.
Naming, restoring, and deleting versions: If you click on a previously saved or autosaved version, it will display that older version of your survey on the left (which is shown below). From here, you'll also see 3 options pop up. You can choose to name the version (which additional saves it), restore the version (which makes it your current version again), or delete the version.
Confirmation of naming, restoring, and deleting: If you click to delete or restore, you will be prompted to confirm the change. If you click to rename a version, you'll be asked to pick a name for the version, and optionally add a description of the version.
Versioning history: As you continue, all of your saved versions will be filed in one folder, with your autosaves saved in another folder.
This versioning upgrade can be especially beneficial when you're collaborating with others in the same Pollfish account, trying to craft the best possible questionnaire.
Try Questionnaire Versioning with Pollfish!
Questionnaire versioning allows you to save, restore, and manage multiple versions of your survey effortlessly. Instead of worrying about making irreversible changes, you can capture and go back to any previous version of your questionnaire. Pollfish automatically creates temporary autosave points to protect your work, ensuring nothing is lost.
In summary, with Pollfish Questionnaire Versioning, you can:
- Save up to 5 snapshots per survey while itâs in draft status.
- Restore any previous version of your questionnaire with a simple click.
- Enjoy automatic autosaves every 3 minutes to avoid losing any changes.
- Manage changes easily, with the flexibility to experiment and fine-tune your survey.
Log in to your Pollfish account to try Questionnaire Versioning now!
Target Granularly w/ Nested Quotas!
𧊠Target Granularly w/ Nested Quotas!
Picture this. You get your survey results, and drill down into males vs. females, and then down into rural vs. urban, but, âoh no!â, of 400, thereâs 367 in one segment, and only 33 in the other!
With Pollfish Nested Quotas, you can ensure you get enough respondents in any selectable demographic or behavioral subgroup. Nesting can go 3 levels deep, supports exclusions, and can be performed with grouped or individual options.
Create Nested Quotas in 4 steps:
Step 1: Select the first demographic/behavior you want to nest with. You can do this by simply scrolling to the desired dropdown selection from the targeting criteria on your "audiences" tab from within your survey. There will be a small grey "merge" icon on the right of all listed criteria. Simply click it to begin a nested quota.
Step 2: Select the second demographic/behavior you want to nest with. For instance, if you want to create a nested quota containing answer selections within both "marital status" and "gender", you can either search or scroll to "gender" and select it.
Step 3: Select your answer choices from each of your nesting selections. Please note: If one of your demographic/behavior selections has only 2 choices, both will be automatically selected.
Step 4: Edit your nested quotas to the desired # of respondents per each subgroup. In most cases, you'll want an even distribution between the quotas, which is why it will preset in that way. However, you can edit as needed for your particular use case.
That's it!
Try Nested Quotas with Pollfish!
Nested quotas allow you to precisely target specific subgroups in your survey by layering demographic and behavioral criteria. Instead of just selecting broad categories (like age or gender), you can combine and intersect criteria, ensuring proportional representation across deeper levels. This helps avoid oversampling one group while leaving others underrepresented.
In summary, with Pollfish nesting, you can:
- Ensure balanced quotas within multiple demographic layers (e.g., age + gender + location).
- Prevent bias in survey results by guaranteeing sufficient responses from key subgroups.
- Support exclusions and more flexible quota management.
- Maximize accuracy in data collection by refining audience targeting.
Log in to your Pollfish account to try Nested Quotas now!
Prodege & Pollfish Unite to Improve Audience Quality
Improved Audience Quality + 6,979 New Behavior Targeting Criteria! đŻ
Many of you have asked and weâve listened. Weâve just finished a major upgrade to Pollfish audiences to empower our research community like never before. 𼳠Starting today, you'll gain the following benefits within the platform:
5 Ways This Enhances Your Research Experience â¨
- Reach niche audiences using
6,979 unique validated behaviors. - Access to Prodegeâs tried and
tested loyalty program members. - Quality checks that meet
ESOMAR Gold Standard. - Combine behaviors and sentiment
targets to understand the âwhyâ. - Recontact respondents for
longitudinal tracking.
"So, exactly what can I target?"
I'm glad you asked!
4 New Behavioral Targeting Criteria đ
The new behaviors are best bucketed into the four following categories, and can be drilled down into sub-categories, and even down to specific stores, brands, sites, and locations.
Brand Purchases đď¸ | Consumer Lifestyle 𧏠| Website Visits đ˛ | Location Visits đŹ |
- Ruffles - Dr. Pepper -Angel Soft - Tabasco - Etc. | - Finance - Travel - Wellness - Gaming - Etc. | - BuyBuyBaby - CNN - Quora Etc. | - Target - Costco - 7-Eleven - Papa Johns - Etc. |
The full list of behavioral targeting criteria can be seen here: Google Sheet: 2024 Audience Upgrade 2024
Selecting Behavioral Audiences in 2 Clicks â
Click 1: From the survey creation page simply navigate to the audiences tab. There you will find two new sections: Consumer Lifestyle and Behavioral Data. Expand the dropdown and click a category.
Click 2: Youâll now be shown that category on the left, in which case you can type or select from thousands of validated website visits as shown in the image below:
In addition to this, you can even add quotas to your behavioral selections, or suppress specific selections to get very specific respondents for your surveys!
Preview All Targeting Criteria đŽ
To preview all of the targeting criteria, access the massive list of 6,979 behaviors here:
Google Sheet: 2024 Audience Upgrade 2024
Already a user? Some changes you'll see in the platform.
To make these upgrades, we've had to make some other changes within the platform.
You'll notice:
- Screening questions now live under the "questionnaire page"
- Completion time has been switched to a yes or no feasibility calculation.
First-party behavior data, powered by Prodege.
This is all now possible thanks to Prodegeâs unique blend of owned and operated publishers, to help you achieve granularity from a more representative and reliable data source. The Prodege proprietary panels are Swagbucks, InboxDollars, MyPoints, and Upromise. Each panel has a unique profile given they cater to people with different interests and because our recruiting methods and incentives differ for each, allowing for a more even and representative audience.
Validated behavior are collected with member consent and in accordance with privacy laws from app and web browsing, supermarket receipts submitted by members and verified online purchases.
Ready to try our new behaviors? They're available right now on our platform. Simply select the targeting criteria you need, and let us do the rest! Whether you're a seasoned Pollfish user or new to our platform, our team of experts is here to help you.
More Than DIY â New Services on Our Platform!
New Services on Our Platform! đ
Get ready for some exciting news! Pollfish, your go-to DIY market research platform, has just leveled up! Thanks to our partnership with Prodege, we now offer additional services directly through our platform. This means that Pollfish clients can now access the whole suite of Prodege market research services in a snap!
đ¤ Pollfish & Prodege: A Dynamic Duo
While some of you are familiar with our relationship with Prodege, many of you may not be aware of the valuable services they offer. To make life easier, we've integrated a section in our platform for users to select Prodege services directly. Let's take a closer look at what's new!
đ˘ Announcing 7 New Services at Your Fingertips!
đ¤ Qualitative / IHUT Recruitment - Need engaged participants for online qualitative research, focus groups, IDIs, or IHUTs? Our sampling experts have your back! Ensure the right audience is recruited for valuable insights, improving research reliability.
đ Questionnaire Design Support - Optimize your questionnaire design with the help of our team of experts! Achieve better data quality and reduce potential biases through a well-designed questionnaire.
đŠ Sample Management - Let our team set up quotas (including nested quotas) and manage fieldwork to ensure you reach your target audience. Streamline fieldwork and reduce sampling errors for more accurate and representative data.
đť Survey Programming - Need to program a survey from a document? Our in-house team provides quick turnaround on surveys from simple to complex. Save time and effort while ensuring your survey is programmed accurately and efficiently.
đ Translations - Have your surveys and open-ended responses translated/back-translated into 50+ languages by our team of native speakers. Reach a global audience and enhance cross-cultural understanding with accurate translations.
đ Data Tables, Visualization & Dashboards - Create custom, branded dashboards or PowerPoint charts and graphs using our innovative Insights Builder tool. Make your data more understandable and visually appealing for impactful presentations.
đ Topline Insights Report - Collaborate with our insights experts to generate topline reports with key findings from your survey data. Quickly identify essential trends and findings, simplifying data-driven decision making.
DIY to DI-WHY-Not: Expanding Your Capabilities
We know that Pollfish users are used to doing things themselves. However, we also understand that you may be pressed for time, need help, or require market research services beyond our capabilities. With our new Prodege-powered services, you no longer have to choose between DIY and expert assistance. You can have it all! By teaming up with Prodege, we're providing you with a seamless, end-to-end market research experience that saves you time đ, money đ°, and resources đł.
How It WorksÂ
As youâll see in the screenshot below, all you need to do is select services tab on the left when you're logged into the Pollfish platform, and you'll be able to select from these 7 services, in which you'll be connected with the appropriate expert:
A Celebration of Collaboration
Ready to try our new services? They're available right now on our platform. Simply select the Prodege services you need, and let us do the rest! Whether you're a seasoned Pollfish user or new to our platform, our team of experts is here to help you take your market research to the next level.
Let's make market research more accessible, efficient, and enjoyable together! Don't miss out on this amazing opportunity to enhance your research capabilities and achieve even greater success with Pollfish and Prodege! đĽł
Pollfish's New Platform Design
Pollfish's New Platform Design
Pollfish has revamped its user interface to enhance the experience for survey creators and to pave the way for numerous exciting features this year.Â
There are two main changes.
Update #1: New survey summary page.
Section 1: Survey Overview.
Need a quick overview of your survey? This section provides information on survey status. Depending on your survey type, this may include statuses such as draft, pending, paused, active, completed. You will also find other relevant information such as incidence rate (IR), estimated length of interview (LOI), completion rate, languages, translations, a brief preview, and easy access to downloads.
Section 2: Survey Performance.
Looking for insight into completion time? This section helps you visualize your survey's completion rate with a straightforward time series chart, and additional details on completion.
Section 3: Audience Snapshot.
Want to verify your target audience quickly? Obtain a concise snapshot of the selected audience(s), including all preselected demographic targeting criteria.
Section 4: Status Funnel.
Curious about drop-offs in your survey? This section enables you to see the number of people reaching each stage of the survey. Depending on your survey this may include stages such as accepted, passed profiling, passed screening, started survey, and finished survey.
Update #2: New menu navigation.
You might notice a subtle change in Pollfish's layout. Folders and surveys now appear in a panel on the left, while account details have moved from the top-right corner to the bottom-left corner.
This update allows direct access to surveys nested in folders from the side panel and, more importantly, prepares the interface for advantageous features arriving this year.
A New Data Visualization Tool for Surveys
Meet Insights Builder, the New Data Visualization Tool for Pollfish Surveys
Want an easier, more colorful way to analyze and present your Pollfish survey data?
Look no further! With our new integration to the Prodege Insights Builder, you can enhance your survey analysis by adding valuable data visualization to your survey data.Â
Add Data Visualization with a Single Click.
Easily add data visualization to your next survey.
- Open a survey in your Pollfish dashboard.
- Click the âInsights Builderâ button.
It really is that simple.
Analyze, Visualize, & Share Results.
With this new data visualization tool, you can:
- Analyze:
- Use a custom interactive online database to display key insights.
- Cross-tabulate, add filter logic, create new variables, apply stat testing.
- Visualize:
- Illustrate your business objectives clearly with visualization techniques.
- Craft rich and easily edited slide visuals in your organization's branding.
- Share:
- Collaborate with your team internally through Pollfish by sharing directly.
- Control access and broadly share with internal and external stakeholders.
How "Insights Builder" Works
Only researchers in the Elite plan can access the Insights Builder tool via the Pollfish interface.
The export to Insights Builder can be initiated per survey by the researchers, but not all surveys are automatically synced. Once initiated, all survey updates (new responses received, new revisions added, survey renames, etc.) are synced in real-time.
First-time survey export
For a survey that has not been synced to the data viz tool, the researcher can view the âInsights Builderâ option on the survey menu and the results page. A new window will ask you to continue, which will then redirect you to a new tab, once you click it.
This action will:
- Place the survey data into the data visualization tool.
- Open a new tab in your browser; this will contain the data visualization folder created for a specific survey.Â
- This means you can return to Pollfish whenever you wish.
- It will authenticate you to the data visualization tool via your Pollfish account, seamlessly.
- You wonât be asked to log in with your credentials to Data Viz.
- The âInsights Builderâ is an option available throughout the lifecycle of the survey (from the status of running to paused/ completed/ finished).
Viewing an exported survey:
- When a survey has been synced to the tool, you can view the âInsights Builderâ option in the survey menu and on the results page.Â
- A loader screen will appear to redirect you to a new tab.
Get Started With Data Visualization Now
Ready to get started?
Log into your account and try it out!
Your support doesn't stop here. Prodege can provide additional support around leveraging data visualization with your Pollfish surveys and for all sorts of additional insights needs.Â
Major upgrade to Pollfish AI survey builder
Major upgrade to Pollfish AI survey builder
Attention all researchers, marketers, and data enthusiasts! Pollfish's AI Survey Builder just got even better! We're proud to introduce our latest upgrade, which includes Maxdiff and Conjoint Analysis questions in addition to our existing arsenal of question types. The days of laboriously creating surveys are over â the Pollfish AI Survey Builder has got you covered!
The More, The Merrier: New Question Types Galore đ
Our upgraded AI Survey Builder now offers an impressive array of question types:
- Single Selection
- Multiple Selection
- Numeric Open-ended
- Text Open-ended
- Rating
- Ranking
- Sliders
- Maxdiff (Maximum Difference Scaling) đ
- Conjoint Analysis đ
With this major upgrade, your surveys can be more comprehensive and engaging than ever, allowing you to capture a wide range of insights from your target audience.
How It Works đ
As you'll see in this example below, all you need to do is select the type of survey you'd like to create.
On the left, you'll find a dropdown below where you'd normally enter your prompt, with options for:
- Main question types
- Conjoint method
- Maxdiff method
Simply pick the type of survey to auto create, and let the magic happen!
Pretty cool, right?
A Whopping 42,978 Survey Prompts Entered & Counting! đ
In just three months, we've had a jaw-dropping 42,978 survey prompts entered into the Pollfish AI Survey Builder! This outstanding number speaks volumes about the trust and value users place in our AI-powered tool. And now, we're taking it up a notch with the introduction of Maxdiff and Conjoint Analysis questions in our latest upgrade. Say goodbye to tedious survey creation â the Pollfish AI Survey Builder has your back!
In Conclusion: Join the AI Revolution Today!
It's time to embrace the future of survey creation! With the latest upgrade to our Pollfish AI Survey Builder, you'll be able to enjoy more the in-depth research capabilities you're already used to... But it will be:
- Way faster to create - thanks to Pollfish AI 2.0.
- Way faster to run - thanks to the massive global reach of the Pollfish audience.
- And soon, way faster to interpret your results - thanks to planned future feature releases.
So, what are you waiting for? Join the AI revolution and try out the upgraded Pollfish AI Survey Builder today!
How to Leverage Data from Conjoint Analysis Exports
How to Leverage Data from Conjoint Analysis Exports
Letâs take another critical dive into conjoint analysis, a super informative market research feature that we released last year.
As you know, a Conjoint Analysis is a kind of tool and research technique that enables you to measure the value that consumers place on the individual features of a product or service.
This kind of analysis is particularly beneficial for product and pricing research, as it unveils a plethora of consumer preferences. You can then use this information to optimize your products and services.
In this article, we specifically focus on Conjoint Analysis exports on the Pollfish platform.
The Polfish market research platform generates optimized Excel, CSV and SPSS files that contain raw data, which you can use to form your own analyses and import the data from the tool to other software.
If you want to run your own Conjoint Analysis to compute and understand part-worth importance, youâll need to use its export, as it contains raw data from your Conjoint Analysis.
This article teaches you how to do so.Â
1. How does Pollfish generate the Conjoint Analysis design?Â
The conceptual model of a conjoint analysis is pretty straightforward; it suggests that the utility of a multi-attributed product can be broken down into specific contributions of each attribute and their interactions.Â
The approach is easy to implement if the number of attributes is small. However, problems arise because of the large number of possible hypothetical alternatives for a given product.Â
For practical reasons, only a subset of possible alternatives is chosen for the study. To that end, experimental design methods exist for selecting good subsets of product configurations for performing the analysis.
Respondents will see a subset of possible options to choose from, which depends on the number of attributes and levels defined.Â
In order to do this, the Pollfish Conjoint algorithm first creates an orthogonal design of the alternatives (product bundles) that are formed by combining the levels of the attributes.
In the case that all possible combinations are more than 20, the total number of alternatives gets reduced using d and g optimality criteria.Â
Then choice sets are created, containing an upper limit of 5 alternatives each. The algorithms that we are using ensure the d-optimality of the choice-set design.
Finally, choice sets are assigned to blocks. One block cannot contain more than 30 choice sets.
The goal for our Pollfish Conjoint algorithm is to ensure that each level in each attribute appears the same number of times and that each alternative appears the same number of times in the design, so that the results have 0 bias and variance.Â
Each respondent gets assigned to only one block randomly. Blocks are distributed evenly to the audience, so that every choice set in the design is seen by the same number of respondents.
2. How is the Conjoint Analysis design distributed? Via the Pollfish algorithm based on Block Designs
A problem that usually arises when the choice-set design is ready to be administered to a target population is that the number of choice sets in the design may be too large for an individual to assess.Â
In these cases, a common technique is to partition the choice sets into blocks of equal size, in principle much smaller than the size of the original design, and then administer the blocks to the population.Â
That way, each individual would have to assess only the choice sets of one block, while collectively the population would assess all the choice sets of interest. However, in order for this partitioning to make sense, some constraints must be satisfied.Â
Each choice set should be evaluated by the same number of users. That means that each choice set should appear the same number of times into the blocks. Note that it is not necessary for a choice set to appear in all blocks. To that end, we may need to replicate the choice sets in order to always be able to divide them into blocks.
Formally speaking, the following equation should be satisfied: vr=bk
where v is the number of choice sets, and r is the replication factor of each choice set, is the number of blocks and is the size of each block, that is, how many choice sets each individual should see. Thus, given a choice-set design of size, we have to find the three smallest numbers () satisfying the above equality. Given these numbers, we can safely partition choice sets into blocks.
3. Understanding the Excel structure
The Excel (CSV and SPSS) file is a complementary feature of the visualized results at the Pollfish dashboard. It contains all responses given by each respondent of the survey. Graphs on the results page are based on these individuals' responses.Â
To take advantage of the Excel for a Conjoint Analysis or MaxDiff Analysis survey, visit the results page of a completed one, select âExport resultsâ and choose âExcelâ as your export option. You will be sent an email that notifies you when your export is complete. Open the document in Excel or Google Sheets to get started.Â
The Excel sheet will include the following two (among the other tabs):
- Experimental design tab: This tab contains the Conjoint or MaxDiffâs design, as generated by the Pollfish Conjoint algorithm.Â
- Individuals tab: This tab contains the selections given to the questions of the survey by each respondent.Â
For the case of the CSV and the SPSS, the experimental design comes in a separate file.
A. The Îxperimental Design Tab
Letâs assume that we have a Conjoint Analysis survey with the following attributes and levels, describing alternatives of detergents:
In the Excel tab, each component is listed in a column and has a unique ID. Blocks, their containing choice sets and alternatives(or âConceptsâ) have their own IDs, as the following screen depicts.
For the specific example, the Pollfish algorithm will generate a design of 4 blocks of 5 choice sets each.
Each choice set contains 5 alternatives (or âConceptsâ). Each row represents a generated alternative (or âConceptâ), whose containing attribute levelsâ columns are located after the âConceptâ column (in our example of detergents Conjoint test, the attributes were 4: Type, Perfume, Quantity and Price).
The settings for the number of choice sets and the number of alternatives, are also displayed in the questionnaire while preparing the Conjoint survey:
B. Conjoint selections at the Individuals tabÂ
Each respondent gets only one of the generated blocks. The blocks are evenly distributed across the sample, hence, a block may be received from many respondents.Â
In our example, from the 120 total sample, each group of 30 respondents will get one of the 4 blocks:
- 30 respondents receive the block with ID 257
- 30 respondents receive the block with ID 258
- 30 respondents receive the block with ID 259
- And 30 respondents receive the block with ID 260
At the Individuals tab, each row represents the responses given to the questions by a single respondent. Data related to the Conjoint Analysis are placed among the other questions contained in the survey and start with the column named âBlockâ which holds the block ID the respondent received.Â
Then the choice-set ID columns follow along with the selected alternative (or Concept) ID columns. The choice-set columns are ordered according to the order the respondent received them.Â
So, ChoiceSet0 ID holds the first choice-set ID the respondent got. Selected Concept 0 ID holds the respondentâs selected alternative ID(or Concept)Â in the first choice set and so forth until selections for all choice sets are provided in the tab.Â
C. Correlate the data at both tabs for further analysis
You do so by combining the following:
- the data from the Experimental design tab, which holds the content for each block and their included choice-setsÂ
- the data from the Individuals tab, which hold the selections per respondent
You can then proceed by then analyzing the data in a different system.Â
How Pollfish Provides Quality Data and Prevents Survey Fraud (2023 Update)
How Pollfish Provides Quality Data and Prevents Survey Fraud (2023 Update)
Pollfish implements a variety of procedures to ensure the highest quality of data. Using a combination of proprietary techniques and machine learning technology, our platform, along with our team of technical experts prevent survey fraud at every turn.Â
When combined with our rigorous quality checks, our participant pool of over 800M+ enables us to discard responses that donât meet our quality standards â without sacrificing the sample size. As such, no question is subject to survey fraud, as the Pollfish platform stamps out inaccurate or poor quality responses.
As a result, we deliver higher quality data than our less-selective, panel-based counterparts.
What is Survey Fraud?
Survey fraud, or market research fraud is the adverse phenomenon that occurs when survey respondents submit fraudulent or bogus responses. This can occur accidentally, such as when responders undergo survey fatigue, or purposefully.
What would prompt a responder to partake in the latter? They can be your competitors (especially if the survey mentions your brand or makes it apparent that it is conducted by your brand), bots, click farms, or respondents eager to finish the survey to receive an incentive.Â
Thus, there are a number of ways respondents can contribute to survey fraud, such:Â
- Providing nonsensical, i.e., gibberish answers
- Breaking rules
- Answering suspiciously quickly
- Hiding their IP address via the use of a VPN
- Leaving one-word responses in open-ended questions that ask for an in-depth answer
- Flatlining
In regards to the latter, remember the old advice to âjust choose Bâ all the way through if you didnât study for a test? That would result in an âFâ from us. Participants that choose the same answer repeatedly, try to take the same survey again, or attempt to submit multiple surveys in quick succession donât pass our test.
Natural Language Processing for Open-Ended Answers (New Feature 2023)
We've recently released an even more advanced type of quality check functionality, in addition to the various others explained in this article. Now, there is a data quality check for open-ended questions; this will apply to the answers to ensure they are of quality. The check applies artificial intelligence, namely Natural Language Processing (NLP).
NLP refers to the ability of computers to understand and interpret written or spoken speech, much like a human would. In this case, our research platform uses its new NLP methods over the given open-ended answers and evaluates their quality. By doing so, it then is able to determine whether to disqualify the respondent who gave the answer or not.
Respondent Verification
We further verify by checking ahead for duplicated IDs via IP or MAC addresses, Google Advertising and mobile device identifiers, and we work with our manually vetted publishers to send unique IDs as an added layer of protection. In-survey questions are designed to add another layer of security against survey fraud, such as requesting an answer to a simple math equation or including identical questions within a survey with the response options re-ordered to verify answer consistency.
Zero Tolerance For Bots
In addition, we have zero tolerance for bot-friendly VPNs, incomplete surveys, or other suspicious activities. We reject responses from any behavior we deem questionableâanything from answering open-ended questions with nonsense to attempting to sign in from multiple countries at once. Weâre even alerted if respondents are spending an inappropriate amount of time across questions within the survey.
Rigid Adherence to Targeting
Our data quality is second to none. As such, weâre skeptical of just about everyone, and thatâs what makes us the best at what we do. Accuracy is our top priority, which is why we are so critical of our sources and their behavior. In fact, we only include respondents that match 100% of the targeting criteria, even if there was nothing fraudulent about their responses. This includes surveys with multiple audiences, as well as those with stringent respondent qualifications, such as specific answers to screening questions. Our combination of technology and expertise ensures the delivery of the highest quality data to our customersâevery survey, every time.
Multiple Layers of Quality Checks
Pollfish supports multiple other layers of quality checks. This is an ongoing process since both the platform and our technical experts continuously work to avoid survey fraud while fetching the preset amount of required survey completions with the correct targeting. There are several layers that we use to improve the data quality. These include:
The Technical layer:Â
- Includes various checks to ensure first-rate data. Hasty answers check: catches respondents who answer faster than the average time required to read the actual questions.
- Reset ID Check: Occurs when the responder answered the same survey previously, but with a different device to avoid the same respondent from partaking more than once.
- Gibberish Check: When the given answers contain text that is considered gibberish, ex: âdsfjkn dfnksj ifjodfâ.
- Same IP Participation: Checks if a survey has been completed before within a certain time from the same IP address of the respondentâs device.
- Carrier Consistency: Assures that the carrier of the respondent is contained in the targeting market.
- VPN: VPN users are automatically disqualified from survey participation.
Quality Questions & Responses
Our platform is designed so that your questionnaire is bound to receive quality responses. In order for it to do so, aside from post-answer quality checks, we provide quality answer triggers. Our quality questions include: trap, red herring and attention questions.
- Trap questions: Checks that respondents are paying attention to a command, usually one that asks to select a negative response. Responders with acquiescence bias, or those who choose positive responses will be caught.
- Ex: Please select "Somewhat Disagree" below
- Strongly Agree
- Somewhat Agree
- Somewhat Disagree
- Strongly Disagree
- Ex: Please select "Somewhat Disagree" below
- Red Herring Questions: These check if the respondent is engaged with the survey, tracking how logically they answer oddball questions.Â
- Which of the following is not a sport?
- Soccer
- Basketball
- Cookies
- Baseball
- Which of the following is not a sport?
- Quality Questions: Checks if respondents are reading and comprehending the question, much like red herring questions.Â
- Ex: Which of the following does not have wheels?
- Bike
- Car
- Skateboard
- Elephant
- Cart
- Ex: Which of the following does not have wheels?
These are injected within various components in the survey flow depending on each surveyâs type to check if the respondents pay attention. Those quality questions are also used in Demographic surveys. If the respondent fails to answer a quality question, a reverse counter is displayed.
Finally, we also check for response quality by banning respondents who provide insufficient quality data from the network. Additionally, our system uses an audit to validate open-ended data to ensure that respondents do not provide inadequate responses. We also have installed a function that wards off copy-pasted answers in open-ended questions and others.