Grouping, Shuffling and Randomization in the Pollfish Questionnaire

As many researchers learn early on, it can be hard to limit bias in surveys which has negative consequences on the quality of data. 

randomize survey

Some best practices for minimizing bias include randomization of questions and the distribution of the questionnaires themselves to reach a diverse audience. Shuffling unordered answers or reversing the order of a scale has also been shown to reduce bias by displaying clear, but differentiated answer options to respondents. 

Questions such as scales that follow an order should be reversed for clarity, whereas answers that do not follow a sequence should be shuffled for increased randomization.

 

Although Pollfish has supported shuffling and grouping within individual questions, we have recently released the ability to fully randomize your questionnaire. You can shuffle groups of questions and even shuffle the questions within the group for even more randomized survey opportunities. 

Here’s how to apply it in your next survey.

Grouping Questions in Pollfish

When you visit the questionnaire section of your Pollfish dashboard, you’ll see we offer new randomization capabilities. Here, you’ll be able to assign questions into groups on the questionnaire. 

This can be helpful for managing your survey in specific sections, for example, if you wanted to first investigate behavior before diving into specific interests. 

There is no need for a group to contain any certain number of questions. A “group” can contain a single question if needed, however if Grouping is applied to the questionnaire, every question will need to be assigned to a group.

You can manually re-order the groups or the questions within the groups by dragging and dropping, adding or deleting questions as you would in a regular Pollfish survey.

Keep in mind that when you delete a group, you may also delete the questions within the group by selecting “Delete All.” If you’d like to remove the grouping feature but keep your questions, make choose the option that says “Delete Group Only.”

Shuffling and Randomizing Questions in Groups

The groups themselves and the questions within the group can be shuffled randomly or reversed in order to reduce bias without disrupting the logical flow of the questionnaire.

However, there is no ”Grouping and Shuffling” as an aggregate option anymore. Instead the option has been divided into ”Randomize Questionnaire” (located on the left side) and Group of Questions (located at the bottom of the Question type drop-down menu).

A group can also be “fixed” while other groups are shuffled. For example, perhaps you want to provide a description before a group of questions that will provide context. In this case, you can choose to put a description question-type in a group and shuffle the remaining groups in your questionnaire.

Don’t forget that you can (and should) include shuffling and reverse order options on your individual questions as well, adding to even more randomization and cleaner data.

Originally, we had 3 main ways to shuffle: 

  • Shuffle the answers in an individual question
  • Shuffle groups of questions in your questionnaire
  • Shuffle the questions within a group

But now, there are more options for randomizing questions.

You can enable Randomize Questionnaire and select Shuffle All, to shuffle a group of questions with the rest of the questions. If you select the group, you have an additional option; you can randomize the questions within the group. When you turn off the Randomize Questionnaire toggle and shuffle the sub-questions, the platform only shuffles the questions within a particular group, the rest of the questions won’t be shuffled.

Essentially, you can shuffle anything you choose while leaving certain questions fixed.

Here’s how else you can shuffle on Pollfish:

  • Shuffle but keep first one fixed
  • Shuffle but keep first two fixed
  • Shuffle but keep last one fixed
  • And many other ways!

Pollfish Question Randomization Capabilities

Now, it’s time to see exactly what our question randomization options look like on the Pollfish platform. To get a visual representation of the functions mentioned in the previous section, look no further.

The following images show how we’re offering randomization capabilities on our platform:

Randomize Questions

randomize survey questions

Randomize Question Options

Randomize Question Options

 

Randomize Sub-Questions

Randomize Sub- Questions

Groups of Questions

Groups of Questions

 

Adding logic to groups

Skip logic isn’t yet supported in grouped questions, however, the product team is hard at work. Advanced skip logic will support grouped and shuffled questions, as well as allow for A/B testing, a critical and useful feature in advertising effectiveness surveys