Grouping and Shuffling in your Pollfish Questionnaire

As many researchers learn early on, it can be hard to limit bias in surveys which has negative consequences on the quality of data. 

Some best practices for minimizing bias include randomization of questions and the distribution of the questionnaires themselves to reach a diverse audience. Shuffling unordered answers or reversing the order of a scale has also been shown to reduce bias by displaying clear, but differentiated answer options to respondents. 

Questions such as scales that follow an order should be reversed for clarity, whereas answers that do not follow a sequence should be shuffled for increased randomization.

 

Although Pollfish has always supported shuffling within individual questions, we have recently released the ability to shuffle groups of questions and even shuffle the questions within the group for even more randomized survey opportunities. 

Here’s how to apply it in your next survey.

Grouping Questions in Pollfish

When you visit the questionnaire section of your Pollfish dashboard, you can select “Grouping and Shuffling” from the lower left-hand menu. By enabling this feature, you’ll be able to assign questions into groups on the questionnaire.

This can be helpful for managing your survey in specific sections, for example, if you wanted to first investigate behavior before diving into specific interests. 

There is no need for a group to contain any certain number of questions. A “group” can contain a single question if needed, however if Grouping is applied to the questionnaire, every question will need to be assigned to a group.

You can manually re-order the groups or the questions within the groups by dragging and dropping, adding or deleting questions as you would in a regular Pollfish survey.

Keep in mind that when you delete a group, you may also delete the questions within the group by selecting “Delete All.” If you’d like to remove the grouping feature but keep your questions, make choose the option that says “Delete Group Only.”

Shuffling questions in Groups

The groups themselves and the questions within the group can be shuffled randomly or reversed in order to reduce bias without disrupting the logical flow of the questionnaire. 

A group can also be “fixed” while other groups are shuffled. For example, perhaps you want to provide a description before a group of questions that will provide context. In this case, you can choose to put a description question-type in a group and shuffle the remaining groups in your questionnaire.

Don’t forget that you can (and should) include shuffling and reverse order options on your individual questions as well, adding to even more randomization and cleaner data.

3 ways to shuffle 

  • Shuffle the answers in an individual question
  • Shuffle groups of questions in your questionnaire
  • Shuffle the questions within a group

Adding logic to groups

Skip logic isn’t yet supported in grouped questions, however, the product team is hard at work. Advanced skip logic will support grouped and shuffled questions, as well as allow for A/B testing, a critical and useful feature in advertising effectiveness surveys