Multiple Selection Questions in Mobile Surveys (Updated)
Multiple Selection Questions in Mobile Surveys (Updated)
Multiple response, multiple selection or multiple-choice questions allow the survey respondent to choose one or more answers from set of provided responses of at least three or more options.
They are popular in survey design because of their convenience to the survey designer, as they can capture the respondent’s views and attitudes to the pre-formatted answers and provide a relative frequency or popularity of answers among a set of choices.
What should you consider about multiple choice in a mobile survey?
When designing a mobile survey, you must consider that the respondents have limited screen space before they have to scroll. So don’t overload the participant with too many answer choices in a multiple-choice question.
Answers to multiple-choice questions can be “shuffled”, or randomized in the Pollfish platform, however, remember that participants are allowed to select multiple answers. There is also the ability to anchor the last answer, by selecting “Shuffle but keep the last one fixed”.
NOTE: There are times when you do not want to shuffle answers, such as when asking for a numerical response.
You also want to limit the number of questions you ask the participant. Users tend to lose interest, “speed through” the survey or drop off completely if a survey contains lots of questions.
If you already have a list of answers, batch answers can speed the design of your survey, and avoid the hassle of typing individual answers one at a time. If you have the list on separate lines in an excel sheet, word doc, or similar, you can select "Add batch answers" to copy and paste your choices from your source document.
Advanced tip: If you want to copy the answers from a previous question you created in the Pollfish platform, select “Add batch answers” there, and the pop-up window will have your choices available to copy and paste.
2022 Feature: The Exclusive Answer Option
On August 10th, we released a new feature to enhance our DIY survey platform: the Exclusive Answer. This answer option is available only in the Multiple Selection Question type and as its name suggests, creates exclusivity in the respondent's answer.
This means that if you've designated an answer as exclusive, it can be the only one chosen, despite the question being a Multiple Selection. As such, it cannot be selected with other answer options.
Exclusive answers do not have any special visual identifier in the mobile tool, they appear just like any other answer, in order not to make the user’s selection biased.
Through this new feature, you can mark answers as "exclusive" by clicking the new icon, which is situated next to the "Add Media" icon. As such, on mobile devices, when a respondent selects this kind of answer, all other answer selections will be unselected.
The “Add Other” option provides the option of exclusivity; just tick the icon next to the “Other” text box. In this way, the “other” option on the left panel works like the “none of the above” option when you click on its relevant box.
The “Limit Answers” option still works as it has worked on non-exclusive answers and thus, does not apply to the exclusive answers.
Carried forward answers cannot be exclusive.
Creating an audio, video, or image survey with Pollfish (Video)
Creating an audio, video, or image survey with Pollfish
Surveys that support media files offer more ways to learn from respondents. They’re a great way to gather more qualitative feedback, such as emotional responses to an ad or concept.
Benefits of adding media to survey questions
Adding media to survey questions is a great way to let respondents get more context about what you’re asking. It can offer an element of accessibility for people who respond better to visual stimuli or those who have trouble reading. Not only that, media components like images and videos are engaging for respondents and add a nice look and feel to the survey.
Media is especially useful in measurement of more abstract concepts, like emotions and reactions. A picture is worth 1000 words, and you can let it speak for itself to convey stronger messaging to respondents that you might not be able to put into words in a single written survey question.
What media in survey questions are used for
Surveys using media questions are a great way to compare different concepts against one another or test and measure consumer reactions against their other, more emotional senses. Many brands use image-based surveys to test corporate logos and color schemes or measure brand recognition. Videos and gifs are excellent ways to test ad concepts before launch to see what consumers like or respond to. Audio additions to your survey may be the perfect way to choose the introductory theme song to your podcast or test for accessibility issues, ensuring that the tone, pitch, and accent being used is understandable and appealing for your target audience.
You can also allow your audience to respond to questions using an image, which is great for testing ad recall. Survey tools (like Pollfish) that offer these are a great help to people in all industries across a variety of different audio and visual testing needs, just make sure you have permission to use all media prior to testing.
Tips and FAQs for media survey questions
What kinds of media can you upload to Pollfish questions?
We support 3 different media types: Images, Video, and Audio files. You can add these into any of our question types, including screening questions. You can also include images in answer choices for the respondent to select a visual response. There is no limit to the number of images and videos you can include in your survey. There is also no limit to a video's duration.
Media, when used sparingly, can help the engagement of a survey, but when overdone can increase drop-offs. We don’t allow videos as answer choices, as these are typically too long for a mobile survey environment.
What sizes and file types are supported for media?
We allow one media file per question (image or video). The maximum size specs for each are as follows. These are NOT the recommended sizes. See below the image for recommended sizes.
The recommended sizes of each media file are as follows:
Image:
- Max Size: 10MB
- Recommended: 1920px x 1080px, 2MB
- Ratio: 16:9
- File type: jpg, jpeg, png, gif
Video:
- Max Size: 100MB
- Recommended: 1920px x 1080px, 25MB
- Ratio: 16:9
- Duration: Unlimited
- File type: avi, mp4, mkv, mov, flv, wmv
Audio:
- Max Size: 100MB
- Duration: less than 120 seconds
- File type: mp3
What plans allow images, video, or audio?
Media additions come standard on all Pollfish plans. You may add as many images, audio files and videos to the questionnaire as you want on any of the question types. Video duration is unlimited while audio files have a maximum of 120 seconds.
Can you upload more than one image to a question in Pollfish?
You are only able to upload one image or video in the question type and one image per each answer selection. However, you can show two images side by side if uploaded as a single photo. Try to ensure a clear line of distinction to show a left and right image.
Can you use media in Pollfish screening questions?
You can add images to screening questions and their answers. Like all Pollfish questions that support media, you can add one image for the description and one image each for the answer choices.
Can you link to a video on a third party site?
We don’t allow any links to be added to our surveys, in text or in visual media. This is to protect surveys from re-routing respondents out of the experience. However, media can be uploaded into the questions so that they are embedded in the survey, keeping all respondents and data in one location.
How do you know if respondents have watched the full video?
Respondents cannot skip questions (unless they are in a skip-logic flow). That includes video— respondents aren’t able to move on to the next question until the video is complete. If the video is too long and the respondent opts out, it will be considered a drop-off and their responses will not be included in the final data. To encourage engagement and reduce drop-offs, we only support videos up to 2 minutes in length, although we suggest making them even shorter when you can.
How do you know what the image or video will look like?
In the Pollfish platform, it's recommended that you preview your questionnaire before launch. This shows you a mock-up mobile screen with an interactive version of your questionnaire. While this is helpful in viewing your media survey, you can also see how questions are formatted, catch any typos, and ensure your answers are shuffled prior to launch. Respondents are able to zoom in on images, but it's better if they don't need to take this action or rotate their device. We recommend uploading images in a vertical direction when possible to ensure they're upright and as large as possible to provide a better respondent experience.
If you have additional questions about our media capabilities or how to use Pollfish, please reach out to our 24/7 customer support team for assistance.
Setting age ranges in the Pollfish survey platform (Video)
Setting age ranges in the Pollfish survey platform
There are a number of reasons that age range can be an important, or even necessary, setting in a survey’s demographic targeting.
When should you set an age range in a survey?
If you are selling a product or service that is tailored to the needs or wants of a certain demographic, it is beneficial to be able to reach only those who fit your desired audience, rather than sending the survey broadly and having to throw away responses from people who aren’t a fit for your research audience.
This is true for any audience targeting criteria: if you are selling farming tools, it would make sense to avoid respondents living in urban areas. Similarly, responses from retirees will likely not give you valuable data if you are attempting to market a product to teens.
Targeting age groups in a survey can be used to include or exclude certain audiences by age range or quota. A common use is to ensure that respondents are of legal age to vote, drink, or smoke before asking questions that are related to those products.
Age range, and other demographics, are set at the beginning of the survey to hone in the right target audience.
How to identify and reach target audience age groups
Most survey tools include some demographic settings and others require a series of questions to hone in on these specifics. The best way to ask a question to learn the target audience’s age is to ask them to select an age range from a list of multiple-choice options.
A common sample question to define age group targeting for a survey is
What is the age group you belong to?
- 16-17
- 18-24
- 25-34
- 35-44
- 45-55
- >54
- Prefer not to say
In Pollfish, all demographic data is gathered from respondents when they join the Pollfish survey audience. Rolling profiling keeps this data up to date to ensure respondent details are accurate.
Because we already have this information, you don’t need to ask respondents about their age or birthday, you can simply set the targeting criteria you want and we will distribute the survey to people who meet the targeting requirements.
You can ask additional screening questions to target specific attitudes or behaviors that will further narrow the target audience.
How to set age range targeting in Pollfish
When you are creating a Pollfish survey, you begin with your audience targeting page. Here, you will find all age groups listed and selected already. If you want to only reach a segment of the population, for example, those age 18-34, you can deselect the other categories. Only those which you have chosen will receive the survey.
You can also apply quotas to these groups in order to weight the percentage of respondents from each cohort who receive the survey.
If you have a specific age group that you’re looking for, such as Gen Z audiences, you may need to input exact ages to reach the range that best fits for you. This can be done by selecting “Specific age range” and inputting a minimum and maximum range for your audience.
Because we are COPPA compliant, we don’t survey children age 15 or younger. The youngest members of the Pollfish audience are 16.
Mobile surveys help you reach target audience age groups
A unique benefit of Pollfish is the mobile distribution model. The world is becoming increasingly accessible by mobile device, with nearly everyone having a smartphone at their disposal. Additionally, mobile is the primary (or only) way that some people connect to the internet.
App-based surveys offer a massive reach to survey audiences of all ages, including harder to reach segments such as Gen Z, international audiences, and specific markets.
Using mobile ad IDs rather than a survey link, we can map demographics back to individual users of a device (as mobile phones are rarely used by more than one person).
This also means that we are able to maintain demographic profiling on respondents so you can target and tailor questions for them, rather than spending your time creating questions asking them their basic data.
This article updated April 2020 to reflect changes to COPPA regulations regarding minimum targeting age.
Frequently asked questions
Why should you set an age range in a survey?
You should set an age range in a survey if you are selling a product or service tailored to the needs of a certain demographic. It is beneficial to be able to reach only those who fit your desired audience, rather than sending the survey broadly and having to throw away responses from people who aren’t a fit for your research audience.
Do age ranges help with targeting an audience in market research?
Age range is one of several targeting criteria in market research. Targeting age groups in a survey can be used to include or exclude certain audiences by age range or quota. A common use is to ensure that respondents are of legal age to vote, drink, or smoke before asking questions that are related to those products.
When should you set an age range in an online survey?
Age range, and other demographics, are set at the beginning of the survey to hone in the right target audience.
How can researchers identify and reach target audience age groups in an online survey?
Most survey tools include some demographic settings and others require a series of questions to hone in on respondents' age. The best way to ask a question to learn the target audience’s age is to ask them to select an age range from a list of multiple-choice options.
A common sample question to define age group targeting for a survey is
Frequently asked questions
Why should you set an age range in a survey?
You should set an age range in a survey if you are selling a product or service tailored to the needs of a certain demographic. It is beneficial to be able to reach only those who fit your desired audience, rather than sending the survey broadly and having to throw away responses from people who aren’t a fit for your research audience.
Do age ranges help with targeting an audience in market research?
Age range is one of several targeting criteria in market research. Targeting age groups in a survey can be used to include or exclude certain audiences by age range or quota. A common use is to ensure that respondents are of legal age to vote, drink, or smoke before asking questions that are related to those products.
When should you set an age range in an online survey?
Age range, and other demographics, are set at the beginning of the survey to hone in the right target audience.
How can researchers identify and reach target audience age groups in an online survey?
Most survey tools include some demographic settings and others require a series of questions to hone in on respondents' age. The best way to ask a question to learn the target audience’s age is to ask them to select an age range from a list of multiple-choice options.
A common sample question to define age group targeting for a survey is
What is a Rating Question? Definition, Examples & More
What is a rating question?
Rating questions (See also: slider questions) allow participants to weigh, or assign numerical values to answers via a graphical interface —using a simple 1-5 star rating system, or 0-100 scale where a higher number is a better score.
Rating Questions are particularly effective on mobile, due to the graphic user interface, and simple tap-to-enter, or drag-and-drop functionality.
Rating questions can be used to evaluate a variety of topics or stimuli, including statements, images, or videos.
Adding rating questions to a mobile survey
Since most mobile survey users prefer simple gestures that can be performed with one hand, these questions are favorites among respondents for their ease of use because of their tapping or drag-and-drop capabilities.
The information you gain is on one parameter—their preference for a particular piece of content.
You can use a series of survey rating questions to understand preferences for different topics and then compare the results.
You may also consider including different types of survey questions to avoid participants falling into a response pattern.
Rating Questions Examples
Here are the rating questions examples you can use when you create a survey with Pollfish.
Star Ratings
Select a rating out of 5 stars, as seen on Yelp or Roger Ebert. This is the simplest form of rating question and provides the cleanest data set. But the nature of the question reduces the depth of response you can get from it.
Slider Ratings
A scale of 1-100 will allow users to rate performance. This question type provides a wider distribution of responses.
Single-Selection Matrix Question
Rate your choices on a matrix of possible options. By reducing to one response per line, you can ask for a rating on multiple features or products inside the same question.
Slider questions in mobile surveys
Slider questions in mobile surveys
Slider questions are a form of a survey rating question where the user inputs feedback via a graphical interface.
There are several advantages to using the slider question type on your mobile survey, including:
- They are fun and engaging to users
- They allow for more specificity than a simpler rating scale
- Users are often more inclined to answer truthfully given the flexibility to rate as they see fit
However results from a slider rating can have a wider variety of responses, which can cause challenges in interpreting the overall sentiment of the sample.
Grouping these numerical responses into ranges based on the ratings provided can be helpful in this interpretation, assuming your survey provider supports answer grouping with this type of survey question.
Rank order survey questions (and how to use them)
Rank order survey questions (and how to use them)
Rank order survey questions (also called Ranking questions) enable participants to compare items and rank them according to their preference.
The Benefit of Rank Order Survey Questions
The benefit of this over other survey question types like multiple answer survey questions or rating questions is that you gain critical data about a participant's preference of one item as compared to another—versus choosing multiple answers without ranking them or rating on an individual basis.
Ranking Survey Questions vs Rating Survey Questions
Ranking survey questions differ from rating questions in that participants are forced into providing a preference for one item over another—whereas in a rating question, two items may both compare similarly. For example:
“Rank your preference on the hotel amenities during your stay”
- Pool
- Gym
- Bar
- Fast Check-in
Will provide different information than
“On a scale of 1-5, please rate the following hotel amenities”
- Pool
- Gym
- Bar
- Restaurant
In the latter, all categories may receive a “5” – which will tell you how satisfied a guest is, whereas survey ranking questions may provide insight on what the most appealing amenity is.
How To Write Ranking Survey Questions
Unlike other types of survey questions, which may tell you how much a user enjoyed your product or service, Ranking questions are often used to determine what about the experience was most valuable.
For example, if you ran a Mexican restaurant, you may want to know the most popular Mexican dish among Mexican food fans in your area. But let's say you are going to invest in certain parts of your business--new menu, top-shelf ingredients, renovation. But you only have money to invest the money into three projects. It might be helpful also to know which parts of your business customers already love and which they feel could use some improvement.
Here's where ranked-choice survey questions are a must.
Just a few examples for your Mexican restaurant:
Please rank the following in order of importance
- Great cocktails
- Vegetarian options
- A delicious dessert menu
- A top-shelf tequila list
- Ample parking
- Speedy pick-up service
Please rank your favorite Mexican restaurants in the area
- Sleepy Joes
- Cantina Dos Segundos
- Casa Taqueria
- Rosa Mexicana
- Jose Pistolas
- Chipotle
Best Practices for Ranking Survey Questions
Don't read too much into the lower rankings
Once people have selected their favorites or their least favorites, results can get a little noisy. Most people clearly know what they like and don't like, and may just be selecting in the middle to get the question over with. Make sure to take this into account.
Aim for six items
Six items in a ranking question allows for a full range, without creating imprecise answering that longer lists can create. Six items allows you to determine a top 3 very clearly, or get a high end and a low end, with enough options in between. In our experience, between 6-10 items usually work well.
Shuffle items
If your survey platform allows it, set items to shuffle regularly. This will reduce survey bias. In some cases, the top result may be ranked higher because it is first on the list.
Split longer lists into categories
If you have a very long list of different product features that you want ranked, doing a long ranking list will provide messy data that is hard to use. Try splitting your items into categories, keeping lists short and creating multiple questions. It will increase response rates and give you a better dataset.
Frequently asked questions
What are rank order survey questions?
Rank order survey questions, also called Ranking questions, enable participants to compare items and rank them according to their preference.
What is the benefit of Rank Order Survey Questions?
The benefit of rank order survey questions over other survey question types (multiple answer survey questions, rating questions) is that you gain critical data about a participant's preference of one item as compared to another—versus choosing multiple answers without ranking them or rating on an individual basis.
How can you reduce survey bias in rank order survey questions?
You can reduce survey bias in rank order questions by shuffling them regularly, if your survey platform allows it. This is because, In some cases, the top result may be ranked higher because it is first on the list.
How do rank order survey questions differ from other types of survey questions?
Unlike other types of survey questions, which may tell you how much a user enjoyed your product or service, ranking questions are often used to determine what about the experience was most valuable.
Frequently asked questions
What are rank order survey questions?
Rank order survey questions, also called Ranking questions, enable participants to compare items and rank them according to their preference.
What is the benefit of Rank Order Survey Questions?
The benefit of rank order survey questions over other survey question types (multiple answer survey questions, rating questions) is that you gain critical data about a participant's preference of one item as compared to another—versus choosing multiple answers without ranking them or rating on an individual basis.
How can you reduce survey bias in rank order survey questions?
You can reduce survey bias in rank order questions by shuffling them regularly, if your survey platform allows it. This is because, In some cases, the top result may be ranked higher because it is first on the list.
How do rank order survey questions differ from other types of survey questions?
Unlike other types of survey questions, which may tell you how much a user enjoyed your product or service, ranking questions are often used to determine what about the experience was most valuable.
Multilingual Surveys: Ask Consumers In Their Native Language
Multilingual Surveys: Ask Consumers In Their Native Language
How to create multilingual surveys
Part of targeting the right survey audience is making your survey respondents as comfortable as possible with clear, concise surveys in their native language.
To help you create multilingual surveys, the Pollfish platform has native support for multiple survey languages, including: English, Bulgarian, Chinese, Czech, Danish, Dutch, Filipino, Finnish, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Polish, Portuguese, Portuguese (Brazil), Romanian, Russian, Serbian, Spanish, Swedish, Thai, Turkish and Vietnamese.
When you select a language from the "Survey Language" menu, the prompts presented to respondents will change to that language.
NOTE: Changing the survey language will not translate the question and answer text, only instructional text such as "make a selection."
At this time, there is no native support for automatic translation of questions or answer options. You will need to translate your questionnaire into the corresponding language prior to launch.
The questions and answers in your survey will be displayed exactly as you designed them, which you can confirm in your Survey Preview. We also recommend preparing for any translation needs once your survey is complete, as answers will be displayed in the language in which the respondent replies.
Unfortunately Pollfish does not offer translation services at this time, however a quick Google search can help you find a translation vendor, or you can use an online tool such as Google Translate. You will need to verify that your keyboard provides support for any special characters required and must be certain that the translations are not offensive or can be misinterpreted by the participant, as such tools do not account for all the nuances in multiple language translation. Many of our clients choose to work with outside translation services for questionnaire development and interpreting results.
Targeting the right survey audience
Targeting the right survey audience
If you are developing a survey, you probably have a broad idea of the audience you want to reach, right? But with all of the options out there for reaching potential respondents, how do you target the right survey audience?
Let's start with some basics.
Demographics
Demographics are populations based on age, sex, income level, employment, education, etc. You probably have a general sense of the age range, gender, and location of the people you want to reach – you may even know the income ranges and education levels.
There are other ways of targeting your audience as well – including psychographics (personality, values, lifestyles), behavioral attributes (loyalty, habits), firmographics (number of employees, functions), even technographics (attitudes toward technology, adoption rates).
In most cases, a combination of demographic, psychographic and behavioral attributes can help you reach your ideal target audience. Pollfish pre-screens all of our respondents, enabling you to select from a mix of predefined criteria to build exactly the audience you want to survey, without having to ask those questions yourself.
However, sometimes your ideal audience target is more narrowly defined, which is where advanced filtering comes into play.
Advanced filtering
If you want to find a certain audience based on factors like their behaviors, opinions, or exposure to events, you can ask a screening question.
Screening questions help you reach the target audience that will meet your criteria. A screening question, when worded properly, will disqualify those respondents who may be a fit from a demographic standpoint but aren't an exact fit for your specific research needs. You can find out more about screening questions here.
Finding The Right Type of Survey Target Audience
Unfortunately, not all survey respondents are created equal. One option is to go to a provider that uses research panels. Panels consist of people who agree in advance to take surveys, often by email, in exchange for a predetermined benefit (monetary payments, prepaid credit cards and airline points are the most common incentives.) There are benefits of using a research panel, including:
- Detailed audience profiling
- Ability to target B2B audiences
- Established approach
However, there are downsides as well, including:
- Cost of panels—survey respondents are paid directly for their participation, which gets passed on to the client
- Speed (or lack thereof)—traditional panel-based surveys can take a significant amount of time to complete
- Randomization—respondents may not be selected randomly, and may not be representative of the broader population
- Data Quality from panelists—aggregate incentives offered by panels can lead to "professional survey takers", resulting in flawed data
- Limited Availability—panels have to be built from scratch for each new market they enter, which can result in delays and lack of availability
Organic RDE Sampling
Surveying via organic RDE methodology—such as on Pollfish’s network—provides the benefits of scale, convenience, and fast response rates, while also saving on cost, providing high-quality data, and access to a variety of sample populations that can help you to reach the target market you are seeking for your survey.