How to Create a Survey with Sensitive Questions and Avoid Being Flagged on Pollfish

How to Create a Survey with Sensitive Questions and Avoid Being Flagged on Pollfish

Setting up a survey that deals with sensitive topics is never an easy feat for any party involved. To avoid trouble in your market research endeavors, we’ve updated our Review Policy for sensitive questions/topics. (See the Pollfish Review Policy section in this article)

Although the Pollfish platform offers a vast pool of question types and makes it easy to incorporate any question type, getting respondents to complete surveys with sensitive questions is not guaranteed.

Our review team may flag your survey and even block it from being launched if we deem it inappropriate.

In fact, if you ask sensitive questions in a reckless manner, it is bound to turn respondents away from completing the survey, or beginning one in the first place. Thus, these kinds of questions mar the survey experience for respondents and can lead to survey attrition. 

But that does not mean you ought to omit surveys and their larger campaigns due to the inconvenience of the questions being sensitive in nature.

You can still create effective surveys by asking the sensitive yet necessary questions as long as you do so with certain best practices to make it a better experience for your respondents. This article explains how.

What are Sensitive Questions?

The notion of a sensitive question is not as subjective as it may appear. These questions make respondents feel embarrassed or uneasy due to the nature of the question, its tone, context or setup. 

Here are some of the question types that can cause respondents to feel uncomfortable:

• Questions that invade privacy: 

1. 1. 1. Religion, political stance, sexual behaviors

• Questions that ask about illegal behaviors:

1. 1. 1. Drug use, theft, perjury, other illegal acts

• Questions that pose a risk or paint the respondent in an unfavorable light:

1. 1. 1. Cheating on partners, tests, taxes, thoughts on their colleagues or job

• Questions on identification (especially if detailed):

1. 1. 1. Income, age, race, job title

• Questions with socially desirable responses:

1. 1. 1. Voting history, exercising, wearing a seatbelt)

• Questions that deal with unpopular ideas:

1. 1. 1. Racism, ageism, etc.

While you cannot change the nature of a question, you can still set it up so that it is poised to be better received — and answered. 

Other Factors Causing a Distressful Survey Experience

Aside from the questions themselves, there are other factors in your survey relating to the questions that can render an uncomfortable survey experience. As such, you should be aware of these to understand the respondents’ perspective.

Additionally, understanding them will help you avoid running into these pitfalls and optimizing your surveys for effective survey campaigns. The following discusses the major causes of distress and survey attrition in regards to sensitive questions.

1. The privacy (or lack thereof): Privacy is of the utmost importance when it comes to filling out any survey. When it comes to surveys that deal with sensitive topics, it is especially important, as no responder will want to associate their identity with their answers.
   1. Although your survey is private, perception is key, as respondents may perceive that their participation may still be traced back to them
2. The mode of the survey: This involves the means of administering the survey; some may send it via email, while others rely on a survey platform.
   1. This is an issue of presentation, which may influence respondents’ perception of privacy.
      1. Ex: Emails sent to respondents remove the air of confidentiality, since it was sent to a particular address, which often is the respondents’ names. 
3. Brand Reputation: When dealing with sensitive topics, respondents are particularly aware of the organization to which they will provide answers on sensitive topics. 
   1. Some surveys do not mention the brand or organization which they represent, while others may.
   2. In relation to sensitive questions, respondents may feel unnerved to share answers with certain companies.  

How Sensitive Questions Harm Your Survey When Used Incorrectly 

As mentioned in the introduction, using sensitive survey questions without heeding critical best practices have grave consequences for your survey campaign. They tarnish the accuracy of the responses, an issue no survey platform can fully evade.

They also lead to respondents unwilling to take your survey, that is, they won’t even open it to give it a chance. This makes fulfilling your survey quotas and number of completes a more difficult and lengthy process.

Survey attrition is another issue spawned when sensitive questions are used improperly. Market researchers can avoid poor quality data and survey attrition by setting up surveys with sensitive questions in a thoughtful manner, one that fully takes best practices into consideration.

How to Ask Sensitive Questions the Correct Way

Market researchers can still extract responses on sensitive subjects by doing so with best practices. 

As such, you need to devise your survey strategically so that you don’t repel much-needed respondents. In order to keep responders in the survey, having them answer honestly and answering all the questions, apply the following methods.

These methods will avoid summoning feelings of distrust and general unease. They will also help you build your survey in a way that is conducive to a good user experience.

1. 1. Be upfront about privacy: Relay a confidentiality guarantee.
      1. Either in the survey callout, invitation or introduction, assure your respondents that their participation is completely confidential. 
      2. Let them know the precautionary methods you’re taking to ensure their answers remain hidden from general view and are not associated with their names.
   2. Be transparent with the purpose of the survey. Tell the respondents upfront what the topic of study is along with its purpose. 
      1. Doing so will avoid surprises and when respondents know what they’ll be dealing with upfront, they’ll be less disinclined to complete their surveys. 
   3. Be honest with anonymity. Don’t relay that a survey is anonymous unless it actually is.
      1. On the Pollfish platform, all responses are anonymous based on our terms, so you can state that the survey is anonymous.
      2. Additionally, the survey must contain one initial screening question that states the study’s content and asks for respondents' age and consent. For example:
         1. “The following survey contains questions regarding [sensitive content input]. Are you willing to take part in the following survey? 1. Yes, I am 18 years old and above and I give my consent  2. No’'
   4. Build up to sensitive questions, don’t ask them right away.
      1. Don’t ever present sensitive questions first; instead, lead up to them by asking more neural questions that become more specific with each incoming question.
      2. This way, you can avoid shocking your respondents and turning them away from answering or providing honest questions.
   5. Use ranges instead of specific values.
      1. Respondents may not feel comfortable with revealing the exact amount, especially if it deals with a negative behavior or societal disapproval. 
      2. For example: instead of providing precise numbers as answers to, “how many times have you fought with your spouse?” or “how many times have you smoked cigarettes last week?” use ranges, such as “under 3 times last week,” “3-5 times,” etc.
   6. Use “question loading” to elicit honest answers.
      1. Not to be confused with loaded questions, question loading involves blanketing a question with context, so as to make the subject in question appear more normal and acceptable.
      2. This will make respondents feel less guilty of partaking in a certain behavior, act or inertia.
      3. Ex: Instead of asking “did you exercise last week?” question load the question to normalize the less acceptable answer (of not exercising). 
      4. Question-Loaded: “Given that the average American worker works 9 hours a day and has several responsibilities, exercising is not always a priority. With that in mind, how many hours of exercise did you do last week?”
   7. Use frequency questions instead of yes or no questions, if the topic deals with partaking in a bad behavior.
      1. Instead of asking “have you ever run a red light when driving?” ask “how many times have you run a red light when driving?”
         1. The latter assumes the respondent has already done so, so their response cannot be a lie.
         2. If you used a yes or no question, on the other hand, the respondent would be less inclined to answer honestly, given that negative behaviors are not socially acceptable.
   8. Provide an “I prefer not to answer” option.
   9. Try as one might to frame a question tactfully, sometimes respondents simply do not want to answer a sensitive question.
   10. Instead of losing a respondent due to one such a question, provide a multiple-choice option of opting out.
   11. This is best suited for surveys that include questions other than sensitive ones. If your survey applies questions on sensitive topics only, you should not include this option — unless it’s on one or two particularly sensitive questions.

The Pollfish Review Policy for Sensitive Questions

Starting in December 2022, Pollfish is introducing a more stringent review policy for all surveys dealing with sensitive topics. 

We don’t take sensitive topics lightly, especially surveys that cross the line and become inappropriate. This includes surveys that feature the following content:

• Rude language (swearing, insults, crude topics)
• Leading questions (those that contain answers that lead to certain opinions that respondents don’t necessarily hold but have no choice but to select)  
• Offensive content or language
• Disturbing content (language, imagery or audio files)
• Insinuations
• Hate speech

This means, if your survey is deemed inappropriate, it will not only be flagged, but will have the possibility of being shut down completely. 

Essentially, you won’t be able to run said inappropriate survey. 

That’s why we ask you to please be aware of the content of your survey. Pay attention to all of the elements of your survey, such as:

1. The overall theme and topic
2. The subtopics
3. The demographics
4. The period of time (proximity to a scandal or controversial event) 
5. The language and tone
6. The narrative the questions (and their answers) form
7. The images and other multimedia files you include
8. How you address certain sensitive issues (disability, illness, sexuality, race, religion, gender, gender identity, etc., political leanings)
9. The positioning of the questions (some will need advanced skip logic so that only the most relevant respondents will be routed to a particular question). 
10. The quotas you set on the demographics and psychographics of your survey audience

Now, let’s get into more basics of our policy. The original policy is composed of three criteria and they are as follows:

1. Respondents must be at least 18 years old to take the survey. You can review your age targeting in the Audience tab.
2. The survey must contain one initial Screening Question that states the study’s content, for example: “The following survey contains questions regarding [sensitive content input]. Are you willing to take part in the following survey?”
   1. Your responses are anonymous. 
3. There must be one question that explains the survey content briefly, along with the purpose of using such information and how you’ll use it.

The Pollfish team needs time to review your survey before pushing it live. If your survey is deemed inappropriate, it’ll be flagged as such in our survey process of approvals. We’ll ask you to edit it by either rephrasing your questions or removing them altogether. 

Remember to be respectful in your surveys as you would be in real life. There is no need to shock or offend, or ask leading questions to push a narrative. Doing so will only tarnish the validity of your survey. 

Extracting the Best Respondent Data

Obtaining quality data relies on several things, such as the questions themselves, the setup of the survey, how it’s introduced and the digital platforms it is deployed in. Additionally, it relies on the aid of a quality check function.

Thus, it’s important to invest in an online survey platform that facilitates the entire survey process — from creation, to deployment, to survey quality checks and post-completion quality checks. The Pollfish platform performs such checks with an AI-powered system. It voids incomplete surveys, those with gibberish answers, users of VPNs and more.

Ensuring the use of a competent online survey tool will maximize the success of your survey, allowing you to avoid the survey bias present in surveys with sensitive questions. 

by lorraine