New Pollfish Feature: A/B Testing Directly in Questions with the Monadic Version 

Pollfish is thrilled to now offer an A/B testing functionality in our dashboard, enabling market researchers to easily test concepts alongside one another. It is now easier than ever to test a variety of concepts, whether it is for marketing, advertising, branding or pure market research needs. 

A/B tests can be used to compare product concepts, communication ideas, or specific ads using equally structured groups of participants. 

In our first iteration of A/B testing, we introduce the Monadic version. Two more versions are slated to be released on the platform — all of which are set up as a question type. The next two are the sequential and conjoint analysis.

The monadic and sequential version is open to all account types, while the conjoint analysis version is for Elite accounts only.

This article explains A/B testing, how to use it in the Pollfish platform and the three versions Pollfish offers.  

Understanding A/B Testing on the Pollfish Platform

A/B testing on Pollfish is performed as a new kind of question type. That means, to perform the test, the A/B testing function has to be added to each survey that researchers wish to apply the test to. As such, A/B testing is an option that researchers can apply to their survey.

There can only be one A/B test per survey.

Researchers cannot apply advanced skip logic in the A/B group. However, they can apply logic to questions before and after the A/B group, as well as include the A/B group in the logic path. 

All respondents who will be directed in the A/B testing will be randomly assigned to a concept, if the test is monadic. Pollfish ensures that all concepts are evenly distributed to the audience.

You cannot apply the question function of Recall Information, which places the text from a previous question onto another or the Carry Forward function, which pipes a part of a question or answer into another, to questions within the A/B Test. Those features are disabled in the A/B test. 

There are different options available to view the A/B test results. Researchers can filter per concept; they can isolate a concept or compare two or more concepts.

Different table views are available for different questions, such as Single, NPS, Rating stars etc.

The researcher can view the concepts per answer or per concept. Attributes (more on this in the following section) will be displayed next to the questions. 

The Monadic A/B Testing Version

Monadic testing is a useful survey design that allows a respondent to claim their preferences for one concept or product they will randomly get, out of the many that the researcher wants to test and compare, instead of being exposed to two or more at once.

Focusing participants’ attention on just one stimulus at a time grants researchers a deep dive into making product, pricing and various marketing decisions.

Monadic testing is typically used for gathering independent findings for each stimulus — a contrast to comparison testing, where several stimuli are tested side-by-side.

In the monadic version, each question in a monadic group counts individually to the cost calculation as a simple question. For example, if the monadic group includes ten questions and you add eight questions outside of the monadic test, the price will start at $1.25 per response in a basic account.

The monadic test uses what is known as “attributes” which allow researchers to use different representations of the concept in a monadic question. These involve different media, such as images, videos, GIFs, audio recordings, long descriptions, short text such as a tagline and use text formatting. 

For example, for a soft drink concept testing, you would add logos, packaging, and pricing attributes. 

The attributes in monadic testing are displayed on a monadic table, which defines the order of the attributes. The rows (in the left panel) are named A1 (the first), A2 (the second) and so on. Thus, the respondents view every question within the A/B test accompanied by the attributes selected at the left panel. 

Researchers can apply logic and direct responses after the monadic test; thereby skipping it. In this case, the researcher must understand that the final results for each concept may not be evenly distributed. As aforesaid, skip logic is not supported within the A/B test, so it can only be applied outside of it.

How to Create a Monadic Version A/B Test

Follow these steps to create and view a monadic A/B test on the Pollfish platform.

  1. To create a monadic test, go to the questionnaire section of a survey. 
  2. In the left panel, click on “add new,” then click on AB Test.
  3. Below that, there is a dropdown menu to select the type of A/B test. Choose Monadic.
  4. In the concepts table, enter the attributes, these can be media files or text or even combined.
  5. You can view these in your results page, while the survey is running and when the results are ready.
  6. As the survey is running, you have the option of translating the survey, including the A/B group of questions.

Using A/B Testing to Accompany All Campaigns

With the recently released feature of A/B testing, Pollfish allows market researchers to test virtually any concept while running a survey in tandem.

With three new kinds of A/B testing soon to be available on the Pollfish online survey platform, it is now easier and smoother than ever for market researchers to incorporate A/B testing into their survey research.

Pollfish supports all exports (except the crosstab) for A/B tests. These are generated containing the concept data per each question. Hence, each question is generated as many times as the concepts, and is accompanied by the concept name to distinguish between the different concepts. Crosstab export with concept data, will be supported shortly after the monadic version is released.