Probability vs Non Probability Sampling

As stated in this article on Wikipedia, survey sampling methods consist of two variations; probability and non probability sampling.

So what are the main differences between the two?

Probability sampling

This means that everyone in the population has a chance of being sampled, and you can determine what the probability of people being sampled is.

Probability sampling includes Simple Random SamplingSystematic SamplingStratified Sampling, Probability Proportional to Size Sampling, and Cluster or Multistage Sampling. And have these elements in common

  1. Everyone has a known (calculated) chance of being sampled
  2.  There is random selection

Non probability sampling

This means that you have excluded some of the population in your sample, and that exact number can not be calculated – meaning there are limits on how much you can determine about the population from the sample.

Nonprobability sampling methods include convenience samplingquota sampling, and purposive sampling – or judgment sampling, and snowball sampling.


Probability Sampling Methods

Simple Random Sampling

Random sampling, in its simplest and purest form, means that each member of the population has an equal (and known) chance at being selected. In a large population, this becomes prohibitive for cost and technical reasons, so the actual pool of respondents becomes biased.

Systematic Sampling

This method is often preferable to simple random sampling, as you select members of the population systematically – that is, every Nth record. As long as there is no ordering of the list, the sampling method is just as good as random – only much simpler to manage.

Stratified Sampling

This is a more commonly used technique, and the population is divided into subsets with a common trait, or “strata”, and then random sampling is performed to reduce sampling bias. The key is to ensure that the sample size is large enough to represent the population.

Non Probability Sampling Methods

Convenience Sampling

One of the most cost-effective sampling methods, researchers choose this method as they can recruit the sample from the population that is close at hand, or convenient to them. It is up to the researcher to ensure that a large enough sample is chosen that can closely represent the population being studied.

An extension of this is judgment sampling, where the research selects a representative sample based on their judgment.

Quota Sampling

Very similar to stratified sampling, where the researcher defines the segments or “stratums” and their representative proportion in the population – quota sampling differs in that respondents are typically filled by convenience or judgment sampling, vs random.

Snowball Sampling

There is another method of acquiring respondents called snowball sampling, where initial subjects refer others to take the survey.

Examples of Survey Bias

Survey bias can rear its ugly head many times during the creation of a survey. From the population you choose unintentionally excluding key respondents, to ensuring you have a sample size that accurately reflects the total population.

You can also create survey bias through the probability or non probability sampling method you select. This is called Sample Bias (or Sampling Bias).

Sample bias is when a sample is collected and, due to the method used, some members of the intended population have a lower probability of being included as others.

Non-Response Bias

Non-response bias (or self-selection bias) can happen when a respondent has knowledge of what the survey is about and can decide whether or not to participate. If the survey offers advanced knowledge of the survey topic and gives users the choice to opt in or out, you may get an increased population of users who know a lot about that topic, and results may underrepresent those who are indifferent or don’t have knowledge on the topic.

Exclusion Bias

Let’s say you are trying to survey “teens who have tried drinking.” You may create a sample that targets by education level to get kids in high school who have tried drinking. But this sample would leave out anyone who dropped out or was home-schooled. An age range survey may make more sense.

Pre-Screening Bias

Some sampling methods may run ads in order to get survey participants. But depending on the targeting of these ads, researchers could bias samples. If ads are only targeted to certain sub-groups inside the sample population, this could create a biased sample.