The Core Trade-Off
What Is the Difference Between Probability and Non-Probability Sampling?
Probability and non-probability sampling differ on one point: whether every member of your target population has a known, non-zero chance of being selected.
Probability sampling supports design-based inference under its selection and analysis conditions; it does not guarantee representativeness in the presence of coverage or nonresponse bias. You can calculate the margin of error on your findings and defend that number in front of a donor, evaluator, or government counterpart. If someone asks "how confident are you in this estimate?" you can give a precise statistical answer.
Non-probability sampling does not provide this guarantee. You select participants based on access, judgment, or referral networks. The findings are real and can be conducted rigorously, but you cannot calculate how representative they are. That is not a flaw; it means they answer a different kind of question.
The choice is not about rigor. A well-executed purposive sample for qualitative research is more rigorous than a poorly administered random survey. The choice is about what conclusion you need to draw.
| Factor | Probability Sampling | Non-Probability Sampling |
|---|---|---|
| Statistical representativeness | Yes, calculable | No |
| Sampling frame required | Yes | No |
| Margin of error | Calculable | Not calculable |
| Generalizability | To the defined target population | Cannot generalize statistically |
| Cost and time | Higher | Lower |
| Bias | Quantifiable if frame is incomplete | Inherent and unquantifiable |
| Best suited for | Coverage surveys, baselines, impact evaluations | Key informants, FGDs, rapid assessments |
Before committing to either approach, use the Sampling Calculator to confirm whether the sample size your probability design requires is feasible within your budget and timeline. If it is not, you need to either adjust your precision requirements or reconsider the method.
Probability Methods
Which Probability Sampling Methods Are Used in M&E?
Four probability methods cover most M&E use cases. Each has different requirements for the sampling frame and different implications for field logistics and analysis.
Simple Random Sampling (SRS): Every unit in the population has an equal and known probability of selection. Simple to understand and straightforward to analyze. Requires a complete, accurate list of the full population. Rare in field M&E because complete population lists almost never exist in practice, particularly in rural or informal settlement contexts where program beneficiaries live.
Systematic Sampling: Select every nth unit from an ordered list. For example, every 8th household on a community registration list, or every 5th health facility in a national health information database. Practical for field teams because the selection rule is simple to explain and verify. A household transect qualifies only if the design covers the target population and gives every eligible unit a known positive probability, with a random start. Walking a convenient route and taking every eighth household is not sufficient. The risk: if the list has hidden periodicity (e.g., community leaders are listed at the start of each village block), your sample will carry that bias forward.
Stratified Sampling: Divide the population into non-overlapping subgroups (strata) such as gender, district, or wealth quintile, then sample randomly within each stratum. Use this when you need precise estimates for specific subgroups, or when subgroups differ substantially from each other in the outcome you are measuring. Requires knowing the stratum sizes before sampling begins. Produces more precise estimates than SRS when strata are internally homogeneous.
Cluster Sampling: Group units into clusters (villages, schools, health facility catchment areas), randomly select clusters, then survey all or a random subset of units within each selected cluster. The practical choice when a full population list does not exist, which is the standard situation in field surveys. The trade-off: people within clusters are more similar to each other than randomly selected individuals would be, which increases variance relative to SRS. This inflation factor is the design effect. See cluster sampling for technical guidance on calculating design effect and adjusting sample size accordingly.
Choosing the Right Probability Method
| Method | Use when | Avoid when |
|---|---|---|
| Simple random | Complete, accurate population list exists | No list available, population geographically dispersed |
| Systematic | Sequential list available (registration, transect) | List has hidden ordering patterns |
| Stratified | Subgroup comparisons required, strata differ substantially | Stratum membership is unknown before sampling |
| Cluster | No full population list, population is geographically clustered | Budget allows individual selection and a list exists |
The most common field choice in international development M&E is probability proportional to size (PPS) cluster sampling: clusters are randomly selected with probability proportional to their estimated population, then a fixed number of units is randomly selected within each cluster. It handles the absence of population lists, controls for unequal cluster sizes, and is straightforward to audit in the field.
If your program serves multiple districts and requires subgroup comparisons (women vs. men, treatment vs. comparison communities), stratified sampling produces more precise subgroup estimates, but you need population data by stratum before you start. For guidance on this specific trade-off, see cluster vs. stratified sampling.
When in doubt between methods, default to cluster sampling with design effect adjustment. It is the most field-practical probability method and the most defensible when a complete population list is unavailable, which is the case in the majority of household surveys conducted in low- and middle-income country contexts.
Non-Probability Methods
Non-Probability Sampling Methods
Five non-probability methods cover the full range of qualitative and access-constrained M&E contexts. Each is legitimate for certain purposes and inappropriate for others.
Purposive (Judgmental) Sampling: You select participants based on specific characteristics or expertise directly relevant to your research question. This is the standard approach for key informant interviews, expert panels, and case study site selection. The criteria for who is selected and who is excluded must be documented explicitly and justified in your methodology section.
Convenience Sampling: You select whoever is accessible at the time of data collection. The easiest method to execute and the most subject to selection bias. Acceptable when genuine access constraints make other methods impossible, but the limitation must be stated clearly in the report and findings must not be generalized beyond the people sampled.
Snowball Sampling: Existing participants refer subsequent participants into the study. Used to reach populations that are hidden, stigmatized, or otherwise difficult to access: people living with HIV, internally displaced persons, undocumented migrants, survivors of gender-based violence. Cannot produce statistically representative findings, but is sometimes the only ethical path to reaching these communities.
Quota Sampling: Set target counts for subgroups (e.g., 25 women and 25 men) and fill them by convenience within each group. Looks like stratified sampling on the surface but lacks random selection within subgroups, which means selection bias is unquantified. Common in rapid assessments and market research. Acceptable when stratified random sampling is not feasible and you need approximate subgroup representation.
Maximum Variation Sampling: Select participants who differ as widely as possible across key variables: age, geography, program exposure, livelihood type, ethnicity. Used in qualitative studies specifically to capture the full range of experience and perspective rather than to estimate a central tendency. The goal is conceptual breadth, not statistical representativeness.
When Does Non-Probability Sampling Make Sense?
Non-probability sampling is the right choice in three scenarios.
Qualitative data collection: Focus group discussions, key informant interviews, case studies, and most significant change processes all use non-probability methods by design. The goal is depth and variety of perspective, not a statistically representative estimate. There is no inherent problem with this; it is the methodologically correct approach for qualitative inquiry and has a well-established evidence base in evaluation science.
Access-constrained contexts: When you are operating in active conflict areas, working with displaced populations, or in communities where constructing a sampling frame is impossible within your timeline, a non-probability sample with a clearly documented and justified selection logic is better than a probability design that cannot be executed properly. A convenience sample honestly acknowledged and bounded is more defensible than a "random" sample that was not actually random in practice.
Rapid operational assessments: When speed is the overriding constraint and findings will inform internal operational decisions rather than public accountability or donor reporting of coverage figures, non-probability methods are acceptable. The limitation must appear clearly in the report and findings must be scoped accordingly.
Do not choose non-probability sampling to avoid the cost or logistics of a sampling frame. If your findings will be used to report coverage rates, beneficiary reach, or program effectiveness to any external audience, probability sampling is required. That cost is part of the program, not an optional extra.
Sector Examples
Sector Examples
The scenarios below are illustrative. Their figures, costs, and review outcomes are teaching assumptions, not reported studies or evidence of donor approval.
Health: Vaccine Coverage Survey in Rural West Africa
A district health team needed to report measles vaccination coverage across 40 health zones with no household registry available. Using probability proportional to size cluster sampling: 20 villages were selected with PPS weighting, then 7 households randomly selected per cluster for 140 total interviews. The design effect of 1.8 was applied, reducing the effective sample size to 78, which gives an approximate worst-case 95% margin of error of 11.1 percentage points at p = 0.5, ignoring finite population correction and small-cluster degrees of freedom. It does not establish 8-point precision without further assumptions. The resulting coverage estimate was used for both program reporting and the district health ministry's dashboard. The method was field-auditable: enumerators could show supervisors exactly which households were selected and why.
Education: Understanding School Dropout in Urban East Africa
A research team studying why girls leave secondary school in urban informal settlements needed depth of understanding, not a prevalence figure. No sampling frame existed for the dropout population. They used purposive sampling: 6 schools selected for variation in dropout rates (2 low, 2 medium, 2 high), 5 focus group discussions with girls who had withdrawn from school, and 12 key informant interviews with teachers, parents, and community leaders. The purpose was to map mechanisms and barriers, not estimate how many girls had left school across the city. Probability sampling was neither feasible nor appropriate for this research question.
WASH: Latrine Coverage in a Food Security Program
A food security program in Southern Africa needed to report end-of-project latrine coverage across 6 target villages. No population list was available. The team first defined village boundaries and listed or mapped all eligible households, including side streets and remote compounds. Within the resulting coverage frame it used a random start and every eighth listed household, recording inclusion probabilities, callbacks, and nonresponse. Without those conditions a walked transect is a route-based nonprobability sample, and a design-based margin of error is not justified.
Livelihoods: Market System Actor Mapping in East Africa
A livelihoods program needed to understand the barriers facing women traders in a regional market system. No list of market actors existed and the population was not bounded in any usable way. The team used snowball sampling starting from known women's association leaders, expanding through referrals to traders, brokers, input suppliers, and transport providers. The purpose was to map the market system and identify entry points for program support, not to count or profile the trader population statistically. The findings directly informed which value chain actors the program prioritized for capacity-building activities.
Common Mistakes
Common Mistakes
Mistake 1: Calling convenience sampling "random." In everyday language, "random" means haphazard or unplanned. In statistics, it means every unit has a known and calculable probability of selection. These are not the same. Labeling a convenience sample as random in your report is inaccurate and will undermine your credibility with any external evaluator who reads carefully. Use the correct term and explain your selection approach plainly.
Mistake 2: Using purposive sampling for coverage estimates. Purposive sampling tells you about the cases you selected, not the population they came from. If you conduct key informant interviews with 15 purposively selected community health workers and 12 say they received training, that is 80% of your sample. It is not 80% of all CHWs in the district. Reporting this as a coverage figure is a category error that invalidates the conclusion.
Mistake 3: Ignoring design effect in cluster samples. Analyzing cluster sample data as if it were simple random sample data produces confidence intervals that are too narrow. You have more variance in your estimates than SRS assumptions account for, because people within clusters are more similar to each other than randomly selected individuals would be. Adjust your sample size for design effect before data collection and account for it in analysis. See how to choose sample size for design effect calculation guidance.
Mistake 4: No documentation of selection logic for non-probability samples. Every non-probability sample needs a written record: who was selected, the criteria used, who was excluded and why, and how participants were approached. Without this documentation, your methodology cannot be replicated, assessed for systematic bias, or defended in an evaluation quality review. One paragraph in the methods section is the minimum.
Mistake 5: Presenting sample percentages as population estimates. A transparent descriptive statement such as 8 of 10 purposively selected interviewees mentioned a theme can be appropriate if coding and denominator are meaningful. It does not estimate population prevalence or carry a design-based margin of error. Counts can also flatten meaning; report themes, context, and negative cases where better suited.
Related Resources
If you are working through a sampling decision for an upcoming survey or evaluation, these resources cover the next steps.
For calculations: The Sampling Calculator calculates required sample size based on population size, desired precision, and confidence level. It adjusts for design effect when you are using cluster sampling and adds a non-response buffer. Run this calculation before finalizing your data collection budget.
For method detail: Sampling Methods provides a full taxonomy of probability and non-probability approaches with guidance on when each applies in M&E contexts. Random Sampling covers technical definitions and the important distinction between random selection (who you include) and random assignment (who gets the intervention), which is a frequent source of confusion in impact evaluations.
For qualitative sampling documentation: Purposive Sampling explains how to document and justify a purposive selection strategy in ways that satisfy reviewers and meet common evaluation quality standards.
For method-to-method comparisons: Cluster vs. stratified sampling helps you decide between the two most common probability approaches once you have confirmed you need probability sampling. How to choose sample size handles the n calculation and design effect adjustment once your method is confirmed.
If your program uses both quantitative and qualitative data collection, see qualitative vs. quantitative vs. mixed methods for how to sequence the two strands and avoid the common error of treating qualitative findings as if they carry quantitative weight.