Sampling Methods

When to Use

Sampling is the right approach when you need to make inferences about a population but cannot or should not measure every unit. Use sampling when:

• A census is impractical: your beneficiary population is too large, dispersed, or fluid to enumerate completely within budget and time constraints
• Statistical inference is required: you need to estimate population parameters (means, proportions) with known precision and confidence levels
• Resource constraints exist: budget, staffing, or time limitations make full enumeration impossible
• Quality over quantity matters: you can achieve higher data quality with a smaller, well-managed sample than a rushed census

Sampling is less appropriate when:

• Population is small: if your entire beneficiary population is under 100-200 units, a census is often more practical and eliminates sampling error
• You need unit-level precision: if every individual household or beneficiary must be measured (e.g., for targeted assistance distribution), sampling won't work
• Sub-population analysis is critical: if you need reliable estimates for small, specific subgroups (e.g., female-headed households in one district), you may need stratified sampling with oversampling or a different design

How It Works

Effective sampling follows a structured sequence. Each step builds on the previous one.

1. Define the target population. Be explicit about who is in scope: geographic boundaries, inclusion/exclusion criteria, and the unit of analysis (household, individual, facility). This definition determines your sampling frame requirements.

2. Determine sample size. Calculate the minimum sample needed based on your desired precision (margin of error), confidence level (typically 95%), and expected prevalence of key indicators. Account for design effects if using cluster sampling (typically 1.5-2.0) and non-response (typically 10-20%).

3. Select the sampling method. Choose based on your population characteristics, available frame, and analysis needs:

   • Simple random sampling (SRS): every unit has equal probability; ideal when you have a complete frame and population is homogeneous
   • Systematic sampling: select every kth unit after a random start; practical when you have an ordered list
   • Stratified sampling: divide population into subgroups (strata) then sample within each; ensures representation of key subgroups
   • Cluster sampling: sample groups (clusters) then units within clusters; cost-effective for dispersed populations
   • Multi-stage sampling: combine methods across selection stages; common in large surveys

4. Develop the sampling frame. Create or verify the list from which you will draw your sample. The frame should be complete, current, and accurate. Document any gaps or known coverage errors.

5. Implement selection. Use random number generators or systematic procedures to select your sample units. For cluster sampling, document cluster selection and within-cluster selection procedures clearly.

6. Manage non-response. Track response rates at each stage. Plan for follow-up attempts and document reasons for non-response. Avoid substituting non-respondents with convenience selections, which introduces bias.

7. Document everything. Record all sampling decisions, frame sources, selection procedures, and response rates. This documentation enables others to assess validity and replicate the approach.

Key Components

A robust sampling design includes these essential elements:

• Clear population definition: explicit inclusion/exclusion criteria, geographic boundaries, and unit of analysis that align with your evaluation questions
• Sample size justification: documented calculation showing how you arrived at your sample size, including assumptions about prevalence, precision, confidence level, design effect, and expected non-response
• Sampling frame: the actual list or mechanism from which you draw your sample, with documentation of its source, completeness, and known limitations
• Selection procedure: step-by-step description of how units are selected, including randomization methods, random start points, and any systematic intervals
• Stratification variables: if using stratified sampling, clear rationale for strata and allocation method (proportional or optimal)
• Cluster selection protocol: for cluster sampling, documented method for selecting clusters and within-cluster units, including any probability-proportional-to-size procedures
• Non-response management: planned follow-up procedures, substitution rules (or lack thereof), and analysis of non-response bias
• Quality controls: verification steps to ensure selection was executed as planned, including spot checks and documentation review

Best Practices

Match the sampling method to your population structure. Simple random sampling works when your population is homogeneous and you have a complete frame. Systematic sampling is efficient when you have an ordered list and the ordering isn't correlated with your outcome of interest. Stratified sampling overcomes problems in simple random sampling by splitting the sample into sub-groups, then randomly selecting respondents within each group to ensure representation.

Use stratified sampling when sub-group analysis matters. If 38% of the population is college-educated and 62% have not been to college, then 38% of your sample should be randomly selected from the college-educated stratum and 62% from the non-college stratum. This proportional allocation reflects the population shares but does not guarantee subgroup precision. Calculate precision for each required subgroup; small strata may need disproportionate allocation and weights.

Document all sampling procedures in a detailed protocol before data collection begins. Your protocol should specify the cluster selection method, within-cluster sampling procedures, randomization methods, and non-response handling. This documentation is essential for assessing validity and enabling replication.

Within selected clusters, units must be selected using simple random sampling or systematic random sampling to maintain the validity of statistical inferences. Using convenience selection within clusters invalidates your error rate calculations and undermines the probability basis of your design.

Choose systematic sampling or SRS for the actual frame. Systematic selection can be practical with a random start and a suitable ordered frame. Check periodicity and inclusion probabilities. SRS is valid and may be preferable when ordering creates risks.

Use random number generators or systematic sampling with random starts for cluster and unit selection to ensure true randomization. Manual selection introduces unconscious bias. Use validated randomization tools and document the random seed or start point.

Plan nonresponse follow-up and adjustments. Increase the initial probability sample where needed for expected response losses, pursue consent-respecting follow-up, and assess differential response. Weighting requires defensible response assumptions and suitable auxiliary data; extra observations do not automatically remove nonresponse bias.

Find a way of selecting samples that is practical, fits within your budget, and avoids major sources of bias. The ideal sampling method is useless if it cannot be implemented. Balance statistical purity with field realities, but never sacrifice randomization for convenience.

Common Mistakes

Applying simple random sampling formulas to cluster data. Cluster sampling introduces design effects that inflate variance. Applying SRS formulas to cluster data severely underestimates standard errors and produces artificially narrow confidence intervals, leading to false precision in your estimates. Always account for design effect in your sample size calculation and use cluster-robust standard errors in analysis.

Using outdated or inaccurate sampling frames. Coverage errors consist of omissions, erroneous inclusions, duplications, and misclassifications of units in the survey frame. Using an outdated beneficiary list, for example, leads to coverage bias and unrepresentative samples, regardless of how well the sampling procedure is executed. Verify your frame against current program records and document known gaps.

Substituting non-responding households or clusters. Replacing non-respondents with convenience selections introduces unknown bias and invalidates the error rate calculations. Report and analyze only the actual sampled units. If non-response is high, document the rate and conduct non-response bias analysis rather than substituting.

Using invalid sampling strategies. Some approaches fundamentally undermine probability sampling: selecting your friends and family, web surveys where respondents self-select, or phone-in surveys where respondents must call in. These convenience methods introduce severe selection bias and cannot support statistical inference about a population.

Underestimating cluster sampling bias risks. Cluster sampling method is potentially biased, as some households may not be available and willing to answer a survey. This non-response within clusters can introduce bias if the non-respondents differ systematically from respondents. Plan for adequate follow-up and document response rates at the cluster level.

Examples

These are illustrative designs, not verified field studies or standard sample-size requirements.

Agricultural Livelihoods - East Africa (Stratified Cluster Sampling)

A 50,000-household agricultural resilience program across 10 districts needed baseline food security data. The team used stratified two-stage cluster sampling: first, districts were stratified by agro-ecological zone; second, 50 clusters were selected with probability proportional to size; third, 20 households per cluster were selected using systematic sampling with random start. The design accounted for a design effect of 1.5 and 15% expected non-response. This approach achieved a representative sample of 850 households while keeping field costs manageable. The stratification ensured each agro-ecological zone was represented proportionally, enabling reliable zone-level comparisons.

WASH Program - South Asia (Systematic Sampling)

A water and sanitation program serving 3,000 beneficiary households used systematic sampling for midline evaluation. The team obtained an ordered list of households from program records, calculated a sampling interval of 10 (3,000 / 300 sample size), selected a random start between 1-10, and then selected every 10th household. This approach was practical for field teams, required only a printed list and random number generator, and achieved a 92% response rate. The ordered list was by village and household registration date, which the team verified was not correlated with water access outcomes.

Emergency Response - West Africa (LQAS for Classification)

For nutrition classification, first define eligible children, the household and within-household selection scheme, lower and upper prevalence levels, and acceptable errors. Do not classify a district as below 15% acute malnutrition with 90% confidence solely because at most 3 of 19 sampled children have the condition. Under an illustrative independent binomial model, that rule accepts the district with probability about 0.6841 at 15% prevalence and 0.4551 at 20%. It could therefore miss substantial need. These calculations expose the unsupported guarantee; they are not a replacement clinical protocol or a statement of current severity thresholds. Select and verify an appropriate design with nutrition and sampling expertise.

Compared To

Sampling methods vary in their assumptions, requirements, and trade-offs:

Relevant Indicators

12 indicators across 4 donor frameworks (USAID, FEWS NET, CHS Alliance, Global Food Security Cluster) relate to sampling design and implementation:

• Sampling method quality: "Proportion of surveys using probability-based sampling methods" (USAID)
• Sample size adequacy: "Sample size justified against desired precision and confidence level" (FEWS NET)
• Response rate: "Non-response rate below 20% for household surveys" (CHS Alliance)
• Frame quality: "Sampling frame documented and verified against program records" (Global Food Security Cluster)

Proposal Context

Sampling decisions in proposals drive both the credibility of planned evaluation and the budget needed to execute it. Common proposal pitfalls: (a) claimed sample sizes without design effect adjustment for cluster sampling (proposed n is under-powered before fieldwork starts), (b) non-probability sampling labeled as representative (purposive or convenience sampling cannot produce representative estimates; labeling them as such is a red flag), (c) sample sizes too small to support the planned disaggregation (total n works for the headline estimate but subgroup n cannot support age-by-sex comparisons), (d) no non-response buffer (budget covers the calculated sample exactly; field attrition leaves the effective sample below threshold), (e) sampling method chosen without explicit justification given the evaluation question. Pair with disaggregation and cluster-sampling where relevant.

Method Source

WHO, World Health Statistics Quarterly 44 (1991), pp. 171-181, LQAS performance levels and classification risks. This is methodological guidance, not a funding requirement.