Intraclass Correlation (ICC) and Effective Sample Size in Cluster Surveys

What Is the Intraclass Correlation?

The intraclass correlation coefficient (ICC, often written rho) is the share of the total variation in an outcome that sits between clusters rather than within them. An ICC of 0 means people in the same village answer no more alike than strangers. An ICC near 1 means everyone in a cluster answers the same way.

It matters because the second interview in a village adds less new information than the first. The ICC tells you how much less, and it feeds straight into the design effect.

From ICC to Design Effect to Effective Sample Size

Three numbers link together. For clusters of roughly equal size m:

• Design effect (DEFF) = 1 + (m - 1) x ICC
• Effective sample size = completed interviews / DEFF
• Required completes = the sample a simple random survey would need x DEFF

The effective sample size is the number of simple random interviews your clustered sample is worth. Two hundred cluster interviews with a DEFF of 1.75 carry the precision of about 114 simple random interviews.

A Worked ICC Estimate

A fictional food security survey interviews 10 households in each of 20 villages (200 completes). The outcome is a household food consumption score. A one-way analysis of variance (ANOVA) on the data gives two mean squares:

The standard ANOVA estimator is:

ICC = (MSB - MSW) / (MSB + (m - 1) x MSW)

ICC = (190 - 100) / (190 + 9 x 100) = 90 / 1,090 = 0.083.

From there:

1. Design effect: 1 + (10 - 1) x 0.083 = 1.75 (1.747 before rounding).
2. Effective sample size: 200 / 1.747 = about 114 simple-random-equivalent households.
3. Planning the next round: to reach the precision of a simple random sample of 385 (95% confidence, 5-point margin, 50% proportion), you need 385 x 1.747 = 672.6, rounded up to 673 completes. At 10 per village that is 68 villages (680 completes).

Round only where you count people: here, once, at the end of each step that gives a number of households or villages.

How Sensitive Is the Result to the ICC?

The same 200 completes at 10 per village, under different ICC values:

A small change in the ICC moves the sample a lot. That is why the ICC assumption deserves a source and a sensitivity check, not a default.

Why Adding Interviews per Cluster Stops Helping

Hold the ICC at 0.10. With 10 per cluster, DEFF is 1.9. With 30 per cluster, DEFF is 1 + 29 x 0.10 = 3.9. Tripling the interviews per village roughly doubles the design effect, so the extra interviews buy little. Adding clusters, not respondents per cluster, is what raises effective sample size. Fewer interviews per cluster usually costs more travel; cost both designs before choosing (see cluster vs stratified sampling).

Where to Get an ICC

• Prior surveys of the same outcome: Published reports from household surveys in the same sector and country often give design effects or ICCs. Match the outcome, cluster type and cluster size.
• Reference manuals: The DHS sampling manual and similar guides publish illustrative values. These are less precise than a local prior survey but better than a guess.
• Your own data: Estimate it with ANOVA or a multilevel model once you have data. Estimates from fewer than about 20 clusters are unstable, and a negative estimate is usually reported as zero.
• Sensitivity range: If you have no source, plan at two or three plausible ICC values and show the cost of each.

Cautions

• The ICC belongs to an outcome, a cluster type and a population. A value for vaccination coverage in villages does not transfer to test scores in schools.
• Binary and continuous outcomes use different estimators, and the numbers are not interchangeable.
• The formula assumes roughly equal cluster sizes. With very unequal clusters, the design effect is larger; use a design-based estimator or specialist advice.
• The ICC says nothing about bias. A well-sized cluster survey can still miss people the sampling frame left out.

Putting It Into the Calculator

The takes a design effect, not an ICC. Work out DEFF with the formula above, enter it, and the tool multiplies the base sample and then adds the non-response allowance. See cluster sampling for how the clusters themselves are chosen, and sample size choices for the full calculation.

Frequently Asked Questions

What Is a Typical ICC for Household Surveys?

There is no universal value. Many household outcomes fall between about 0.02 and 0.15, and outcomes strongly shaped by the cluster, such as school learning, can be higher. Use a value from a comparable survey where you can.

What Is the Difference Between ICC and Design Effect?

The ICC describes how alike people in a cluster are. The design effect uses the ICC and the cluster size to say how much variance rises compared with simple random sampling.

Can the ICC Be Zero or Negative?

It can be near zero, and sample estimates can come out slightly negative from chance. Planning values are normally set at zero or above, which gives a design effect of at least 1.