Confidence Interval: How to Read and Report One

What Is a Confidence Interval?

A confidence interval is a range of values computed from sample data by a stated procedure. If the same study were repeated many times and the procedure applied each time, about 95% of the 95% intervals would contain the true value, provided the assumptions behind the procedure hold. Those assumptions include how the sample was drawn, how the data were collected and the model used for the estimate.

For one interval already computed, do not say there is a 95% probability that the true value lies inside it. The range also does not describe the spread of individual people; it describes how precisely the average, share or difference is estimated. Statistical significance tests one value, usually zero. An interval shows the whole range of values the data remain compatible with.

What Makes an Interval Wide or Narrow

A Worked Example

A fictional household survey interviews 400 households chosen by simple random sampling. Of these, 62% report using a safe water source. All numbers are made up for the example.

The standard error is the square root of (0.62 x 0.38 / 400), which is 0.0243. A simple 95% interval is 0.62 plus or minus 1.96 x 0.0243, that is, 57.2% to 66.8%.

Now suppose the survey actually used clusters of households, and the design effect for this outcome is assumed to be 2.0. The standard error grows by the square root of 2, to 0.0343, and the interval becomes 55.3% to 68.7%. Same data, same estimate, a wider and more honest range. A team that ignores the clustering would report the narrower interval and overstate its precision.

Quadrupling the sample to 1,600 simple random households would narrow the first interval to about 59.6% to 64.4%, a half-width of 2.4 points instead of 4.8.

Comparing Two Groups

Say the same survey finds 55% safe water use among 400 households in a second district. The two simple intervals are 57.2% to 66.8% and 50.1% to 59.9%. They overlap, which tempts a reader to conclude there is no difference.

That shortcut is unreliable. The interval for the difference itself is what matters: 7 points, with a standard error of 3.5 points, giving about 0.2 to 13.8 points. It barely excludes zero, so the data are compatible with a difference ranging from almost none to a large one. Report the interval for the difference, and keep in mind that two groups from different places are not a comparison of a program against its absence.

Cautions

• An interval covers sampling variation only. A narrow interval from a biased sample is precisely wrong. Nonresponse, a flawed sampling frame, and measurement error are not captured. See bias.
• The simple formula is not for every case. For small samples, or shares close to 0% or 100%, it can fall outside 0 to 100% or cover less often than claimed. With 3 of 20 households, the simple formula gives -0.6% to 30.6%; a Wilson interval gives about 5% to 36%. Use a method suited to the data, or ask a statistician.
• The design has to be in the calculation. Strata, clusters and weights all change the variance. Specify them in the analysis software, and see reading a survey estimate for a full case.
• Few clusters. With few clusters, the variance estimate rests on few degrees of freedom and is unstable. As a rough guide, take extra care below about 20 clusters, use a t value in place of 1.96, and avoid reporting precise intervals.
• A wide interval is information. It says the data cannot settle the question. It is not a failure to hide.
• Multiple intervals. If you report many outcomes or subgroups, some will miss the true value by chance.

How to Report One

Give the estimate, the interval, the confidence level, the number of completed interviews, and the design used (for example, "stratified two-stage cluster sample, weighted"). Write the unit with it: "62% (95% CI 55.3% to 68.7%), n = 400 households". Then compare the interval with the change that would matter for the decision, as in statistical vs practical significance. Check an AI-drafted results table for this: ask which design assumption each interval uses, and recompute at least one by hand or in your own software.

Frequently Asked Questions

Does a 95% Interval Mean There Is a 95% Chance the True Value Is Inside?

Not for an interval already computed. The 95% describes the long-run performance of the procedure under its assumptions. In practice, people read the interval as a plausible range for the quantity, which is reasonable when the sample is sound and the assumptions are credible.

Is Margin of Error the Same Thing?

Usually it is the half-width of a 95% interval, so 62% plus or minus 4.8 points. Check what a report means by it, and whether it reflects the design effect.

Can I Use a Confidence Interval With a Non-Random Sample?

The formulas assume random selection. With a purposive or convenience sample, an interval does not describe the population, and any interval should be labeled as describing only the sampling model assumed. Often it is better to describe the group studied and avoid population claims.

What If the Interval Includes My Target?

The data are compatible with meeting the target and with falling short. Say so. A larger or better-designed sample can narrow the range if the decision needs a clearer answer.