What Is a Confidence Interval?
A confidence interval expresses the uncertainty around a data estimate. Understanding it prevents over-confidence in business analytics.
Key Takeaways
- A confidence interval gives a range of values within which the true answer is likely to lie.
- A 95% confidence interval means: if we repeated this analysis 100 times, 95 times the true value would fall in the range.
- Small samples produce wide confidence intervals — be cautious about decisions based on limited data.
What a confidence interval is
A confidence interval (CI) is a range of values within which the true answer is likely to lie, given the data you have. Instead of saying 'our conversion rate is exactly 2.3%', a confidence interval says 'we are 95% confident the true conversion rate is between 1.9% and 2.7%'. The width of the interval reflects the uncertainty in your estimate.
Why it matters for decisions
Most business analytics report a single number — a point estimate — without expressing the uncertainty around it. This creates false precision. A conversion rate of 2.3% based on 100 visitors has a very wide confidence interval (say 1.0–4.5%). Based on 10,000 visitors, the interval is much narrower (say 2.1–2.5%). The same number carries very different reliability depending on sample size.
Confidence intervals in A/B testing
A/B testing is one of the most common business contexts where confidence intervals matter. If variant B has a conversion rate of 2.8% vs variant A at 2.3%, is that a real improvement or random noise? The confidence intervals around each estimate determine whether the difference is statistically significant — i.e., unlikely to be due to chance. Most A/B testing tools calculate this automatically.
The practical takeaway
When reviewing analytics reports, ask: how many data points is this based on? The smaller the sample, the more caution is warranted. A single week's data for a new product launch is barely indicative. A year's data for an established product line is reliable. Context and sample size matter as much as the numbers themselves.