Standard deviation, standard error, and the 95% CI
How to read and interact with this page
- Enter what you observed. S correct out of N, or click a preset. Everything on the page follows from those two numbers.
- Read the three bars. All three sit around the same observed accuracy. The SD bar is the spread of the individual correct/wrong outcomes; the SE bar is how much the accuracy itself would wobble on a fresh test set; the 95% CI is the interval the textbook recipe produces.
- Read the three cards. Each gives the number, the formula and the sentence you are allowed to say about it.
- Grow N. Click 4 / 5, then 80 / 100, then 8,000 / 10,000, and watch which bars shrink. The SD bar does not; that is why it should not be on an accuracy chart.
Experiment results
Observed accuracy —
Uncertainty measures
Three error bars from the same two numbers
Dot: observed accuracy. Bars clipped at 0% and 100% show an arrow where they run past the edge.
Standard deviation
Spread of the individual correct/wrong outcomes.
SD = √( p̂ (1 − p̂) )—
Fixed by p̂ alone: it tells you nothing the accuracy did not already tell you, and it does not shrink as N grows. An SD bar on an accuracy chart is almost always a mistake. (An SD across repeated runs or folds is a different quantity and can be legitimate.)
Standard error
How much the observed accuracy would scatter across fresh test sets of size N.
SE = SD / √N—
A statement about the estimate, not about individual answers. It shrinks with √N: 100× the data buys 10× the precision.
95% confidence interval
The interval the recipe produces from this one eval.
Wilson score interval—
Not: “95% probability the true accuracy is in here”. But: “repeat the eval; 95% of the intervals built this way catch the true accuracy.” A property of the recipe, not of this interval. For a probability about the accuracy you need a belief: see Estimating Accuracy.