The idea: to get a sum of s, we enumerate every way X₁ could contribute some amount k while X₂ contributes the rest s − k. Independence lets us multiply those two probabilities, then we sum over all possible splits.
For N variables, we simply convolve repeatedly: PS = PX₁ ⊛ PX₂ ⊛ ⋯ ⊛ PXₙ
Why E[S] = E[X₁] + E[X₂] + ⋯ + E[Xₙ]
This is the linearity of expectation — it always holds, even without independence:
E[S] = E[X₁ + X₂ + ⋯ + Xₙ]
By definition, expectation is a sum (or integral) — and sums can be split:
= E[X₁] + E[X₂] + ⋯ + E[Xₙ]
No conditions needed. This works for any random variables, correlated or not.
For N identical variables with mean μ: E[S] = Nμ. That's all there is to it.
Rearrange into individual variances + a cross term: = Var(X₁) + Var(X₂) + 2(E[X₁X₂] − E[X₁]E[X₂])
The cross term is 2 Cov(X₁, X₂). If X₁ and X₂ are independent, then E[X₁X₂] = E[X₁]E[X₂], so the covariance is zero and:
= Var(X₁) + Var(X₂) ✓
The same argument extends by induction to N variables. For N identical independent variables with variance σ²: Var(S) = Nσ².
Key contrast: expectation always adds. Variance only adds when the variables are independent. If they're positively correlated, variance grows faster than linearly; if negatively correlated, it grows slower.
The Building Block: One Coin Flip
Each flip Xᵢ is a Bernoulli(p) random variable: it equals 1 (heads) with probability p and 0 (tails) with probability 1−p. Every flip is independent of every other.
P(X = 0) = 1−p P(X = 1) = p
E[X] = p Var(X) = p(1−p)
Two Coins: The Convolution Table
To find S = X₁ + X₂, we list all joint outcomes. Independence means we multiply probabilities. Then we collect cells with the same sum.
X₁ ╲ X₂
0 (T) · (1−p)
1 (H) · p
0 (T) · (1−p)
S=0 (1−p)²
S=1 p(1−p)
1 (H) · p
S=1 p(1−p)
S=2 p²
S=1 gets probability from two cells — paths HT and TH both produce a sum of 1, each contributing p(1−p). So P(S=1) = 2p(1−p). This "piling up" in the middle is the essence of convolution — and the seed of the bell curve.
N Coin Flips
As we add more independent flips, S = X₁ + X₂ + ⋯ + Xₙ, the distribution gets more values and the shape converges toward a Gaussian (Central Limit Theorem).
2
E[S] = Np = 1.0
Var(S) = Np(1−p) = 0.5
Each new coin's distribution is convolved with the running total. Variance adds because the coins are independent: Var(S) = Var(X₁) + ⋯ + Var(Xₙ). (Chart uses p = ½.)
The Building Block: One Fair Die Roll
Each roll Dᵢ is uniform on {1, 2, 3, 4, 5, 6}, each with probability ⅙. A richer starting distribution than the coin.
P(D = k) = ⅙ for k ∈ {1,…,6}
E[D] = 3.5 Var(D) = 35/12
Two Dice: The 6×6 Convolution Table
Same principle: list all 36 equally likely joint outcomes, then count how many give each sum from 2 to 12.
Sum=7 has 6 paths (1+6, 2+5, 3+4, 4+3, 5+2, 6+1) while sum=2 and sum=12 have only 1. The triangular shape is the convolution of two uniform distributions.
N Dice: Triangular → Bell Curve
With 2 dice the distribution is triangular. Add more dice and it rapidly smooths into a Gaussian — the CLT at work.
2
E[S] = 3.5N = 7.0
Var(S) = 35N/12 ≈ 5.8
Even though each die is uniform (flat), the sum quickly looks normal. The convolution smears and smooths, and the CLT guarantees convergence.
The Building Block: One Exponential(λ) Variable
Think of it as the waiting time until an event: a server request, a radioactive decay, a bus arrival. It's continuous, always positive, and heavily right-skewed — the exact opposite of a bell curve.
f(x) = λe−λx for x ≥ 0
E[X] = 1/λ Var(X) = 1/λ²
Chart uses λ = 1
N Exponentials: Skewed → Gamma → Gaussian
The sum of N independent Exp(λ) variables is exactly a Gamma(N, λ) distribution. As N grows, even this extremely skewed starting shape becomes bell-shaped.
1
E[S] = N/λ = 1.0
Var(S) = N/λ² = 1.0
The sum S ~ Gamma(N, λ). At N=1 it's a pure exponential (maximally skewed). By N≈5 the skew is visible but softening. By N≈15–20 it's practically Gaussian. The CLT doesn't care that the starting distribution was wildly asymmetric.
The Building Block: One Poisson(λ) Variable
Counts of rare independent events in a fixed interval: emails per hour, typos per page, earthquakes per year. Discrete, non-negative, and skewed for small λ.
P(X = k) = λke−λ / k!
E[X] = λ Var(X) = λ
Chart uses λ = 2
N Poisson Variables: A Beautiful Exact Result
The sum of N independent Poisson(λ) variables is exactly Poisson(Nλ). No approximation needed — Poisson is closed under convolution! And as Nλ grows, it too becomes Gaussian.
1
E[S] = Nλ = 2.0
Var(S) = Nλ = 2.0
Poisson ⊛ Poisson = Poisson. The family is closed under convolution — the sum stays Poisson, just with a larger rate parameter. Mean and variance are both Nλ (they're always equal for Poisson). As Nλ grows large, the distribution approaches Gaussian.
The Building Block: A Weird Trimodal Distribution
This is deliberately pathological — three sharp peaks at different locations with a gap between them. Nothing like a bell curve. The point: the CLT doesn't care.
Three peaks at x ≈ 1, 5, 9
with different heights and widths
E[X] ≈ 4.6 Var(X) ≈ 9.4
As ugly as it gets!
N Trimodals: from Mess to Harmony
Watch the three peaks blur together. By N≈3–4 you can still see bumps. By N≈8 it's basically a bell curve. The CLT is relentless.
1
E[S] = N × 4.6 ≈ 4.6
Var(S) = N × 9.4 ≈ 9.4
This is the strongest demonstration of the CLT: even a trimodal, asymmetric, gapped distribution converges to Gaussian under repeated convolution. Each convolution step smears the peaks further until they merge into a single smooth bell.