Hidden order: The central limit theorem

Imagine dropping a ball through a board filled with pegs. Each time the ball hits a peg, it has an equal chance of bouncing left or right.

One ball follows an unpredictable path. But if we drop thousands of balls, they form a smooth pile: most land near the middle, and fewer land near the sides. The pile looks like a bell.

Why does randomness create such an orderly shape?

Turning bounces into numbers

We can represent each bounce with a number:

The small $i$ labels the bounce. $X_1$ is the first bounce, $X_2$ is the second, and so on.

Adding all the bounces tells us where the ball finishes:

Left and right bounces mostly cancel. After 100 bounces, the typical distance from the middle is on the scale of

steps, not 100 steps.

The central limit theorem

The central limit theorem describes what happens when we add many independent random changes.

“Independent” means that knowing one change does not tell us what the next change will be. Let every change have average $\mu$, pronounced “mu,” and a typical spread $\sigma$, pronounced “sigma.”

To compare sums of different sizes, we centre and resize the total:

Each part has a job:

• $X_1+\cdots+X_n$ adds all the changes.
• The expected total is $n\mu$, so subtracting it moves the centre to 0.
• The total’s typical spread is $\sigma\sqrt{n}$, so dividing by it puts sums of different sizes on the same scale.

As $n$ grows, a histogram of $Z_n$ gets closer to the same bell-shaped curve:

The curve is tallest near $z=0$, so results close to the average are common. Farther from 0, the curve becomes lower, so very large differences from the average are rare.

The individual changes do not have to look like a bell. They might be coin flips, measurement errors, or left-right bounces. Under the theorem’s conditions—especially independence and a finite spread—their centred and resized sum approaches this curve.

Prediction without predicting one path

We cannot predict the exact left-right path of one ball. We also cannot know exactly where one future ball will land.

But when we release thousands of balls, we can predict the shape of the whole pile. Most balls gather near the middle. Far fewer reach the edges.

That is the hidden order: each path is random, but the collection follows a stable mathematical pattern.

What should you remember?

• Independent random changes can be added together.
• Their total spreads on the scale of $\sqrt{n}$.
• After centring and resizing, many kinds of sums approach the same bell curve.
• One result remains unpredictable, while the overall pattern becomes predictable.

Next: How next-token guesses grow into sentences →

Back to the Hidden order series