One number for surprise
How surprised should you be by a coin, a market, or a rainforest?
Shannon (1948) gave us one number that answers all three. In bits.
Below is a little machine. The four coloured bars are the odds of four outcomes. Grab any bar and drag it up or down — you're rewriting the odds. Samples start firing instantly. Each colored ball that pops out has a "+1.42 bits" chip above it: that's how surprised you should be by that sample. Rare things pop with bigger chips. The average pop size, over many samples, is what Shannon called entropy — one number, in bits, that says "how unpredictable is this?"
Drag the bars · watch the surprise per sample
Same math, second job — diversity
An ecologist calls this the Shannon–Wiener index and uses it to score how varied a forest is. An economist calls $e^H$ the effective brand count and uses it to score market variety. Same formula, three vocabularies. A pine plantation has entropy near zero. A rainforest has entropy near $\log_2 K$. One dominant brand? Low. All niches evenly filled? High.
Same math, third job — one number sorts every story into four kinds
Watch anything long enough — a clock, a market, a fluid, an ecosystem — and Stephen Wolfram (2002) noticed you'll see exactly one of four kinds of long-run behaviour. Entropy is what tells them apart. Instead of proving that here, let's just watch it in the simplest possible playground: a row of pixels that follow a tiny rule.
How the pixel world works. A row of black-and-white cells lives across the top. Every tick, each cell looks at itself and its two neighbours — three cells in all. The rule looks up "when I see this 3-cell pattern, what colour do I become?" — and applies it everywhere at once. The row of pixels evolves downward like a fabric being woven. Different rules → different fabrics.
Below are four of these pixel worlds, all running side by side. Every one uses the setup you just read — three cells in, one cell out — but with a different lookup rule inside. Look at what each rule produces, not at the numbers.
Four rules · four fates · playing live
Now build a rule of your own
So what is a "rule"? It's a lookup table. There are exactly 8 possible 3-cell input patterns — from ░░░ (all off) through ▓▓▓ (all on). For each pattern, the rule says "output on" or "output off". So a rule is just 8 yes-or-no choices — 2⁸ = 256 possible rules in total. Most produce chaos. A few produce stillness. A rare handful produce structures nobody has ever seen twice.
Below is your lookup table. Click any of the 8 output squares to flip it. Watch your custom rule play out live. Or hit a preset — those are the four rules from the zoo above, revealed.
Design a 3-cell rule · click any output to flip it
Or browse the gallery — click any tile
Where entropy shows up
File compression (fewer bits per symbol when a language has lower entropy), cryptography (unpredictability = security), machine learning (cross-entropy is the loss function of almost every classifier), physics (Boltzmann's version differs from Shannon's only in units), diversity measurement in ecology and antitrust. One number. Everywhere.
Related models
- 🎮 Life & Local Majority — the four-class story told with 2-D cellular automata instead of 1-D rules.
- 🔔 Normal / CLT — the normal distribution is max-entropy given mean + variance. One of three canonical max-ent answers.
- 🚶 Random walks — the entropy of a walker's position grows exactly like √t, which is the random walk model's main result.
