Uncertainty & dynamics

The world forgets

Weather patterns. Attention spans. Political freedom. Brand loyalty. Addiction.
All Markov chains. All bound by one surprising theorem.

A Markov chain has states and transition probabilities — no memory. Where you go next depends only on where you are, not how you got there. That's it. That's the whole model. Yet from those simple rules comes a startling theorem: any Markov chain (with four mild conditions) converges to a unique long-run distribution called π. Same destination, no matter where you started. Below, watch it happen.

Pick a scenario · watch a walker wander · see time-in-state converge to π

Time steps

Currently in

—

Empirical mix

—

→ approaches π

The bars on the right show the walker's fraction of time in each state so far. The red diamonds mark the theoretical stationary distribution π. Give it a few hundred steps and the bars line up with the diamonds — regardless of where you started. That's the theorem.

The mechanism. Each state has an outgoing set of "if I'm here, where might I go next?" probabilities. Watch the walker's arrows: thicker means more likely. Over time the walker naturally spends most time in the most-connected states. That's what π captures — not "where you are now" but "where you end up if you wait long enough."

Three different starts, one common destination

So a single walker converges to π. But so does a whole population of walkers, even if they all started in different states. Watch three worlds run in parallel.

Three worlds · same rules · totally different starts

Each panel starts 1000 walkers in a different state (all in the first, all in the last, or spread uniformly). Every panel runs the same transition rules P. Watch the coloured stacked areas shift and stabilise — and notice how all three panels end up with the same shape. History dies.

The Markov Convergence Theorem in one sentence. If every state can reach every other, and the chain doesn't cycle in perfect lock-step, then no matter where you start, the state distribution converges to the same π. All three worlds above ended in the same place. That's the theorem — proved by A. A. Markov around 1906, and central to everything from PageRank to political economy.

A one-time fix vs. a change in the rules

Here's where Markov chains reshape how we think about policy. Say you're a mayor with a drug crisis. You can spend the budget on a one-time intervention — pay to send every addict to rehab today — or you can invest in changing the rules — better education, treatment infrastructure, harm reduction. Which one actually moves the equilibrium?

Two policies · one runs the "addiction cycle" chain forever

The chart shows the fraction of the population in each state over time. Click 🚨 One-time intervention to instantly move everyone into "✅ Sober" — then watch it decay back to the original π. Click 🔧 Change the rules to permanently reduce the transition from Sober → Using — and watch a new, lower stationary emerge.

The lesson every policy-maker should tattoo somewhere. One-time interventions always decay — the transition probabilities pull the system back to its original π. Only changing the transition probabilities creates lasting change. This isn't cynicism; it's arithmetic. And it applies to organisations, teams, personal habits, retirement policy, addiction treatment, and just about every "why doesn't this fix stick?" question you've ever asked.

Markov chains are everywhere

Google's original PageRank is the stationary distribution of a random surfer clicking links on the web. High-π pages rank high.
Weather forecasting uses (much bigger) Markov models to predict tomorrow from today.
Democracy studies (Freedom House data) show countries drift among Free / Partly Free / Not Free at nearly stationary rates — one of the more sobering applications.
Card shuffling: how many riffle shuffles until a deck is "random"? Markov analysis says seven (Bayer & Diaconis, 1992).
Speech recognition, machine translation, and reinforcement learning all lean on Markov structure at their core.
Board games: Snakes & Ladders is literally a Markov chain — solvable in closed form.

Related models

🚶 Random walks — the simplest Markov chains in the world: next step depends only on where you are.
🏺 Path dependence — the exact opposite: Markov forgets, Pólya urns remember forever.
🦠 SIR epidemic — SIR at the individual level is a Markov chain on {S, I, R}; averaged over the population it's the ODE.

← Random walks Path dependence →