The Strange Math That Predicts (Almost) Anything

A Russian math rivalry over independence produced Markov chains, the engine behind modern prediction from nuclear bombs to Google.

The Feud That Started It

• Pavel Nekrasov claimed the Law of Large Numbers required independent events, tying it to free will.
• Andrey Markov proved dependent events also converge, analyzing 20,000 letters in Pushkin's Eugene Onegin (0:00-10:02).

Monte Carlo Method

• Stanislaw Ulam conceived statistical sampling while playing solitaire; John von Neumann added Markov chains to track neutron states.
• Simulations on ENIAC calculated the neutron multiplication factor k for bomb design (10:04-20:07).

PageRank and Search

• Google models the web as a Markov chain of pages and links, using a 15% damping factor to avoid dead ends.
• Early search engines ignored this; Page and Brin prioritized result quality over ad views (20:07-30:10).

Modern Uses

• Predictive text and LLMs extend Claude Shannon's letter-by-letter Markov models.
• Seven riffle shuffles randomize a 52-card deck; other methods need thousands (30:11-32:23).

00:00:00The Law of Large Numbers

• 00:00:00 Core idea: as you repeat independent trials, the average of outcomes converges to the expected value. This is the Law of Large Numbers, proven by Jacob Bernoulli in 1713 and foundational to probability theory when independence holds.
• 00:01:00 Independent events example: fair coin flips. Early probability assumed independence; each trial does not influence the next.
• 00:02:30 Social data and independence: Nekrasov argued that social statistics (crime, births, marriages) follow the same pattern, implying underlying independent decisions (free will) drive these statistics.
• 00:03:40 Nekrasov vs. Markov: Nekrasov believed large-sample convergence implies independence; Markov challenged that dependent processes can still converge to the same law, breaking the link between convergence and independence.
• 00:04:37 The memoryless caveat (leading to Markov): Markov showcased that dependencies (like letter sequences in a poem) can still produce a convergent distribution, undermining Nekrasov’s claim that convergence proves independence.

00:04:37What is a Markov Chain?

• 00:04:37 Core idea: a Markov chain is a memoryless stochastic process where the next state depends only on the current state, not on the past history. This is the memoryless property.
• 00:05:30 Two-state example: vowels and consonants as states; transitions defined by empirical probabilities (e.g., 13% chance from vowel to vowel, 87% to consonant).
• 00:06:30 Building the diagram: transitions are arrows between states with probabilities summing to 1 from each state.
• 00:07:25 Simulation: start in a state (e.g., vowel) and generate next state by a random draw against the transition probabilities; over time, the long-run distribution emerges (43% vowels, 57% consonants).
• 00:08:30 Insight: Markov chains show that dependence within a system does not preclude eventual convergence to a steady-state distribution.
• 00:09:43 Final takeaway: Markov chains enable probability with dependent events and can model many real-world processes where current state drives future ones.

00:09:43Ulam and Solitaire

• 00:09:43 The Solataire motivation: Ulam, during recovery from encephalitis, wondered about the chances of winning a randomly shuffled deck; exact calculation is intractable due to 52! possible orders.
• 00:10:40 Monte Carlo impulse: instead of analytic enumeration, simulate many games to estimate the probability of winning; this seeds Monte Carlo thinking.
• 00:11:25 Difference from other problems: neutrons in a bomb are not independent; their behavior depends on position and history, so a naïve card-shuffle approach won’t work.
• 00:12:21 Markov chains re-emerge as a tool to model a chain of dependent events (neutrons triggering fission) with varying transition probabilities.
• 00:13:10 Von Neumann’s insight: need a chain of events where each step influences the next; Markov chains provide a practical framework to approximate complex systems via repeated stochastic steps.

00:12:21Nuclear Fission

• 00:12:21 Ulam’s re-emergence: to understand how neutron behavior affects chain reactions, a stochastic model helps estimate the multiplication factor k (average neutrons produced per neutron).
• 00:13:50 Markov chain in physics: states can be “neutron travels, scatters, absorbed, or causes fission,” with transition probabilities depending on energy, position, and material configuration.
• 00:14:40 ENIAC: the first electronic computer used to simulate chains of neutrons, calculating k by stepping through many trials and compiling the results into a histogram.
• 00:15:46 The Monte Carlo method is born: random sampling to approximate complex differential equations; the term comes from the Monte Carlo Casino, reflecting gambling-like sampling.
• 00:16:32 Early impact: Argonne and other labs quickly adopted Monte Carlo methods for reactor design and other hard problems.
• 00:17:40 Quote: Ulam observed how a few chalk marks changed the course of human affairs; Monte Carlo methods revolutionized computational science.

00:15:46The Monte Carlo Method

• 00:15:46 Core idea: approximate solutions to difficult problems by random sampling and averaging outcomes over many trials.
• 00:16:32 History: Ulam + von Neumann; from particle physics to broader computational science.
• 00:17:50 Conceptual workflow: model a system with a stochastic process (e.g., chain reactions), simulate many random runs, gather statistics (like k), and estimate the desired quantities.
• 00:18:40 Why Monte Carlo works: many complex systems are too hard to compute analytically but can be explored via stochastic sampling.

00:16:32The first search engines

• 00:19:07 Yahoo origin story: Jerry Yang and David Filo created Yahoo; Masayoshi Son offered $100M; marketing competition among search engines dominated the early internet.
• 00:20:15 Limitation of early search: ranking by keyword frequency could be gamed by keyword stuffing; quality and usefulness of results weren’t measured.
• 00:21:15 Libraries analogy: card catalog endorsements inspired a quality metric similar to popularity or trust signals.

00:19:07Google is born

• 00:21:50 Brin and Page’s insight: model the web as a network of endorsements; each link is a vote that is worth less as more pages link out (diminishing value).
• 00:22:40 Markov chain framing: the web graph becomes a Markov chain where the next page is chosen according to link structure with a damping factor to avoid dead ends and ensure exploration.
• 00:23:45 PageRank: ranking pages by long-run visitation probability in the Markov chain; the more time spent on a page, the more important it is.
• 00:24:40 Initial reception: some feared that better ranking would reduce ad revenue; Brin and Page argued that a superior product would attract users regardless.
• 00:25:16 Go-to-line: “This is a democracy that works. If all pages were equal, anybody can manufacture as many pages as they want.”
• 00:26:45 Outcome: by 1998, Google emerged and, over a few years, dethroned Yahoo as the dominant search engine; Google’s parent company, Alphabet, became enormously valuable.
• 00:27:30 Core takeaway: at the heart of Google is a Markov chain model (PageRank) that ranks pages by stationary distribution with a damping factor, improving relevance dramatically.

00:25:16How does predictive text work?

• 00:25:16 The idea: early information theory and Markov chains underpin text prediction; context length affects accuracy.
• 00:26:30 Claude Shannon’s exploration: using two-gram (two preceding tokens) and then moving to longer sequences to see how predictability improves when you remember more context.
• 00:27:10 Tokens vs. letters: modern language models use tokens (which can be characters, words, or punctuation) as the basic units; longer memory improves predictive power.
• 00:28:15 Attention in modern models: unlike simple Markov chains, large language models use attention to weigh past context and focus on the most relevant parts of the input sequence.
• 00:29:41 Limitation: if training data contains feedback loops (models trained on data produced by prior models), the system can drift toward a dull, repetitive state; non-Markovian feedback challenges.

00:27:10Are Markov chains memoryless?

• 00:27:10 Key property: Markov chains are memoryless – the next state depends only on the current state, not on how you got there.
• 00:28:30 Caveat: in some complex systems, memory and feedback loops make Markov modeling less effective; e.g., climate systems with feedback (water vapor increasing temperature) require more nuanced modeling.
• 00:29:41 Insight: for many dependent systems, Markov chains provide a powerful simplification that still yields meaningful predictive power.

00:29:41How to perfectly shuffle a deck of cards

• 00:30:11 Question posed: how many shuffles are needed to achieve randomness in a 52-card deck?
• 00:30:50 Classic result: seven riffle shuffles suffice to make all arrangements almost equally likely (a Markov chain on the 52! states, with seven steps to approximate uniformity).
• 00:31:20 Personal experience: the host demonstrates a different shuffle and asks how many such shuffles would be needed to approach randomness; a playful segue into probabilistic thinking.
• 00:31:50 Brilliance plug: Brilliant.org is introduced as a way to explore these questions interactively and build mental models.
• 00:32:20 Final take: seven riffles is the established rule of thumb for near-random shuffles; if you shuffle differently, you may not reach true randomness yet.

00:32:20 Resources and wrap-up

• 00:32:20 Sponsor and resources: Brilliant.org link for a 30-day free trial; Snatoms; newsletter; etc.
• 00:33:20 Closing note: the power of these simple ideas (law of large numbers, Markov chains, Monte Carlo) to explain a wide range of phenomena—from card shuffles to search engines to predictive text.

Resources Mentioned

• https://ve42.co/RefsMarkov
• https://ve42.co/7Z
• brilliant.org/veritasium
• https://ve42.co/SnatomsV
• https://ve42.co/Newsletter
• Additional referenced experts: Brian Hayes, David Aldous, Geoff Engelstein, Jeffrey Rosenthal, Jimmy He, Mark Priestley, Michael Choi, Peter Norvig, Sam Power, Thomas Haigh, Madeline Johnson.

Visuals and diagrams described

• A two-state Markov chain (vowels and consonants) with arrows representing transitions; visually demonstrates dependence and memoryless property.
• A toy four-page web with states Amy, Ben, Chris, Dan; transitions show how PageRank emerges as the stationary distribution of a Markov chain with damping.
• The “cards” concept used to discuss shuffling: deck as a Markov chain with 52! states; seven riffles approximate a uniform distribution over states.
• A historical timeline tying together the Law of Large Numbers, Markov chains, Monte Carlo, and PageRank, showing how these ideas evolved from mathematics to computing and the internet.

Notable lines (verbatim)

• >Thus, free will is not necessary to do probability.
• Googling is like oxygen to teenagers.
• This is a democracy that works.
• The Monte Carlo method was born.
• It is still an unending source of surprise for me to see how a few scribbles on a blackboard could change the course of human affairs.
• If all pages were equal, anybody can manufacture as many pages as they want.
• The damping factor makes sure that we explore all possible parts of the web without ever getting stuck.
• Markov chains are memoryless; you can ignore almost all of the past and focus on the current state for predictions.

Definitions and explanations introduced

• Law of Large Numbers: the average of the results from a large number of trials should be close to the expected value.
• Independence: the outcome of one trial does not affect others.
• Dependent events: outcomes influenced by previous events; not independent.
• Markov chain: a memoryless stochastic process where the next state depends only on the current state and not on the past history.
• Transition probability: the probability of moving from one state to another in a Markov chain.
• Stationary distribution: the long-run distribution of states that the chain converges to, regardless of starting state, under certain conditions.
• Damping factor (PageRank): a probability (often around 0.85) that a surfer follows a link; with 15% probability, the surfer jumps randomly to any page. This ensures ergodicity and prevents dead ends.
• PageRank: a link-based ranking algorithm that scores web pages by their relative importance as determined by the web’s link structure modeled as a Markov chain.
• Monte Carlo method: a computational algorithm that relies on repeated random sampling to obtain numerical results, typically used to estimate complex integrals or solve differential equations.
• ENIAC: one of the first electronic general-purpose computers used to perform large stochastic simulations.
• Tokens: basic units used by language models (could be letters, words, or punctuation) as the atomic units for prediction.
• Attention: mechanism in modern language models that weights different parts of the input context to improve prediction quality beyond simple Markov memorylessness.

Key ideas summarized

• Memorylessness does not preclude useful probabilistic modeling: many real-world systems are dependent, yet Markov chains can capture essential dynamics and converge to meaningful steady states.
• The ability to approximate complex, high-dimensional processes (like neutron diffusion, web ranking, language prediction) hinges on replacing intractable exact calculations with simulations and probabilistic reasoning.
• The same mathematical ideas (Markov chains, Monte Carlo methods, and chain-based ranking) underlie diverse technologies we rely on daily (nuclear physics simulations, search engines, and predictive text).

Important contrasts highlighted

• Independence vs. dependence: independence makes some results clean, but many real systems are dependent; Markov chains generalize probability to dependent sequences.
• Deterministic computation vs. stochastic simulation: when exact analysis is impossible, stochastic sampling provides practical estimates.
• Simple Markov chains vs. attention-based models: modern NLP models extend beyond memoryless assumptions by using attention to weigh long-range dependencies.

Potential applications and implications

• For data scientists: use Markov chains to model sequential data where current state provides the primary predictive signal; consider damping to ensure ergodicity.
• For ML practitioners: distinguish when a simple Markov assumption suffices versus when attention mechanisms and longer-context modeling are necessary.
• For historians of science: this narrative shows how a mathematical feud contributed to a cascade of ideas shaping modern computation and the internet.

Final notes

• The video ties a century of mathematics to real-world technologies, showing how a few core ideas—law of large numbers, Markov chains, Monte Carlo methods—produce profound predictive power across domains.
• The deck-shuffle example cements a memorable conclusion: Markov chains can quantify how many steps are needed to approach randomness, with seven riffles standing as a practical rule of thumb for a standard deck.

Resources Mentioned (recap)

• https://ve42.co/RefsMarkov
• https://ve42.co/7Z
• brilliant.org/veritasium
• https://ve42.co/SnatomsV
• https://ve42.co/Newsletter
• References to Ulam, von Neumann, ENIAC, Markov, Nekrasov, Brin, Page, Shannon, and the broader history of probability, computation, and search.