The Day No-Limit Hold’em Changed

Read along

The full narration. Tap a line to jump there.

1. 0:00In March 2015, a Polish programmer posted a thread on a poker forum. He was selling a piece of software that cost 249 dollars, and it ran on an ordinary home computer. Nobody held a press conference. No champion was beaten on a stage.
2. 0:13But if you had to pick the day no-limit hold’em changed, that thread is the best candidate we have. To understand why, we have to go back almost a hundred years.
3. 0:231928In 1928, the mathematician John von Neumann proved something called the minimax theorem. In a two-player game where one side’s gain is the other side’s loss, there is always a way to play that cannot be beaten in the long run.
4. 0:35Von Neumann was thinking about poker, not chess. Chess, to him, was just calculation. Poker had hidden cards, and hidden cards meant bluffing.
5. 0:421944In 1944, together with Oskar Morgenstern, he worked through a simplified poker game and showed that bluffing is not a trick or a personality trait. It is part of the mathematically correct strategy.
6. 0:521950In 1950, John Nash extended the idea of equilibrium to games with any number of players. So by 1950, the theory said: a perfect poker strategy exists. Nobody had any idea how to find it.
7. 1:05The problem was size. To solve a game the old way, you had to list every complete strategy a player could follow. For real poker, that list is beyond astronomical.
8. 1:131994In 1994, Daphne Koller, Nimrod Megiddo and Bernhard von Stengel found a way around it. Their method, the sequence form, describes a game in a size that grows with the game tree itself, not with the number of strategies. For the first time, small poker games could be solved exactly, with linear programming.
9. 1:302003At the University of Alberta, a research group took that and aimed it at real poker. In 2003, they published the first approximation of a game-theoretic strategy for full heads-up limit hold’em.
10. 1:42Their trick was abstraction. Real limit hold’em has about ten to the eighteenth game states. They squeezed it into models of about ten million, solved those, and played the result in the real game. It held its own against strong human players.
11. 1:55But linear programming had hit a wall. Memory ran out long before the game did.
12. 2:012007In 2007, four researchers at Alberta, Martin Zinkevich, Michael Johanson, Michael Bowling and Carmelo Piccione, published a paper with a dry title: “Regret Minimization in Games with Incomplete Information.” It introduced counterfactual regret minimization. CFR.
13. 2:18The idea is almost embarrassingly simple. Let the program play against itself. After each hand, at every decision it faced, it asks one question: how much better would I have done if I had chosen differently? That number is the regret.
14. 2:31Next time, it leans toward the actions it regrets not taking. Repeat that millions of times, and something remarkable happens. The average strategy drifts toward an equilibrium. No linear program, no giant matrix. Just two strategies grinding against each other until neither can improve.
15. 2:47And crucially, the memory it needs grows only with the number of decision points. The paper solved poker abstractions with a trillion states. That was a hundred times larger than anything before.
16. 2:572008A year later, in July 2008, Alberta’s program Polaris beat a team of professionals at heads-up limit hold’em in Las Vegas.
17. 3:062009CFR worked. Now it had to get fast. In 2009, Marc Lanctot and colleagues introduced Monte Carlo CFR. Instead of walking the entire game tree on every pass, it samples parts of it. Each pass is cheaper, and big games come within reach.
18. 3:202014In 2014, a programmer named Oskari Tammelin posted a paper on his own, describing a variant he called CFR+. One of its changes: never let regret go negative. It converged faster than earlier methods, often by ten times or more.
19. 3:362015In January 2015, the Alberta group used it to finish the job. Their program Cepheus essentially solved heads-up limit hold’em. The result was published in the journal Science. It took 4,800 processors running for 68 days.
20. 3:50And that was limit hold’em: two players, fixed bet sizes. No-limit is a different animal. Because you can bet any amount, heads-up no-limit has around ten to the hundred and sixtieth decision points. More than the number of atoms in the universe, by a wide margin.
21. 4:04In early 2015, solving no-limit looked like a job for supercomputers and decades.
22. 4:10And then came the thread. The programmer was Piotrek Lopusiewicz, an online poker player from Poland with a background in chess and bridge. Together with Kuba Straszewski, he built a program called PioSolver.
23. 4:22According to the New York Times Magazine, the first version sold for 249 dollars, and he said it was directly inspired by chess engines.
24. 4:30So how do you fit a game with ten to the hundred and sixtieth decision points onto a desktop? You don’t. You stop trying to solve poker, and you solve one hand.
25. 4:39Give the solver both players’ starting ranges. Give it one flop. Give it a short menu of bet sizes. Suddenly the tree is not astronomical any more. PioSolver’s own documentation puts a typical single-raised pot at a little over one gigabyte of memory.
26. 4:52And inside that small tree, there is no abstraction at all. Every hand combination is treated separately. Every blocker counts.
27. 4:58The method was the one the researchers had proven: two strategies playing each other, over and over, until the solver can tell you exactly how exploitable the answer is.
28. 5:06We should be honest about one thing here. PioSolver’s engine has never been published. Its documentation lists an original PioSolver algorithm and, separately, an implementation of CFR. How much the first version borrowed directly from the Alberta papers, only its authors know. What is certain is that it stood on the same idea, and that it arrived two months after Cepheus.
29. 5:25That same spring in Pittsburgh, a research program called Claudico played 80,000 hands of heads-up no-limit against four top professionals. The humans finished ahead.
30. 5:34So in the spring of 2015, the best poker AI in the world could not beat the best humans. And for the price of a tournament buy-in, anyone could own a tool that showed them, spot by spot, how unbeatable poker is played.
31. 5:46What came out of those solves did not look like poker as people knew it. Strong players had bet big with strong hands. The solver often bet small, with everything.
32. 5:54Betting more than the pot had been a rare, wild play. The solver did it routinely, whenever its range was either very strong or very weak.
33. 6:01It cared about blockers, the way the cards in your own hand change what your opponent can hold. And it mixed. The same hand, in the same spot, bet sixty percent of the time and checked forty.
34. 6:10Arguments that had filled forums for decades were over. You could just look up the answer.
35. 6:152017The research did not stop. In 2017, two programs, DeepStack and Libratus, beat professionals at heads-up no-limit. In 2019, Pluribus did it at a six-player table.
36. 6:252020And there was a darker side. If a solver can teach you after the game, it can also whisper to you during it. In 2020, one poker site banned forty accounts for real-time assistance and confiscated more than a million dollars.
37. 6:38The former high-stakes pro Doug Polk once said that solvers kind of killed the soul of the game. Lopusiewicz sees it differently. In his view, solvers did not change poker. They revealed what it had been all along.
38. 6:49Von Neumann proved in 1928 that a perfect strategy exists. It took 87 years, a handful of papers with unreadable titles, and one forum thread before ordinary players could see it.
39. 7:00That was the day no-limit hold’em changed.