Artificial Intelligence and Poker
Poker is an imperfect-information game in which good play requires randomized strategies, reasoning about ranges of hidden hands, and protection against exploitation. The goal is not to guess one hidden state and then run minimax.
Why ordinary minimax is insufficient
Players do not observe one another's private cards. Different underlying deals can therefore be indistinguishable to the acting player. These states form an information set. A legal strategy must choose consistently from the information available; it cannot act differently based on a hidden card it has not seen.
Strong poker also requires mixed strategies. If an agent always bluffs or always checks in a particular situation, an opponent may exploit that predictability.
Core concepts
| Concept | Meaning |
|---|---|
| Range | A probability distribution over possible private hands |
| Information set | States that are indistinguishable to a player |
| Mixed strategy | A probability distribution over actions |
| Nash equilibrium | A strategy profile in which no player benefits by changing strategy alone |
| Exploitability | How much a best-responding opponent can gain over the equilibrium value |
| Regret | The loss from not having chosen a better action in retrospect |
Counterfactual regret minimization
Counterfactual regret minimization (CFR) repeatedly estimates how much better each action would have performed at each information set. Positive regrets increase the probability of selecting the corresponding actions in later iterations. Under appropriate conditions, average strategies approach an equilibrium.
Abstraction and continual re-solving
No-limit hold'em game trees are too large to enumerate directly. Poker programs therefore combine:
- abstraction to group strategically similar situations or reduce the betting choices;
- offline computation to construct a broad strategy;
- real-time re-solving to refine the strategy for the current subgame;
- randomized action selection to avoid predictable play.
Milestones
| System | Contribution |
|---|---|
| DeepStack (2017) | Combined continual re-solving with learned estimates of future values in heads-up no-limit hold'em |
| Libratus (2017/2018) | Defeated leading human specialists in heads-up no-limit hold'em using equilibrium-oriented computation and subgame refinement |
| Pluribus (2019) | Defeated elite professionals in six-player no-limit hold'em, extending success beyond two-player zero-sum play |
| Student of Games (2023) | Unified search, learning, and game-theoretic reasoning across perfect- and imperfect-information games |
Primary reports: Libratus, Pluribus, and Student of Games.
Equilibrium versus exploitation
An equilibrium-oriented strategy protects the agent against strong adversaries. Against a predictably weak opponent, an agent may earn more by deviating to exploit that weakness—but an incorrect opponent model can make exploitation dangerous. This creates a general agent-design tradeoff between robustness and adaptation.