Eric Jang – Building AlphaGo from scratch

Eric Jang's sabbatical project began with a personal question: how could a relatively shallow neural network support decisions that depend on extremely deep Go search? AlphaGo's 2014–2016 breakthroughs had made an intractable search problem workable through deep learning. The later KataGo project offered another signal of how much implementation efficiency could change: it reportedly reduced the compute needed to train a strong Go bot by a factor of 40. Jang argues that LLM-assisted coding now makes a small-scale reconstruction feasible for a few thousand dollars of rented compute, compared with DeepMind's earlier team and multimillion-dollar effort.

The rules themselves expose a gap between human judgment and algorithmic evaluation. Jang describes how Chinese, Japanese, and Tromp-Taylor rules differ, and why computer Go systems favor Tromp-Taylor's unambiguous resolution. Human players can agree that a surrounded group has no realistic chance to live and stop early; the rules then rely on the players' shared judgment. Tromp-Taylor instead lets a program play to a mechanically decidable result. This can produce a scoring result that looks odd to a human, because territory that a player considers lost may still count under the formal rule. The distinction anticipates the later role of a learned value function: humans also make a rapid judgment about whether a position is effectively won.

The combinatorial scale explains why direct search was not enough. On a 19-by-19 board, a move can have roughly 361 choices, and a game can continue for 250 to 300 moves. A naive tree therefore grows on the order of 361 to the 300th power. Some move sequences lead to the same board, so the full tree has redundancies, but merging equivalent states does not make exhaustive search practical. Go is deterministic, which means an exact winning strategy exists in principle; the obstacle is the quantity of futures a computer would have to examine.

Monte Carlo tree search addresses this scale by expanding selected branches rather than building the entire tree in advance. Jang frames nodes as game states and tracks how often a child has been visited. That representation fits a deterministic game: the child state implies the action that produced it. UCB1 helped inspire AlphaGo's selection rule. Its score combines the estimated value of an action with an exploration bonus, so a branch that has not been sampled can receive attention even when its current estimate is weak. The bonus fades as the branch is visited, allowing selection to shift toward actions with stronger evidence.

The UCB1 idea is to balance an action's estimated value Q(a) with a bonus for exploration. Jang describes how this lets the search examine an unvisited branch, even when it currently has no evidence of value, while repeatedly visited branches receive a smaller bonus. He also notes that the tree nodes store visit counts and that an action can be inferred from its child state in deterministic Go. These choices make the search data structure compact and give it a way to trade off trying alternatives against exploiting the strongest current estimate.

The selection process is still only an approximation. Jang points out that Go itself has no randomness; probabilities enter later when a practical algorithm samples from a search distribution. A naive uniform choice among legal moves would spread work across a diffuse tree and spend much of it on low-value paths. The useful insight is to choose which branches to explore rather than enumerate every possibility. Even then, without neural networks, finding the few valuable actions among many poor ones remains computationally expensive.