Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough

Dwarkesh Patel opens the interview with Dario Amodei, the CEO of Anthropic, by asking why scaling works at all: why the universe is organized so that big blobs of compute thrown at a wide enough distribution of data produce intelligence. Amodei answers candidly that the field still does not know. In his words, it is almost entirely an empirical fact, something you can sense from the data in many places, but without a satisfying explanation.

He offers only a hand-waving sketch, which he flags as such. Physics has ideas about long-tailed or power-law distributions of correlations and effects: most of the data sits in the early fat part of the distribution, before the tails. For language, that means first learning coarse regularities such as parts of speech and nouns following verbs, then ever more subtle correlations. This makes it plausible that each order of magnitude added captures more of the distribution. What is not clear at all is why performance scales so smoothly with parameters and with the amount of data.

Amodei mentions two candidate explanations without endorsing either: a bucket-and-water picture in which parameters are the bucket and data is the water, so the two are proportional, and work by Anthropic's chief scientist Jared Kaplan on fractal manifold dimension. His summary is that there are all kinds of ideas, but nobody knows for sure.

Patel clarifies for listeners that scaling here means the very predictable, smooth decline in next-token loss as one goes from Claude-1 to Claude-2, and asks whether specific abilities can likewise be forecast from the loss. Amodei says they cannot. The statistical average, the loss or entropy, is predictable sometimes to several significant figures, which he notes is unusual outside physics for such a messy empirical field. Specific abilities are very hard to predict. He recalls wondering, while working on GPT-2 and GPT-3, when arithmetic would arrive or when models would learn to code, and says it is sometimes very abrupt. His analogy is weather: its statistical averages can be forecast, but the weather on one particular day cannot.

Asked in plain terms what happens when a model suddenly knows addition, Amodei again says the answer is unknown and that Anthropic is trying to answer it with mechanistic interpretability. One image is circuits snapping into place, yet there is evidence of something gradual underneath: the probability of the right answer climbs from about one in a million to one in 100,000 to one in a thousand long before the model reliably answers correctly. He says he does not understand this at all. He also does not know whether an addition circuit exists in weak form and strengthens, or whether it works poorly from the start.

On whether some abilities will not emerge with scale, Amodei singles out alignment and values. A model trained to predict the world is learning facts, not values, and its job is to predict what comes next. Questions such as what one should do, think or value are free variables, with no bits in the data that settle them. In his words, "I think that's not going to emerge."

Patel asks what explanation Amodei would give if scaling stalled before human-level intelligence. Amodei separates practical problems from problems with the fundamental theory. Practically, data could run out, and he says that looked at naively we are not far from that point, or all available compute could be used up without being enough. He would not bet on either, though both could happen.

Fundamentally he thinks it very unlikely that the scaling laws simply stop. One possible reason would be an inadequate architecture: an LSTM or RNN would give a different slope, because some things are hard to represent without the ability to attend far into the past that transformers have. A wall unrelated to architecture would surprise him, since the things models cannot do no longer seem different in kind from the things they can do. A few years ago one could argue that models cannot reason or program and draw boundaries; that case was more plausible then and is less plausible now.

His own preferred explanation for a hypothetical wall concerns the loss function. To learn to program at a very high level, some tokens matter much more than others, and they are rare. A next-word-prediction loss may over-focus on the parts responsible for the most bits of entropy and neglect what is essential, so the signal gets drowned in noise. He says he does not expect this to happen, but if told that a much bigger 2024 model was no better and every architecture had been tried, this is the explanation he would reach for.

If next-token prediction had to be abandoned, Amodei says one would turn to some kind of reinforcement learning, of which there are many varieties, including RL from human feedback, RL against an objective, Constitutional AI, and amplification and debate, which are both alignment methods and ways of training models. The focus would have to be on what we actually care about the model doing. He adds that we are a little lucky that predicting the next word yields so much else, since there is no guarantee. The cost of switching is speed: with RL you must design how the loss works, whereas next-token prediction is "the easiest thing in the world" and simply there for you.

On data, Amodei predicts it will not be the constraint. He declines to give details for various reasons, but says there are many sources of data in the world and many ways to generate it. When Patel asks whether he means multimodal data, he says only that there are many different ways to do it.

Amodei says his view formed gradually between 2014 and 2017, beginning with his first experience of AI. He saw early work around AlexNet in 2012. He had long wanted to study intelligence, but back to 2005 AI looked far away: he had read Ray Kurzweil and some of Eliezer's early Internet writing, and the AI of the day was nowhere close. AlexNet made him think the field was actually starting to work, so he joined Andrew Ng's group at Baidu, coming from a different field.

He credits luck in the task he and his colleagues were given: build the best speech recognition system possible. Plenty of data and many GPUs were available, and the problem was posed in a way that made it easy to discover scaling as the solution. He contrasts this with a postdoc's incentive to come up with something clever and new to make a mark as an inventor. He simply ran the simplest experiments and fiddled with dials: more layers in the RNN, longer training, how long it takes to overfit, new data repeated fewer times. Very consistent patterns appeared. He did not know this was unusual and calls it almost beginner's luck; he did not think beyond speech recognition at first, and found it odd that the pattern held there.

Just before OpenAI started, he met Ilya Sutskever, who told him that the models just want to learn and that he had to understand this. Amodei likens it to a Zen koan that left him enlightened. It told him his observation was not a random quirk but broad and general. Over the years he formalized and assembled these ideas. His summary is that if you remove obstacles, give the models good data and enough space to operate in, and avoid something stupid like conditioning them badly numerically, they will learn.

Patel notes that many people knew these systems were very good at speech recognition or constrained games, but very few, like Amodei and Ilya, extrapolated to something generally intelligent. He asks what was different in Amodei's thinking: why conclude that consistent improvement at speech implies consistent improvement at everything.

Asked what set his thinking apart from others who saw speech recognition improve, Amodei says he genuinely does not know. At first he assumed the pattern was specific to speech or a narrow class of models. Between 2014 and 2017 he tried it on many things and kept seeing the same result, including with Dota and with robotics. Many people treated robotics as a counterexample, but he reasoned that while data for robotics is hard to get, the data that does exist shows the same patterns.

His explanation of the difference is modest. People were focused on solving the problem in front of them, looking vertically instead of horizontally, so a lack of robotics data easily became the conclusion that scaling does not work for lack of data. He suggests it is hard to explain why one person sees things one way and another a different way, and that his obsession with this direction may simply have been random.

Patel asks when language became obvious as the way to feed in huge amounts of data, rather than a default after other domains ran short of data. Amodei points to next-word prediction as self-supervised learning with rich structure: the next word might require knowing that two plus two equals four, or might continue a story about a character. It poses to the model the equivalent of developmental tests given to children, such as a story where Mary puts an item in a room, Chuck removes it unseen, and the question is what Mary thinks. To predict well, models must solve theory-of-mind and math problems, so the reasoning was to scale as far as possible, with no obvious limit.