Sholto Douglas, Trenton Bricken: AMA: career advice given AGI, how I research ft. Sholto & Trenton
Guests: Sholto Douglas, Trenton Bricken
In-depth read · Key ideas, reasoning, and examples, organized by topic.
§01A Book Built Out of Adjacency
Dwarkesh opens what is nominally an Ask Me Anything by spending the first stretch of the episode selling a book, and the way he sells it is itself a claim about what the podcast has been doing for five years. The Scaling Era, published with Stripe Press, is described as a compilation and curation of the sharpest fragments of his interviews: Dario Amodei on why scaling works, Demis Hassabis on whether DeepMind goes the reinforcement-learning route and how much of AlphaZero survives into the next generation of language models, Bricken and Douglas on the mechanics. The unit of value being claimed is not the interview but adjacency. The reader turns a page and two incompatible vocabularies are suddenly in contact, and the contact is the product.
you can read Dario addressing, why does scaling work?
The stronger version of the argument is disciplinary. Dwarkesh calls AI the most multi-disciplinary field one can imagine, and he means it structurally rather than as a compliment to his own guest list: any account of a society containing billions of additional workers, or of an intelligence exceeding the rest of humanity combined, will draw on economics, evolutionary biology, philosophy of mind and institutional history at once. He offers Carl Shulman on scaling regularities in primate brains from chimpanzees to humans, and Tyler Cowen on why explosive growth might be eaten by bottlenecks, as evidence that the relevant literature is scattered across fields that do not cite each other. The book is an attempt to force the citation.
no field, no domain of human knowledge that is not relevant to understanding what a future society of different kinds of beings will look like
Trenton Bricken supplies the question that any editor would want asked, and labels it honestly as a plant: the flyball question of why ordinary people, specifically his mother, should buy the book. The answer Dwarkesh gives is a compression argument rather than a novelty argument. Nothing in the book is a discovery; the value is that material which previously required forty hours of audio and a tolerance for jargon now fits in a weekend. Two of the interviews included have never been released publicly, which matters for the credibility of the project: Jared Kaplan explaining scaling from the mathematics of data manifolds, and a second guest on why general intelligence evolved at all. Those two chapters are the evidence that this is not simply a reprint.
The most defensible justification, though, comes out of an admission. After the Bricken-Douglas episode aired, readers independently produced blog posts and flashcard decks reconstructing concepts the conversation had skipped over. That is a documented failure mode of the technical interview format: the guest and host share enough context that entire load-bearing definitions never get said out loud. The book's side captions, its introduction and its diagrams are a patch for that failure, applied retroactively to the whole archive. Bricken singles out exactly those features as what made the volume usable, which is a more interesting compliment than praising the prose.
The closing joke of the segment is that their parents will finally understand what they do for a living, and it is worth taking seriously as a design constraint. Dwarkesh's answer to the accessibility question is not that the material has been simplified but that the missing context has been supplied at the margin, where a curious non-specialist can take it or leave it. That is a different editorial bet from popularisation, and it is the one the rest of the episode keeps testing.
§02Everything Memorised, Nothing Connected
The first real question, from Brian Krav, returns to something Dwarkesh pressed Dario Amodei on and has tweeted about since: models hold all of human knowledge in memory, yet they do not make the cross-domain connections that a moderately intelligent human with the same memory would make. The framing is deliberately quantitative. This is not a complaint that models are uncreative; it is a claim that a specific, measurable capability that humans demonstrably possess is absent at a rate of zero.
The existence proof he offers is Don Swanson noticing that the brain changes produced by magnesium deficiency have the same structure as those seen during a migraine, and inferring that magnesium supplementation might treat migraines. It worked. Dwarkesh's point is not that this was easy but that it was the kind of move that requires nothing except two pieces of memorised knowledge and the habit of putting them side by side. He then states the asymmetry that the rest of the chapter has to explain.
I don’t know of a single example of LLMs ever having done it
Scott Alexander's reply, which Dwarkesh raises against himself, is that humans are not logically omniscient either: the combinatorial space of possible connections is so large that we ignore almost all of it, which is why nobody spends the afternoon wondering why two words share an etymology. That explains why humans do not connect everything. It does not explain why the observed rate for models is zero rather than merely low, and Dwarkesh says as much before handing the question over. The distinction matters because the two hypotheses imply different remedies: a rate that is low for attentional reasons yields to better retrieval and longer context, while a rate that is zero for structural reasons requires something else entirely.
Sholto Douglas's answer locates the gap in the training objective. Pretraining installs flexible general knowledge about the world, but the skill of making a novel connection is a research skill, and research skill is acquired the way doctoral students acquire it: through years of exploring a narrow problem, being wrong in public, and interacting with material that pushes back. On that view the missing ingredient is not data but a training signal that rewards the behaviour of a researcher, and he is explicit that the field has not yet run that experiment at scale. What he wants to see is early evidence from models that interact with the world and attempt real discoveries, evaluated against the behaviour we expect from people in those roles.
Trenton Bricken adds two mechanisms that are more specific and, in the near term, more actionable. The first is memory policy. A human told something new immediately reasons about what form the memory should take, because she is aware of her own forgetting; a model has no such awareness and no opportunity to choose its own representation. He points to Claude Plays Pokemon as a natural experiment: the model got far, and then a different employee reworked the memory scaffold and improved it quickly, which suggests the bottleneck was the scaffold rather than the policy.
The second is the idiot savant analogy, and Kim Peek is the right comparison because the deficit and the gift sit in the same person. Peek, born without a corpus callosum by Bricken's account, read two pages at once with independent hemispheres and retained everything, while functioning poorly in domains that require integration. The analogy predicts precisely the profile observed in language models: extraordinary performance on narrow, well-specified topics and collapse on tasks that require stitching two of them together.
Keep reading as a member.
You've read the free preview. Become a member to read the rest, with side-by-side reading and downloads where available.