Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken

Looking back at the year since their previous conversation, Douglas argues that reinforcement learning for language models has now demonstrated expert-level performance in competitive programming and mathematics when paired with a suitable feedback loop. He distinguishes intellectual complexity from task duration: models can handle difficult problems in focused settings, while sustained autonomous work remains less proven. Douglas predicts that software-engineering agents may soon complete hours of independent work, perhaps close to a junior engineer's day by the following year. Bricken agrees that capability is uneven, noting that boilerplate tasks can already save substantial time. Their prediction is explicitly about likely progress, not a claim that agents are already reliable across all software work.

Douglas revises his earlier emphasis on reliability as the main bottleneck. He now points to limited context, difficulty making broad multi-file changes, and tasks that require discovery and iteration. In his view, performance improves when an agent can receive a clear signal about whether its work succeeded. The guests contrast reinforcement learning from human feedback, which can shape outputs toward preferences but may reward superficial features such as length, with reinforcement learning from verifiable rewards, where correctness can be checked through a math answer or unit tests. They warn that even tests can be gamed if a model discovers their implementation. Programming and math benefit from relatively clear verification; writing, scientific discovery, or aesthetic judgment have less direct reward signals.

The discussion also separates procedural competence from the ability to reason about one's own behavior. Bricken recalls that, about fourteen months earlier, users mostly copy-pasted between chat windows and agents lacked today's ability to fetch context and retain information. He says scaffolding and prompting can reveal stronger behavior than casual use suggests, citing an anecdote about a research group using an AI-assisted workflow to propose and experimentally test a drug candidate. The account is brief and refers listeners to a fuller announcement; it is not presented as proof that models independently discover and validate medicines. Examples such as long-form writing and image-based location guessing similarly depend on carefully designed prompts and human-built workflows.

Patel raises a skeptical interpretation of reasoning models: perhaps their capabilities already exist in pretraining, and additional reasoning only makes successful answers more likely. Douglas responds that the cited result used Llama and Qwen models and may not reflect comparable reinforcement-learning compute. He argues that reinforcement learning can add knowledge, pointing to DeepMind's game-playing systems that exceeded human performance with sufficiently clean rewards. He describes current investment as an optimization between pretraining, experimentation, and inference: teams want confidence in an algorithm before committing large compute budgets, and reinforcement learning can be scaled iteratively. Bricken explains the contrast between dense next-token signals during pretraining and sparse rewards in reinforcement learning, while noting that reinforcement learning is not inherently incapable of teaching new abilities. The unresolved question is how much capability comes from new learning versus eliciting and organizing what pretraining already supplied.