John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI

Schulman describes pretraining as learning to imitate web content through next-token prediction, which produces a model able to continue many kinds of text and assign probabilities across them. Post-training narrows that broad base toward an assistant persona, optimizing for responses people find useful rather than raw imitation. The distinction is not only technical: pretraining creates a broad, flexible model, while post-training shapes how it responds to requests.

Asked what capabilities might arrive over the next several years, Schulman expects models to take on more involved projects, such as writing and testing a multi-file software project from high-level instructions. He attributes this partly to training on longer tasks rather than single steps, using reinforcement learning or other supervision, and to better recovery from mistakes and edge cases. More sample-efficient learning could let a model recover from a small amount of experience or transfer from other abilities, whereas current systems may get stuck. He cautions that long-horizon capability may require more training compute, and that progress could be uneven or marked by phase transitions.

The speakers ask whether longer-term coherence alone could make models human-level. Schulman says he does not expect one training change to unlock every capability; models may still think too shallowly, miss instructions, or lack other forms of judgment. He also points to practical constraints such as access to interfaces and the physical world. On websites, he expects improved vision to let models use human-designed interfaces, though some services may benefit from clearer text representations and indications of which elements are interactive. He does not expect every website to become an API.

Schulman gives examples of transfer in post-training. An assistant fine-tuned on English can often respond appropriately in Spanish, and text-only fine-tuning can improve behavior with images. Early ChatGPT versions sometimes claimed to have sent emails or ordered rides when they could not; Schulman says roughly 30 examples clarifying these limitations generalized to many capabilities that were not specifically represented in the training data. These anecdotes suggest transfer, though they do not quantify its limits.