Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)

The conversation opens by challenging a familiar habit in AI work: treating proximity to the latest model or framework news as a proxy for progress. Huyen argues that user conversations and feedback can improve an application far more directly. Her point is not that technical choices never matter, but that teams should first ask how much difference an apparently optimal choice will actually make and how costly it will be to reverse. This frames AI engineering as a product and operations discipline before it is a race to adopt new technology.

Rachitsky pairs that argument with a practical tension: many companies are building AI products, yet many are struggling to make them work. Huyen’s example of managers preferring an additional hire while executives may prefer AI assistance shows why broad productivity claims are hard to interpret. The two roles face different constraints and care about different metrics, so a tool’s value cannot be established simply by announcing that it is available.

The technical discussion separates pre-training from post-training without treating either as magic. Huyen describes supervised fine-tuning as training on demonstrations of desirable answers. She also notes that distillation can teach a smaller model to emulate the output of an already capable model, while cautioning that this differs from creating a strong model from scratch. She then flags reinforcement learning as another major part of the post-training picture, setting up a later discussion rather than claiming a complete account here.

For an intuitive account of pre-training, the speakers describe a model as code with learned weights that predict likely next tokens. Huyen makes the distinction between characters, words, and tokens concrete with the example of splitting a novel word such as podcasting into meaningful pieces. This statistical view also makes generation choices visible: a model represents possible continuations, while the sampling strategy determines whether it repeatedly selects the most likely continuation or permits more variety. She presents sampling as an important, often underappreciated performance lever.