Reiner Pope – Chip design from the bottom up

This episode is a blackboard lecture rather than an interview in the usual sense. The promise is a full stack in one sitting: begin at AND, OR and NOT gates, and climb until it is clear why GPUs, TPUs, FPGAs and the human brain each ended up with the shapes they have. The visual channel matters here — Dwarkesh tells listeners to watch on YouTube precisely because the argument is carried by diagrams being drawn and crossed out.

The guest is Reiner Pope, CEO of MatX, a new AI chip company, and previously at Google working on software efficiency, compilers and TPU architecture. Dwarkesh discloses that he is an angel investor in MatX, which is worth holding in mind: the conversation is technical throughout, but the person explaining which design choices are hard is also the person making them.

Pope starts at the smallest unit of chip design — logic gates wired together by metal traces — and states the destination: a production chip and its components. The bridge between the two is the operation AI chips exist to perform, matrix multiplication, whose inner primitive is a multiply-accumulate over pairs of numbers. The plan is to compute one by hand, then read the circuit off the arithmetic.

The worked example is a four-bit number times a four-bit number, accumulated into an eight-bit running total. Pope gives two reasons the primitive is natural. The first is that a matrix multiply is a triple for-loop over i, j and k, and a multiply-accumulate happens at every step of it. The second is specific to AI chips: precision is almost always higher in the accumulation than in the multiplication, because products of low-precision numbers are fine but the errors from summing many of them pile up fast.

Dwarkesh pushes on whether this is really one principle or two — magnitude versus rounding — and the answer separates them cleanly. The value growing larger is one thing; the accumulation of rounding error is another. The apparent objection, that a single multiply-accumulate only sums two numbers, is resolved by the loop: the summation repeats j times, so errors compound across iterations even though each individual multiply contributes only one rounding step.

The hand calculation is long multiplication, kept in one pass rather than split into two. The four-bit multiplicand is multiplied by each bit position of the multiplier in turn: by 1 it reproduces itself, by 0 it vanishes, shifted once it reappears, shifted again it vanishes. Those partial products plus the accumulator term become a five-way sum, which is the object the circuit has to compute.

The first cost accounting is exact and small. Producing each partial-product bit is an AND of two input bits, so a four-by-four multiply consumes sixteen AND gates, and in general a p-bit by q-bit multiply consumes p times q of them. This is the first appearance of the quadratic relationship that dominates the rest of the lecture.

Most of the work, Pope notes, is in the summing, which introduces the other end of the gate-size spectrum: the full adder. Despite the name it does not add thirty-two-bit numbers; it adds three single bits and returns two, since 0+1+1 can be anything from 0 to 3 and two bits suffice. Hence the alternative name, a 3-to-2 compressor.

“It’s essentially just counting the number of things and expressing that in binary. This circuit captures what we as humans naturally do when we’re summing along a column.”

A clarifying exchange pins down the semantics, which are easy to misread from audio alone. The three inputs are three bits in the same column, not an X, a Y and an incoming carry; the two outputs are drawn vertically and horizontally to encode that same-column bits share a position while adjacent columns do not. One output is the sum bit, the other the carry out.

The reduction proceeds right to left, and the one deliberate departure from human habit is that carries are written down explicitly instead of remembered. Three bits in a column go into a full adder, two come out, and consumed bits are crossed out. Repeat across the grid until a single number remains — the standard area-efficient scheme, which Pope names as the Dadda multiplier.