Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI compute

The conversation opens with a mismatch between headline spending and immediately usable compute. The four largest technology companies are projected to spend $600 billion in capital expenditure, a sum that would correspond to nearly 50 gigawatts if treated as a year of rented compute. Yet only about 20 gigawatts of new US capacity is expected to come online this year. Much of the spending buys lead time: turbine deposits, future data-center construction, power-purchase agreements and other commitments made well before capacity is ready. The supply chain total could approach a trillion dollars once these upstream investments are included.

That distinction matters for the AI labs, whose growing revenue creates an immediate need for inference capacity. Patel estimates that Anthropic could require roughly four additional gigawatts simply to serve projected revenue growth, even if its research and training fleet stopped growing. Dwarkesh frames the resulting question as a timing problem: if the capacity is needed before new facilities are built, where can it come from? Patel's answer is partly indirect supply through cloud partners and partly a willingness to accept providers or contract terms the lab would previously have avoided.

The guests contrast Anthropic's cautious contracting with OpenAI's aggressive commitments. Patel argues that OpenAI's broad set of suppliers gives it more access to compute by year-end, while Anthropic's conservative approach leaves it looking for newer or less established providers. The episode also recalls the earlier market fear that OpenAI might not be able to pay for the contracts it signed. Once the lab raised substantial capital, counterparties became more willing to contract with it. The point is not that risk disappeared, but that financing credibility changes who can secure scarce capacity and on what terms.

Scarcity makes timing valuable. Shorter-term Hopper capacity can be sold at high hourly prices as older contracts expire, and a buyer with cash may crowd out other customers. Cloud providers can also serve models through products such as Bedrock, Vertex or Foundry, sharing revenue instead of transferring all the compute directly. Dwarkesh describes the resulting premium as either a markup on spot capacity or a revenue share. In either case, a company that reserved compute earlier may have a cost advantage even if its model lead lasts only a few months.

The discussion then turns to GPU depreciation. A conventional total-cost-of-ownership model includes chips, servers, data centers, networking, staff, spare parts and financing. Under one five-year model, an H100 costs about $1.40 per hour to deploy; a $2 hourly contract yields roughly 35% gross margin. A shorter accounting life would raise annual depreciation and make cloud investment look less attractive. But Patel's counterpoint is that a GPU's economic usefulness depends on the value of the work it can still perform, not only on how quickly a newer chip arrives.

There are two competing price forces. New chips deliver much more performance, which can push the market value of older GPUs down; Patel sketches Hopper falling from about $2 per hour in 2024 to $1 in 2026 and $0.70 as Rubin scales. Yet a newer model can also extract more useful output from the same older GPU. GPT-5.4 is described as cheaper to serve than GPT-4 while producing more tokens per H100, and the market opportunity attached to those tokens may be much larger. Under capacity constraints, the chip's value is therefore tied to the best valuable workload it can run, not just its position in a hardware generation chart.

The immediate strategic implication is that a temporary lead in model quality may still be valuable if it lets a lab sign contracts, reserve capacity and secure better prices before competitors arrive. The guests connect this to the depreciation debate: if older GPUs remain useful for longer than pessimistic accounting assumptions imply, committing to them early can be a rational way to lock in supply. Their conclusion is conditional on continued demand and on the GPU retaining useful workloads; it does not establish a single correct depreciation period for every chip or cloud contract.

Patel offers a second way to think about GPU value: the utility a chip can deliver now. If the newest Rubin chips were available without limit, older Hopper GPUs would likely fall in price as newer hardware improved performance per dollar. But semiconductor supply and deployment schedules are constrained, so the price is shaped less by the substitute a buyer could theoretically purchase and more by the value the existing chip can produce today.