Jensen Huang: NVIDIA – The $4 Trillion Company & the AI Revolution

Lex Fridman opens by calling NVIDIA one of the most important and influential companies in the history of human civilization and "the engine powering the AI revolution." He credits much of its success to Jensen Huang's will and to his bets and decisions as a leader, engineer and innovator. This is the host's framing. The excerpt offers no evidence for it beyond the discussion that follows.

Fridman says NVIDIA has moved from chip-scale design to rack-scale design. For a long time, winning meant building the best GPU possible, and NVIDIA still does that. He argues it has since expanded to "extreme co-design" across the GPU, CPU, memory, networking, storage, power, cooling, software, the rack, the announced pod and the data center. His first question is what the hardest part of co-designing a system with that many components and design variables is.

Huang starts with the reason for co-design rather than its hardest part. He says the workloads have outgrown a single machine:

the reason why extreme co-design is necessary is because the problem no longer fits inside one computer to be accelerated by one GPU.

He states the goal as speeding up faster than the number of computers added. His example is adding 10,000 computers while wanting the work to go a million times faster. To get there, he says, the algorithm must be broken up and refactored, and the pipeline, the data and the model must all be sharded. He distinguishes distributing a problem from simply scaling it up. Once the problem is distributed, "everything gets in the way."

He ties this to Amdahl's law, under which the speedup from improving one part of a workload depends on how much of the total workload that part represents. He gives a numerical illustration:

if computation represents 50% of the problem, and I sped up computation infinitely like a million times, you know, I only sped up the total workload by a factor of two.

So faster compute alone is not enough. Huang lists the CPU, the GPU, networking, switching and distributing the workload across the computers as problems in their own right at NVIDIA's scale. He calls the result "a massively complex computer science problem" that requires bringing every technology to bear. Otherwise, he says, scaling would be linear or limited by Moore's Law. He describes Moore's Law as having slowed:

Moore’s Law, which has largely slowed because Dennard scaling has slowed.

Huang later sets out three questions: what extreme co-design is, why it is needed, and how it is done. The "what" has two layers. The first is optimization across the computing stack, from architectures to chips, systems, system software, algorithms and applications. The second goes beyond CPUs, GPUs, networking chips, scale-up switches and scale-out switches to include power and cooling. He adds a qualification: the computers are "very energy efficient" yet "in aggregate still consume a lot of power." He gives no power figures. For the "why," he repeats that distributing the workload is meant to beat the gain from simply adding computers.

Fridman presses on the human side. Each area has its own world experts, including high-bandwidth memory, networking and NVLink, NICs, optics and copper, power delivery and cooling. He asks how those specialists are brought together to design one rack, and how specialists and generalists divide the work. Huang's short first answer is "That’s why my staff is so large."

Huang treats the "how" as an organizational question and calls it "kind of the miracle of this company." He compares the company to a computer that needs an operating system. A company should first decide what it wants to produce, then build itself as the machinery for that output. He criticizes organization charts that look alike across very different businesses:

Hamburger organization charts, soft organization charts, and car company organization charts. They all look the same. And it doesn’t make any sense to me.

He argues that "the architecture of the company should reflect the environment by which it exists," and says that environment "almost directly says what you should do with the organization."

His concrete example is his own staff. He says his direct staff is 60 people. When Fridman checks whether he still has 60 reports, Huang answers "More, yeah." Huang says he does not hold one-on-ones with them because that is impossible at that size. Fridman suggests most of them have at least a foot in engineering. Huang replies "Almost all of them." He names experts in memory, CPUs, optics, GPUs, architecture, algorithms and design.

Huang links the meeting format directly to the engineering method:

And no conversation is ever one person. That’s why I don’t do one-on-ones. We present a problem and all of us attack it.

He says "the company is doing extreme co-design all the time." Fridman describes a discussion about cooling or networking where everyone listens and can object that a choice doesn't work for power distribution or memory. Huang agrees with each point.

Huang also says attendance is not policed: "whoever wants to tune out, tune out." He trusts staff to know when to pay attention. If someone could have contributed and did not, he says he will call them out. The excerpt gives no outside evidence on how well this structure works. Huang's account is the only source.

Fridman asks when the environment changed, from gaming GPUs through the early deep learning revolution to thinking of NVIDIA as a company that builds "a factory that makes AI." Huang does not give a date. Instead he explains the strategy, starting with NVIDIA's origins as an accelerator company.

He says accelerators have a trade-off. Specialization makes them highly optimized for their job, but the application domain is narrow. He says a narrower market reach alone is acceptable. The real constraint is that market size limits R&D capacity:

the market size also dictates your R&D capacity. And your R&D capacity ultimately dictates the influence and impact that you can possibly have in computing.

Huang says NVIDIA always knew the specific accelerator was only the first step, and that it needed to become "accelerated computing." He notes that the two words pull against each other. A general-purpose computing company loses its specialization: "The better computing company we become, the worse we became as a specialist." He says the company had to follow a "really narrow path," widening its computing range step by step without giving up its most important specialization.

Huang lists the steps in order: (1) inventing a programmable pixel shader, the first step toward programmability; (2) putting IEEE-compatible FP32 into the shaders, which he calls "a huge step in the direction of computing"; (3) putting C on top of FP32, which became Cg; (4) the Cg path leading eventually to CUDA; (5) putting CUDA on GeForce.

Huang credits the FP32 step with drawing in outside researchers. People working on stream processors and other data flow processors "discovered us." They saw a computationally intensive GPU that was now IEEE-compliant and could run software they had previously written for CPUs.

Huang calls putting CUDA on GeForce a strategic decision that was "very, very hard to do":

it cost the company enormous amounts of our profits, and we couldn’t afford it at the time. But we did it anyway because we wanted to be a computing company.

His reasoning is a chain of requirements. A computing company has a computing architecture, and "A computing architecture has to be compatible across all of the chips that we build." He does not put a number on the profit hit. When Fridman asks him to explain the decision, Huang calls it "the first strategic decision that is as close to an existential threat."

Fridman adds his own hindsight verdict. He calls it "one of the most incredibly brilliant decisions ever made by a company" and says CUDA became a foundation for computation in AI infrastructure. Huang's agreement is more measured: "it turned out to have been a good decision."

Huang then explains why compatibility across consumer GPUs mattered. After CUDA widened the range of applications NVIDIA could accelerate, the question was how to attract developers. He argues that developers do not come to a platform because it can do something interesting. They come because the install base is large and they want their software to reach many people. His conclusion:

the install base is, in fact, the single most important part of an architecture.

He supports this with a historical comparison. He says no architecture has drawn more criticism than x86, which he calls "less than elegant," yet it is "the defining architecture of today." By contrast, he says, many RISC architectures failed despite being well designed:

so many RISC architectures which were beautifully architected, incredibly well-designed by some of the brightest computer scientists in the world, largely failed.

Huang explicitly connects installed base to the decision to put CUDA on GeForce: millions of GPU sales could seed a computing architecture while NVIDIA cultivated developers. He then describes the cost and profit trade-off; this is his historical account, not independent market data.

Install base defines an architecture. Not… Everything else is secondary, okay?

Jensen Huang presents CUDA's success as the result of a strategic bet on distribution. He says CUDA came out into a crowded field. OpenCL existed, and there were "several other competing architectures." What he says NVIDIA got right was not a technical feature. It was the realisation that a new computing architecture succeeds through its install base. By then GeForce was already a commercial success, and he treats that success as the vehicle for getting CUDA into the world.

The plan he describes had two parts. First, NVIDIA would put CUDA on every GeForce GPU, of which it was already selling "millions and millions" a year, and ship it into every PC "whether customers use it or not." Second, the company would court developers directly: it went to universities, wrote books, taught classes and "put CUDA everywhere." Huang ties the logic to the computing landscape of the time. The PC was the main computing vehicle and "There was no cloud," so a GPU in a PC could put "a supercomputer in the hands of every researcher" and every student. He frames the payoff as open-ended: "eventually something amazing will happen."

He also explains why consumer hardware reached scientists. Many researchers were gamers, many built their own PCs, and in university labs many built clusters themselves out of PC components. Lex Fridman adds that this base became "the platform and the foundation for the deep learning revolution," and Huang agrees, calling that "another great, great observation." The link to deep learning comes from Fridman's prompt and Huang's agreement. Huang does not lay out how it happened in this passage.

I always say that NVIDIA is the house that GeForce built, because it was GeForce that took CUDA out to everybody.

Huang is frank that the install-base strategy was very expensive. He says CUDA raised the cost of a consumer GPU so much that "it completely consumed all of the company's gross profit dollars." His valuation figures are openly approximate. He puts the company's worth before the launch at around $8 billion, then corrects himself to "six, $7 billion or something like that." He says the market cap later fell to "like one and a half billion dollars," and that the company stayed there for a while before it "clawed our way back slowly."

Asked by Fridman what the internal discussions were like, Huang gives more specific numbers. NVIDIA "increased our cost by 50%" while being "a 35% gross margin company." He says he had to make the goal clear to the board, and the management team knew gross margins "were gonna get crushed." The core tension was that gamers would not value or pay for CUDA: "they only pay certain price and it doesn't matter what your cost is."

The justification he offers was a longer-term hope, not a near-term return. CUDA might eventually reach workstations and supercomputers, where "maybe we can capture more margin." He calls it "quite a difficult decision" and admits the timeline was long. You could reason your way into affording it, "but it still took… It took a decade." The account is Huang's own recollection. Apart from his hedged estimates, the passage offers no outside figures to check it against.

Fridman shifts from the board conversation to the psychology of NVIDIA's "bold bets," asking how Huang makes leaps like these as a company. Huang starts with curiosity. Then he describes a point where "a reasoning system" convinces him an outcome "will happen," after which he believes it and, in his words, manifests that future. He does not downplay the cost: "There's a lot of suffering in between, but you've gotta believe what you believe." He adds that the reasoning is shared, saying the management team also reasons about why a future "must exist."

He contrasts his approach with a style of leadership he rejects. In that style, leaders stay quiet, then issue a manifesto at the start of a new year with a new plan, a "Big huge layoff," a large organisational change, a new mission statement and "brand new logos." He says, "I never do things that way." Instead, when an idea starts to influence his thinking, he tells the people near him that it is interesting and will make a difference, and he reasons through it "step by step by step."

He openly admits that this is often persuasion after the fact: "Oftentimes, I've already made up my mind." He then uses external information, new insights, engineering revelations and milestones to "shape everybody else's belief system." He says he does this "literally every single day" with his board, his management team and his employees. He gives the Mellanox acquisition and going "all in on deep learning" as examples. In each case, he says, the groundwork had already been laid across different organisations, so by announcement day most people had bought into many of the pieces.

Sometimes it looks like you're leading from behind, but you've been shaping their, you know, to the point where on the day that I declared it, 100% buy-in.

He describes the alternative as announcing a sudden pivot to management, the board, employees and customers who respond "Where's this coming from?" He extends the method to NVIDIA's GTC keynotes. There, he says, he shapes the beliefs of industry partners as well as his own employees. His recent example, as it appears in the transcript, is "we just announced Grok." He says he had been "talking about the stepping stones for two and a half years" before that announcement. The evidence for the method is anecdotal. The "100% buy-in" figure and the imagined employee reaction, "what took you so long?", are Huang's own descriptions, not measured outcomes.

Fridman suggests that this belief-shaping reaches beyond NVIDIA to the "broader global landscape of innovation." Huang answers with a structural reason why persuasion matters for his business. NVIDIA, he says, does not build computers or clouds. It is "a computing platform company," and "nobody can buy anything from us. That's the weird thing." The company designs and optimises the stack in a vertically integrated way. It then opens "the entire platform at every single layer" to be built into other companies' products, clouds, supercomputers and OEM computers.