The future of AI in software development | Inbal Shani (CPO of GitHub)

The conversation is between Lenny and Inbal Shani, Chief Product Officer at GitHub, who is introduced as a former general manager at AWS and at Microsoft and a former senior software developer on Amazon Robotics. Lenny lists the ground the interview will cover: the end state of Copilot and AI tooling for engineers, what is overhyped and underhyped, which metrics show whether Copilot is working, what mistakes teams make when they adopt AI, and Copilot's design philosophy. The same segment contains sponsor messages for Jira Product Discovery (Atlassian) and Sanity, which are advertising and not part of Shani's argument.

Lenny opens with a broad question: he says Shani is at the center of the storm in software development, that GitHub almost kicked it off with the launch of Copilot, and asks what is overhyped and what is underhyped.

Shani answers from the position of someone who says she has just completed her first year at GitHub as CPO. Her starting conviction is that developer happiness and productivity depend strongly on environment and developer tooling, which is why she calls AI critical to the mission. She says AI has become table stakes for developers' expectations and cites the figure that something like 92% of developers already use AI tools.

Her first 'overhyped' claim is that generative AI will replace humans, which she does not expect in the near future. Her reasoning is that people are needed in the loop, that creative thinking is central to humanity, and that a successful AI tool needs data, which requires humans using and acting with tools.

you always need that human in the loop because AI cannot replace innovation.

She also pushes back on the idea that generative AI will be the solution for everything, calling it a tool. Her qualifiers matter: she limits the claim about creativity to 'at least not in the near future', so this is a judgment about the near term, not a claim that replacement is impossible.

What she considers underhyped is AI-driven testing. She says she is starting to hear a little more about it, but not much. Her argument is a chain of consequences: if AI makes teams generate much more code, more lines and more applications, then testing becomes a critical part of the development journey, and she wants to hear how companies use AI to generate a larger test suite to validate the work.

Asked whether testing means unit and integration tests, Shani says it means all of them: load, infrastructure, security and penetration testing, as well as unit, functional and performance testing. Writing all of that by hand takes a lot of people, so she asks whether AI can generate the suite. She adds that teams do a lot of testing they do not even count as testing, so they do not spend much time on it, and that if software is at the center of everything and every company becomes a software company in some form, more code makes testing more critical. She presents this as an observation and a wish to hear more, and she gives no data on AI-generated testing in this part of the interview.

Lenny asks how software development will differ in three to five years and where Copilot-style tools end up as they keep improving.

Copilot is a copilot. It's not a pilot.

Shani repeats her line that the human stays in the loop, and draws a consequence: the person using AI tools to develop software must think differently. First, more about systems and architecture, and about how to use the tools to be successful. The work shifts from only writing code to the big picture, the connected experience and connected systems.

She ties this to the junior-senior gap. By her account, system-level thinking is found today more among senior developers and less among juniors, because juniors are expected to write simple code while they are still learning to code. If an AI assistant helps them write code, she argues, they can spend time from the start understanding the system, the environment and the product they are building. This is her expectation of what will happen over time ('in the fullness of time'); she offers no evidence that it is happening yet.

Her second expected change is in hardware development. She says AI has a big footprint and needs many resources, so meeting demand requires improving hardware (CPUs, GPUs), compute and code optimization. She describes this today as a niche area that she expects to become more aligned across the software development lifecycle.