Pieter Abbeel: Deep Reinforcement Learning

Lex Fridman introduces Pieter Abbeel (transcribed as Peter) as a UC Berkeley professor and director of the Berkeley Robotics Learning Lab, among the top researchers on making robots understand and interact with the world through imitation and deep reinforcement learning. The conversation is part of the MIT course on artificial general intelligence.

Asked when a robot could beat Roger Federer at tennis, Abbeel says that here, unlike many AI challenges where software is the missing piece, hardware is nowhere near ready either: Boston Dynamics robots are starting to run but not at human level, and then they would have to swing a racket. Both hardware and software are needed and will progress independently; he guesses hardware in 10 to 15 years, sooner if the robot can be on wheels rather than a full humanoid.

Swinging a racket with a fixed industrial arm watching a ball machine seems learnable to him, with reinforcement learning requiring a lot of trial and error and perhaps some pre-training in simulation. What would be interesting is how much precision it could reach, since human players hit the lines, and whether RL could learn to put spin on the ball.

The most impressive physical feat Abbeel has seen are the Boston Dynamics videos, such as the robot running up stairs and doing parkour; even if hard-coded underneath, the physical ability is impressive. He met Spot Mini at the Mars event Jeff Bezos organizes, where it followed Bezos around, and notes that Fridman also felt an immediate psychological connection to Spot Mini despite knowing how it works.