Dmitri Dolgov: Waymo and the Future of Self-Driving Cars
Guests: Dmitri Dolgov
Full transcript · Read the full conversation in order, with its original context.
Contents17 chapters
§01Transcript · 00:00–08:00§02Transcript · 08:01–16:27§03Transcript · 16:29–24:03§04Transcript · 24:04–33:30§05Transcript · 33:33–42:27§06Transcript · 42:31–51:44§07Transcript · 51:48–59:49§08Transcript · 59:53–1:08:28§09Transcript · 1:08:29–1:18:24§10Transcript · 1:18:28–1:27:55§11Transcript · 1:27:57–1:36:48§12Transcript · 1:36:50–1:45:48§13Transcript · 1:45:50–1:53:58§14Transcript · 1:54:00–2:02:41§15Transcript · 2:02:45–2:10:40§16Transcript · 2:10:44–2:19:33§17Transcript · 2:19:35–2:23:03§01Transcript · 00:00–08:00
The following is a conversation with Dmitry Dolgov,
the CTO of Waymo,
which is an autonomous driving company
that started as Google's self-driving car project in 2009
and became Waymo in 2016.
Dmitry was there all along.
Waymo is currently leading
in the fully autonomous vehicle space
in that they actually have an at-scale deployment
of publicly accessible autonomous vehicles
driving passengers around with no safety driver,
with nobody in the driver's seat.
This, to me, is an incredible accomplishment of engineering
on one of the most difficult
and exciting artificial intelligence challenges
of the 21st century.
Quick mention of a sponsor
followed by some thoughts related to the episode.
Thank you to Trial Labs,
a company that helps businesses apply machine learning
to solve real-world problems.
Blinkist, an app I use for reading
through summaries of books.
BetterHelp, online therapy
with a licensed professional.
And Cash App, the app I use to send money to friends.
Please check out the sponsors in the description
to get a discount at the support of this podcast.
As a side note, let me say that autonomous
and semi-autonomous driving was the focus
of my work at MIT and is a problem space
that I find fascinating and full of open questions
from both a robotics and a human psychology perspective.
There's quite a bit that I could say here
about my experiences in academia on this topic
that revealed to me, let's say,
the less admirable sides of human beings.
But I choose to focus on the positive, on solutions,
on brilliant engineers like Dmitry and the team at Waymo
who work tirelessly to innovate
and to build amazing technology that will define our future.
Because of Dmitry and others like him,
I'm excited for this future.
And who knows, perhaps I too will help contribute
something of value to it.
If you enjoy this thing, subscribe on YouTube,
review it with Five Stars and Up, a podcast,
follow on Spotify, support on Patreon,
or connect with me on Twitter @LexFriedman.
And now, here's my conversation with Dmitry Dolgov.
When did you first fall in love with robotics
or even computer science more in general?
- Computer science first, at a fairly young age.
Then robotics happened much later.
I think my first interesting introduction to computers
was in the late '80s when we got our first computer.
I think it was an IBM, I think IBM AT.
Remember those things that had like a turbo button
in the front? - Turbo button, yeah.
- That you would press it and make the thing
go faster.
- Did that already have floppy disks?
- Yeah, yeah, yeah, yeah.
Like the 5.4 inch ones.
- I think there was a bigger inch.
So go on something, then five inches and three inches.
- Yeah, I think that was the five.
Maybe that was before that was the giant plates
and I didn't get that.
But it was definitely not the three inch ones.
Anyway, so we got that computer.
I spent the first few months just playing video games,
as you would expect.
I got bored of that.
So I started messing around and trying to figure out
how to make the thing do other stuff.
Got into exploring programming.
And a couple of years later, it got to a point
where I actually wrote a game, a little game.
And a game developer, a Japanese game developer
actually offered to buy it for me for a few hundred bucks.
But for a kid in Russia.
- That's a big deal.
- That's a big deal, yeah.
I did not take the deal.
- Wow, integrity.
- Yeah, I instead--
- Stupidity.
- Yes, that was not the most acute financial move
that I made in my life.
Looking back at it now, I instead put it,
well, I had a reason.
I put it online.
It was, what'd you call it back in the day?
So it was a freeware thing, right?
It was not open source, but you could upload the binaries,
you would put the game online.
And the idea was that people like it
and then they contribute
and they send you little donations, right?
So I did my quick math of like,
of course, thousands and millions of people
are gonna play my game,
send me a couple of bucks a piece,
should definitely do that.
As I said, not the best financial decision of my life.
- You're already playing with business models
at that young age.
Remember what language it was?
What programming it was?
- Pascal.
- Which, what?
- Pascal.
- Pascal.
And it had a graphical component, so it's not text-based?
- Yeah, it was like, I think 320 by 200,
whatever it was, I think that kind of the earlier version.
- That's the resolution.
- VGA resolution, right?
And I actually think the reason why this company
wanted to buy it is not like the fancy graphics
or the implementation,
it was maybe the idea of my actual game.
- The idea of the game, okay.
Well, one of the things, it's so funny,
I used to play this game called Golden Axe
and the simplicity of the graphics
and something about the simplicity of the music,
like, it still haunts me.
I don't know if that's a childhood thing,
I don't know if that's the same thing
for Call of Duty these days for young kids,
but I still think that the,
when the games are simple,
that simple purity makes for,
like, allows your imagination to take over
and thereby creating a more magical experience.
Like now with better and better graphics,
it feels like your imagination doesn't get to create worlds,
which is kind of interesting.
It could be just an old man on a porch,
like waving at kids these days that have no respect,
but I still think that graphics
almost get in the way of the experience.
I don't know.
- Flippant Bird.
- Yeah, that's true.
- Well, I don't know if the imagination-
- It's closed.
- Okay, it's closed.
I don't, yeah, but that's more about games that,
like, that's more like Tetris World
where they optimally, masterfully,
like, create a fun, short-term dopamine experience
versus, I'm more referring to, like, role-playing games
where there's like a story,
you can live in it for months or years.
Like, there's an Elder Scrolls series,
which is probably my favorite set of games.
That was a magical experience.
And the graphics are terrible.
The characters were all randomly generated,
but they're, I don't know, it pulls you in.
There's a story.
It's like an interactive version
of an Elder Scrolls Tolkien world.
And you get to live in it.
I don't know.
I miss it.
It's one of the things that suck about being an adult
is there's no, you have to live in the real world
as opposed to the Elder Scrolls world.
- You know, whatever brings you joy, right?
Minecraft, right?
Minecraft's a great example.
You create, like, it's not the fancy graphics,
but it's the creation of your own worlds.
- Yeah, that one is crazy.
You know, one of the pitches for being a parent
that people tell me is that you can, like,
use the excuse of parenting
to go back into the video game world.
And, like, that's like, you know,
father, son, father, daughter time,
but really you just get to play video games with your kids.
So anyway, at that time,
did you have any ridiculous, ambitious dreams
of where as a creator you might go as an engineer?
What did you think of yourself as an engineer,
as a tinker, or did you want to be like an astronaut
or something like that?
- You know, I'm tempted to make something up about,
you know, robots, engineering,
or, you know, mysteries of the universe,
but that's not the actual memory that pops into my mind
when you ask me about childhood dreams,
so I'll actually share the real thing.
When I was maybe four or five years old,
I, as we all do, I thought about,
you know, what I wanted to do when I grow up,
§02Transcript · 08:01–16:27
and I had this dream of being a trainer
and a traffic control cop.
You know, they don't have those today, I think,
but, you know, back in the '80s and, you know, in Russia,
you probably are familiar with that, Lex.
They had these, you know, police officers
that would stand in the middle of an intersection all day,
and they would have their, like,
striped, black and white batons
that they would use to, you know,
control the flow of traffic.
And, you know, for whatever reasons,
I was strangely infatuated with this whole process,
and like that, that was my dream.
That's what I wanted to do when I grew up.
And, you know, my parents, both physics profs, by the way,
I think were, you know, a little concerned
with that level of ambition coming from their child,
at, you know, that age.
- Well, that, it's an interesting,
I don't know if you can relate,
but I very much love that idea.
I have a OCD nature that I think lends itself
very close to the engineering mindset,
which is you want to kind of optimize,
you know, solve a problem
by creating an automated solution,
like a set of rules, a set of rules you can follow,
and then thereby make it ultra efficient.
I don't know if that's, it was of that nature.
I certainly have that.
There's like SimCity and factory building games,
all those kinds of things,
kind of speak to that engineering mindset.
Or did you just like the uniform?
- I think it was more of the latter.
I think it was the uniform and the, you know,
the stripe baton that made cars go in the right directions.
But I guess, you know, it is, I did end up,
I guess, you know, working on the transportation industry
one way or another.
- No uniform, no.
- That's right.
- Maybe it was my, you know, deep inner infatuation
with the traffic control batons that led to this career.
- Okay, what, when did you,
when was the leap from programming to robotics?
- That happened later.
That was after grad school.
After, and actually, then it was self-driving cars
was I think my first real hands-on introduction
to robotics.
But I never really had that much hands-on experience
in school and training.
I worked on applied math and physics.
Then in college, I did more kind of abstract
computer science.
And it was after grad school that I really got involved
in robotics, which was actually self-driving cars.
And, you know, that was a big, big flip.
- What grad school?
- So I went to grad school in Michigan,
and then I did a postdoc at Stanford,
which is, that was the postdoc where I got to play
with self-driving cars.
- Yeah, so we'll return there.
Let's go back to Moscow.
So, you know, for episode 100, I talked to my dad.
And also I grew up with my dad, I guess.
(laughing)
So I had to put up with him for many years.
And he went to the FISTIEH, or MIPT.
It's weird to say in English,
'cause I've heard all of this in Russian.
Moscow Institute of Physics and Technology.
And to me, that was like, I met some super interesting,
as a child, I met some super interesting characters.
It felt to me like the greatest university in the world,
the most elite university in the world.
And just the people that I met that came out of there
were like, not only brilliant, but also special humans.
It seems like that place really tested the soul.
(laughing)
Both in terms of technically and spiritually.
So that could be just the romanticization of that place,
I'm not sure, so maybe you can speak to it.
But is it correct to say that you spent some time
at FISTIEH?
- Yeah, that's right, six years.
I got my bachelor's and master's in physics and math there.
And it's actually interesting, 'cause my dad,
and actually both my parents went there,
and I think all the stories that I heard,
just like you, Alex, growing up about the place
and how interesting and special and magical it was,
I think that was a significant, maybe the main reason
I wanted to go there for college.
Enough so that I actually went back to Russia from the US.
I graduated high school in the US.
- You went back there?
- I went back there, yeah.
- Wow.
- Exactly the reaction most of my peers in college had,
but perhaps a little bit stronger,
that point me out as this crazy kid.
- Were your parents supportive of that?
- Yeah, yeah, I gave them your previous question.
They supported me in letting me pursue my passions
and the things that I was interested in.
- That's a bold move, wow.
What was it like there?
- It was interesting.
Definitely fairly hardcore on the fundamentals
of math and physics, and lots of good memories
from those times.
- So, okay, so Stanford,
how'd you get into autonomous vehicles?
- I had the great fortune and great honor
to join Stanford's DARPA Urban Challenge Team in 2006.
This was a third in the sequence of the DARPA challenges.
There were two grand challenges prior to that,
and then in 2007, they held the DARPA Urban Challenge.
So, you know, I was doing my postdoc.
I joined the team and worked on motion planning
for that competition.
- So, okay, so for people who might not know,
I know from a certain,
autonomous vehicles is a funny world.
In a certain circle of people, everybody knows everything,
and in a certain circle, nobody knows anything
in terms of general public.
So, it's interesting.
It's a good question what to talk about,
but I do think that the Urban Challenge is worth revisiting.
It's a fun little challenge,
one that first sparked so many incredible minds
to focus on one of the hardest problems of our time
in artificial intelligence.
So, that's a success from a perspective
of a single little challenge.
But can you talk about what did the challenge involve?
So, were there pedestrians?
Were there other cars?
What was the goal?
Who was on the team?
How long did it take?
Any fun sort of specs?
- Sure, sure, sure.
So, the way the challenge was constructed
and just a little bit of backgrounding,
as I mentioned, this was the third competition
in that series.
The first two were the Grand Challenge,
called the Grand Challenge.
The goal there was to just drive
in a completely static environment.
You know, you had to drive in a desert.
That was very successful.
So, then DARPA followed with what they called
the Urban Challenge, where the goal was to have,
you know, build vehicles that could operate
in more dynamic environments
and, you know, share them with other vehicles.
There were no pedestrians there.
But what DARPA did is they took over
an abandoned Air Force Base.
And it was kind of like a little fake city
that they built out there.
And they had a bunch of robots, you know, cars,
that were autonomous in there, all at the same time,
mixed in with other vehicles driven by professional drivers.
And each car had a mission.
And so, there's a crude map that they received
at the beginning.
And they had a mission, you know, go here and then there
and over here.
And they kind of all were sharing this environment
at the same time they had to interact with each other,
they had to interact with the human drivers.
So, it's this very first, very rudimentary version
of a self-driving car that, you know, could operate
in an environment, you know,
shared with other dynamic actors.
That, as you said, you know, really, in many ways,
you know, kick-started this whole industry.
- Okay, so who was on the team and how'd you do?
I forget.
(laughing)
- I came in second.
Perhaps that was my contribution to the team.
I think the Stanford team came in first
in the DARPA challenge, but then I joined the team.
And, you know, we came-
- You were the only one with a bug in the code.
I mean, do you have sort of memories
of some particularly challenging things or,
you know, one of the cool things, it's not, you know,
this isn't a product, this isn't a thing that, you know,
it, there's, you have a little bit more freedom
to experiment, so you can take risks,
§03Transcript · 16:29–24:03
and there's, so you can make mistakes.
Is there interesting mistakes?
Is there interesting challenges that stand out to you
as something that taught you a good technical lesson
or a good philosophical lesson from that time?
- Yeah, you know, definitely, definitely
a very memorable time.
Not really a challenge, but like one of the most vivid
memories that I have from the time,
and I think that was actually one of the days
that really got me hooked on this whole field was
the first time I got to run my software on the car.
And I was working on a part of our planning algorithm
that had to navigate in parking lots.
So it was something that, you know,
called free space motion planning.
So the very first version of that was, you know,
we tried on the car, it was on Stanford's campus
in the middle of the night, and you had this little,
you know, course constructed with cones
in the middle of a parking lot.
So we're there in like 3 a.m., you know,
by the time we got the code to, you know,
compile and turn over, and, you know, it drove.
I could actually did something quite reasonable.
And, you know, it was of course very buggy at the time
and had all kinds of problems,
but it was pretty darn magical.
I remember going back and, you know,
late at night and trying to fall asleep
and just, you know, being unable to fall asleep
for the rest of the night, just my mind was blown.
And that's what I've been doing ever since
for more than a decade.
- In terms of challenges and, you know,
interesting memories, like on the day of the competition,
it was pretty nerve wracking.
I remember standing there with Mike Montemarolo,
who was the software lead and wrote most of the code.
I think I did one little part of the planner,
Mike, you know, incredibly did pretty much the rest of it
with a bunch of other incredible people.
But I remember standing on the day of the competition,
you know, watching the car, you know, with Mike,
and, you know, cars are completely empty, right?
They're all there lined up in the beginning of the race.
And then, you know, DARPA sends them, you know,
on their mission one by one.
So then leave, and Mike, you just,
they had these sirens,
(imitates siren)
they all had their different silence, right?
Each siren had its own personality, if you will.
So, you know, off they go and you don't see them.
You just kind of, and then every once in a while,
they, you know, come a little bit closer
to where the audience is,
and you can kind of hear, you know,
the sound of your car and, you know,
it seems to be moving along.
So that, you know, it gives you hope.
And then, you know, it goes away
and you can't hear it for too long.
You start getting anxious, right?
So it's a little bit like, you know,
sending your kids to college and like, you know,
kind of you invested in them.
You hope you build it properly,
but like it's still anxiety inducing.
So that was an incredibly fun few days.
In terms of, you know, bugs,
as you mentioned, you know, one,
that was my bug that caused us the loss of the first place,
is there still a debate that, you know,
occasionally have with people on the CMU team.
CMU came first, I should mention.
- CMU, haven't heard of them, but yeah.
It's a small school.
It's really a glitch that, you know,
they happen to succeed at something robotics related.
- Very scenic though.
So most people go there for the scenery.
Yeah, it's a beautiful campus.
(laughing)
I apologize. - Unlike Stanford.
- So for people, yeah, that's true, unlike Stanford.
For people who don't know,
CMU is one of the great robotics
and sort of artificial intelligence universities
in the world.
CMU, Carnegie Mellon University.
Okay, sorry, go ahead.
- Good PSA.
So in the part that I contributed to,
which was navigating parking lots,
and the way that part of the mission worked is,
in a parking lot, you would get from DARPA
an outline of the map.
You basically get this giant polygon
that defined the perimeter of the parking lot.
And there would be an entrance,
and maybe multiple entrances or access to it.
And then you would get a goal within that open space.
XY heading, where the car had to park.
It had no information about the obstacles
that the car might encounter there.
So it had to navigate a completely free space
from the entrance to the parking lot
into that parking space.
And then once you're parked there,
it had to exit the parking lot.
While of course, encountering and reasoning
about all the obstacles that it encounters in real time.
So our interpretation,
or at least my interpretation of the rules
was that you had to reverse out of the parking spot.
And that's what our cars did,
even if there's no obstacle in front.
That's not what CMU's car did.
And it just kind of drove right through.
So there's still a debate.
And of course, you know,
as you stop and then reverse out
and go out the different way,
that costs you some time, right?
So there's still a debate,
whether it was my poor implementation
that cost us extra time,
or whether it was CMU violating
an important rule of the competition.
And I have my own opinion here.
In terms of other bugs,
and I have to apologize to Mike Montemariella
for sharing this on air,
but it is actually one of the more memorable ones.
And it's something that's kind of become a bit of a metaphor,
I don't know, a label in the industry since then.
I think at least in some circles,
it's called the victory circle or victory lap.
And our cars did that.
So in one of the missions in the urban challenge,
in one of the courses,
there was this big oval right by the start
and finish of the race.
So the ARPA had a lot of the missions would finish
kind of in that same location.
And it was pretty cool
because you could see the cars come by,
you know, kind of finish that part,
like over the trip,
or that like over the mission,
and then, you know, go on and finish the rest of it.
And other vehicles would, you know,
come hit their waypoint and, you know,
exit the oval and off they would go.
Our car in the hand, which hit the checkpoint,
and then it would do an extra lap around the oval
and only then, you know, leave and go on its merry way.
So over the course of, you know, the full day,
it accumulated some extra time.
And the problem was that we had a bug
where it wouldn't, you know,
start reasoning about the next waypoint
and plan a route to get to that next point
until it hit a previous one.
And in that particular case,
by the time you hit that one,
it was too late for us to consider the next one
and kind of make a lane change.
So every time it would do like an extra lap.
So, you know, that's the Stanford victory lap.
- The victory lap.
(laughing)
Oh, that's, I feel like there's something
philosophically profound in there somehow,
but I mean, ultimately,
everybody is a winner in that kind of competition.
And it led to sort of famously to the creation
of Google self-driving car project.
And now Waymo.
So can we give an overview of how is Waymo born?
How's the Google self-driving car project born?
What is the mission?
What is the hope?
What is it is the engineering kind of set of milestones
that it seeks to accomplish?
There's a lot of questions in there.
- Yeah.
(laughing)
I mean, you're right.
Kind of the DARPA Urban Challenge
and the previous DARPA Grand Challenges
kind of led, I think, to a very large degree
to that next step.
And then Larry and Sergey, Larry Page and Sergey Brin,
Google Founders Group, saw that competition
and believed in the technology.
So the Google self-driving car project was born.
You know, at that time, and we started in 2009,
it was a pretty small group of us,
§04Transcript · 24:04–33:30
about a dozen people who came together
to work on this project at Google.
At that time, we saw that incredible early result
in the DARPA Urban Challenge.
I think we're all incredibly excited about where we got to.
And we believed in the future of the technology,
but we still had a very rudimentary understanding
of the problem space.
So the first goal of this project in 2009
was to really better understand what we're up against.
And with that goal in mind, when we started the project,
we created a few milestones for ourselves
that maximized learnings, if you will.
The two milestones were,
one was to drive 100,000 miles in autonomous mode,
which was at that time, orders of magnitude
that more than anybody has ever done.
And the second milestone was to drive 10 routes.
Each one was 100 miles long.
They were specifically chosen to be kind of extra spicy,
extra complicated, and sampled the full complexity
of that domain.
And you had to drive each one from beginning to end
with no intervention, no human intervention.
So you would get to the beginning of the course,
you would press the button that would engage in autonomy,
and you had to go for 100 miles,
beginning to end with no interventions.
And it sampled, again,
the full complexity of driving conditions.
Some were on freeways.
We had one route that went all through all the freeways
and all the bridges in the Bay Area.
You know, we had some that went around Lake Tahoe
and kind of mountains roads.
We had some that drove through dense urban environments
like in downtown Palo Alto and through San Francisco.
So it was incredibly interesting to work on.
And it took us just under two years,
about a year and a half, a little bit more,
to finish both of these milestones.
And in that process,
A, it was an incredible amount of fun,
probably the most fun I had in my professional career.
And you're just learning so much.
The goal here is to learn and prototype.
You're not yet starting to build a production system.
So you just, you were,
this is when you're kind of working 24/7
and you're hacking things together.
And you also don't know how hard this is.
I mean, that's the point.
Like, so, I mean, that's an ambitious,
if I put myself in that mindset, even still,
that's a really ambitious set of goals.
Like just those two, just picking,
just picking 10 different difficult, spicy challenges
and then having zero interventions.
So like not saying gradually we're going to like,
you know, over a period of 10 years,
we're going to have a bunch of roots
and gradually reduce the number of interventions.
You know, that literally says like,
by as soon as possible, we want to have zero
and on hard roads.
So like, to me, if I was facing that,
it's unclear that whether that takes two years
or whether that takes 20 years.
I mean, it may be-- - It took us under two.
- And I guess that speaks to a really big difference
between doing something once and having a prototype
where you're going after, you know,
learning about the problem
versus how you go about engineering a product that,
where you look at, you know,
do you properly do evaluation?
You look at metrics, you know, drive down,
and you're confident that you can do that.
And I guess that's the, you know,
why it took a dozen people, you know,
16 months or a little bit more than that,
back in 2009 and 2010,
and with the technology of, you know,
more than a decade ago,
that amount of time to achieve that milestone
of 10 routes, 100 miles each and no interventions.
And, you know, it took us a little bit longer
to get to, you know, a full driverless product
that customers use.
- That's another really important moment.
Is there some memories of technical lessons
or just one, like, what did you learn
about the problem of driving from that experience?
I mean, we can now talk about like what you learned
from modern day Waymo,
but I feel like you may have learned some profound things
in those early days, even more so,
because it feels like what Waymo is now
is the trying to, you know, how to do scale,
how to make sure you create a product,
how to make sure it's like safety and all those things,
which is all fascinating challenges.
But like you were facing the more fundamental,
philosophical problem of driving in those early days.
Like what the hell is driving as an autonomous vehicle?
Maybe I'm again, romanticizing it,
but is there some valuable lessons
you picked up over there at those two years?
- A ton.
The most important one is probably
that we believe that it's doable
and we've gotten far enough into the problem that,
you know, we had a, I think only a glimpse
of the true complexity of the domain.
You know, it's a little bit like, you know,
climbing a mountain where you kind of, you know,
see the next peak and you think that's kind of the summit,
but then you get to that and you kind of see
that this is just the start of the journey.
But we've tried, we've sampled enough of the problem space
and we've made enough rapid success,
even, you know, with technology of 2009, 2010,
that it gave us confidence to then, you know,
pursue this as a real product.
- So, okay.
So the next step, you mentioned the milestones
that you had in those two years.
What are the next milestones that then led
to the creation of Waymo and beyond?
- Yeah, it was a really interesting journey.
And, you know, Waymo came a little bit later.
Then, you know, we completed those milestones in 2010.
That was the pivot when we decided to focus
on actually building a product, you know,
using this technology.
The initial couple of years after that,
we were focused on a freeway, you know,
what you would call a driver assist,
maybe, you know, an L3 driver assist program.
Then around 2013, we've learned enough about the space
and have thought more deeply about, you know,
the product that we wanted to build that we pivoted.
We pivoted towards this vision of, you know,
building a driver and deploying it fully driverless vehicles
without a person.
And that's the path that we've been on since then.
And it was exactly the right decision for us.
- So there was a moment where you also considered like,
what is the right trajectory here?
What is the right role of automation
in the task of driving?
Or is still, it wasn't from the early days,
obviously you want to go fully autonomous.
- From the early days, it was not.
I think it was in 20, around 2013, maybe,
that we've, that became very clear and we made that pivot.
And it also became very clear
and that it's, even the way you go building
a driver assist system is, you know,
fundamentally different from how you go building
a fully driverless vehicle.
So, you know, we've pivoted towards the latter
and that's what we've been working on ever since.
And so that was around 2013.
Then there's a sequence of really meaningful for us,
really important, defining milestones since then.
In 2015, we had our first,
actually the world's first fully driverless ride
on public roads.
It was in a custom built vehicle that we had.
I must've seen those.
We called them the Firefly, that, you know,
funny looking, marshmallow looking thing.
And we put a passenger, his name was Steve Mann,
a great friend of our project from the early days.
The man happens to be blind.
So we put him in that vehicle.
The car had no steering wheel, no pedals.
It was an uncontrolled environment, you know,
no lead or chase cars, no police escorts.
And, you know, we did that trip a few times in Austin, Texas.
So that was a really big milestone.
- That was in Austin.
- Yeah. - Cool, okay.
- And, you know, we only, but at that time we're only,
it took a tremendous amount of engineering.
It took a tremendous amount of validation
to get to that point.
But, you know, we only did it a few times.
Maybe we only did that.
It was a fixed route.
It was not kind of a controlled environment,
but it was a fixed route and we only did a few times.
Then in 2016, end of 2016, beginning of 2017,
is when we founded Waymo, the company.
That's when we, kind of, that was the next phase
of the project where I wanted,
we believed in kind of the commercial vision
of this technology.
And it made sense to create an independent entity,
§05Transcript · 33:33–42:27
you know, within that alphabet umbrella
to pursue this product at scale.
Beyond that in 2017, later in 2017,
was another really huge step for us,
really big milestone where we started,
I think it was October of 2017,
where when we started regular driverless operations
on public roads, that first day of operations,
we drove in one day, in that first day,
a hundred miles in driverless fashion.
And then we've, the most important thing
about that milestone was not that, you know,
a hundred miles in one day,
but that it was the start of kind of regular,
ongoing driverless operations.
- And when you say driverless, it means no driver.
- That's exactly right.
So on that first day, we actually had a mix
and we didn't want to like, you know,
be on YouTube and Twitter that same day.
So in many of the rides,
we had somebody in the driver's seat,
but they could not disengage, like the car,
- I gotcha. - not disengaged.
But actually on that first day,
some of the miles were driven
and just completely empty driver's seat.
- And this is the key distinction
that I think people don't realize,
you know, that oftentimes when you talk
about autonomous vehicles,
there's often a driver in the seat
that's ready to take over,
what's called a safety driver.
And then Waymo is really one of the only companies,
at least that I'm aware of,
or at least as like boldly and carefully
and all that is actually has cases.
And now we'll talk about more and more
where there's literally no driver.
So that's another, the interesting case
of where the driver's not supposed to disengage.
That's like a nice middle ground.
They're still there,
but they're not supposed to disengage.
But really there's the case when there's no,
okay, there's something magical
about there being nobody in the driver's seat.
Like, just like to me,
you mentioned the first time you wrote some code
for free space navigation of the parking lot,
that was like a magical moment.
To me, just sort of as an observer of robots,
the first magical moment is seeing
an autonomous vehicle turn,
like make a left turn,
like apply sufficient torque to the steering wheel
to where like there's a lot of rotation.
And for some reason,
and there's nobody in the driver's seat,
for some reason that communicates
that here's a being with power that makes a decision.
There's something about like the steering wheel,
'cause we perhaps romanticize the notion
of the steering wheel.
It's so essential to our conception,
our 20th century conception of a car.
And it turning the steering wheel
with nobody in driver's seat,
that to me, I think maybe to others,
it's really powerful.
Like this thing is in control.
And then there's this leap of trust that you give,
like I'm gonna put my life
in the hands of this thing that's in control.
So in that sense,
when there's no driver in the driver's seat,
that's a magical moment for robots.
So I got a chance to last year to take a ride
in a Waymo vehicle.
And that was the magical moment.
There's like nobody in the driver's seat.
It's like the little details.
You would think it doesn't matter
whether there's a driver or not,
but like if there's no driver
and the steering wheel is turning on its own,
I don't know, that's magical.
- It's absolutely magical.
I have taken many of these rides
in a completely empty car.
No human in the car pulls up.
You call it on your cell phone, it pulls up.
You get in, it takes you on its way.
There's nobody in the car but you, right?
That's something called fully driverless,
our rider only mode of operation.
Yeah, it is magical.
It is transformative.
This is what we hear from our riders.
It kind of really changes your experience.
And that really is what unlocks
the real potential of this technology.
But coming back to our journey,
that was 2017 when we started
truly driverless operations.
Then in 2018, we've launched
our public commercial service
that we called Waymo One in Phoenix.
In 2019, we started offering
truly driverless rider only rides
to our early rider population of users.
And then 2020 has also been
a pretty interesting year.
One of the first ones, less about technology,
but more about the maturing
and the growth of Waymo as a company.
We raised our first round of external financing this year.
We were part of Alphabet,
so obviously we have access to significant resources.
But as kind of on the journey of Waymo maturing as a company,
it made sense for us to partially go externally
in this round.
So we raised about $3.2 billion from that round.
We've also started putting our fifth generation
of our driver, our hardware,
that is on the new vehicle,
but it's also a qualitatively different set
of self-driving hardware
that is now on the JLR pace.
So that was a very important step for us.
- The hardware specs, fifth generation,
I think it'd be fun to maybe,
I apologize if I'm interrupting,
but maybe talk about maybe the generations
with a focus on what we're talking about
on the fifth generation in terms of hardware specs,
like what's on this car?
- Sure.
So we separated out the actual car
that we are driving from the self-driving hardware
we put on it.
Right now we have,
so this is, as I mentioned, the fifth generation.
We've gone through,
we started building our own hardware many, many years ago.
And that Firefly vehicle also had the hardware suite
that was mostly designed, engineered, and built in-house.
Lighters are one of the more important components
that we design and build from the ground up.
So on the fifth generation of our drivers,
of our self-driving hardware
that we're switching to right now,
we have, as with previous generations,
in terms of sensing, we have lighters, cameras, and radars.
And we have a pretty beefy computer
that processes all that information
and makes decisions in real time on board the car.
So in all of the,
and it's really a qualitative jump forward
in terms of the capabilities and the various parameters
and the specs of the hardware
compared to what we had before
and compared to what you can kind of get off the shelf
in the market today.
- Meaning from fifth to fourth or from fifth to first?
- Definitely from first to fifth, but also from the fourth.
- That was the world's dumbest question.
- Definitely, definitely from fourth to fifth.
As well as this, the last step is a big step forward.
- So everything's in-house.
So like LIDAR is built in-house
and cameras are built in-house?
- It's different.
We work with partners.
There's some components that we get from our manufacturing
and supply chain partners.
What exactly is in-house is a bit different.
We do a lot of custom design on all of our sensing models.
There's lighters, radars, cameras.
Exactly, there's, lighters are almost exclusively in-house
and some of the technologies that we have,
some of the fundamental technologies there
are completely unique to Waymo.
That is also largely true about radars and cameras.
It's a little bit more of a mix
in terms of what we do ourselves
versus what we get from partners.
- Is there something super sexy about the computer
that you can mention that's not top secret?
Like for people who enjoy computers for, I mean,
see, there's a lot of machine learning involved,
but there's a lot of just basic compute.
There's, you have to probably do a lot of signal processing
on all the different sensors.
You have to integrate everything.
It has to be in real time.
There's probably some kind of redundancy type of situation.
Is there something interesting you could say
about the computer for the people who love hardware?
§06Transcript · 42:31–51:44
- It does have all of the characteristics,
all the properties that you just mentioned.
Redundancy, very beefy compute for general processing
as well as inference and ML models.
It is some of the more sensitive stuff
that I don't wanna get into for IP reasons,
but yeah, it can, we've shared a little bit
in terms of the specs of the sensors
that we have on the car.
We've actually shared some videos
of what our lidars see in the world.
We have 29 cameras, we have five lidars,
we have six radars on these vehicles,
and you can kind of get a feel for the amount of data
that they're producing.
That all has to be processed in real time
to do perception, to do complex reasoning.
So it kind of gives you some idea
of how beefy those computers are,
but I don't wanna get into specifics
of exactly how we build them.
- Okay, well, let me try some more questions
that you can't get into the specifics of,
like GPU wise, is that something you can get into?
I know that Google works with GPUs and so on.
I mean, for machine learning folks,
it's kind of interesting, or is there no,
how do I ask it?
I've been talking to people in the government about UFOs
and they don't answer any questions.
So this is how I feel right now asking about GPUs.
(laughs)
But is there something interesting that you could reveal
or is it just, you know,
or leave it up to our imagination, some of the compute?
Is there any, I guess, is there any fun trickery?
Like I talked to Chris Latner for a second time
and he was a key person about TPUs
and there's a lot of fun stuff going on in Google
in terms of hardware that optimizes for machine learning.
Is there something you can reveal
in terms of how much, you mentioned customization,
how much customization there is for hardware
for machine learning purposes?
- I'm gonna be like that government, you know,
you've got a guy, a person who bought UFOs.
But I, you know, I guess I will say that it's really,
compute is really important.
We have very data hungry and compute hungry ML models
all over our stack.
And this is where, you know,
both being part of Alphabet as well as designing
our own sensors and the entire hardware suite together
where on one hand you get access to like really rich,
raw sensor data that you can pipe from your sensors
into your compute platform.
Yeah, and build like build a whole pipe
from sensor, raw sensor data to the big compute
as then have the massive compute to process all that data.
This is where we're finding that having a lot of control
of that hardware part of the stack is really advantageous.
- One of the fascinating magical places to me,
again, might not be able to speak to the details,
but it is the other compute, which is like, you know,
this we're just talking about a single car,
but the, you know, the driving experience
is a source of a lot of fascinating data.
And you have a huge amount of data coming in on the car
and, you know, the infrastructure of storing
some of that data to then train or to analyze or so on.
That's a fascinating like piece of it
that I understand a single car,
I don't understand how you pull it all together
in a nice way.
Is that something that you could speak to
in terms of the challenges of seeing the network of cars
and then bringing the data back and analyzing things
that like edge cases of driving,
be able to learn on them to improve the system,
to see where things went wrong, where things went right
and analyze all that kind of stuff.
Is there something interesting there
from an engineering perspective?
- Oh, there's an incredible amount
of really interesting work that's happening there,
both in the real time operation of the fleet of cars
and the information that they exchange with each other
in real time to make better decisions,
as well as on the kind of the off-board component
where you have to deal with massive amounts of data
for training your ML models, evaluating the ML models,
for simulating the entire system
and for evaluating your entire system.
And this is where being part of Alphabet
has once again been tremendously advantageous.
I think we consume an incredible amount of compute
for ML infrastructure.
We build a lot of custom frameworks to get good
on data mining, finding the interesting edge cases
for training and for evaluation of the system
for both training and evaluating sub-components
and sub parts of the system and various ML models,
as well as evaluating the entire system and simulation.
- Okay, is that first piece that you mentioned
that cars communicating to each other, essentially,
I mean, through perhaps through a centralized point,
but what, that's fascinating too.
How much does that help you?
Like, if you imagine, you know, right now,
the number of Waymo vehicles is whatever, X.
I don't know if you can talk to what that number is,
but it's not in the hundreds of millions yet.
And imagine if the whole world is Waymo vehicles,
like that changes potentially the power of connectivity.
Like the more cars you have, I guess, actually,
if you look at Phoenix, 'cause there's enough vehicles,
there's enough, when there's like some level of density,
you can start to probably do some really interesting stuff
with the fact that cars can negotiate,
can be, can communicate with each other
and thereby make decisions.
Is there something interesting there that you can talk to
about like, how does that help with the driving problem
from as compared to just a single car
solving the driving problem by itself?
- Yeah, it's a spectrum.
I, first I'll say that, you know, it helps
and it helps in various ways, but it's not required.
Right now, the way we build our system,
like each cars can operate independently,
they can operate with no connectivity.
So I think it is important that, you know,
you have a fully autonomous, you know,
fully capable driver that, you know,
computerized driver that each car has.
Then, you know, they do share information
and they share information in real time.
It really, really helps.
All right, so the way we do this today is, you know,
whenever one car encounters something interesting
in the world, whether it might be an accident
or a new construction zone,
that information immediately gets, you know,
uploaded over the air and it's propagated
to the rest of the fleet.
So, and that's kind of how we think about maps
as priors in terms of the knowledge of our drivers,
of our fleet of drivers that is distributed
across the fleet and it's updated in real time.
So that's one use case.
You know, you can imagine as the, you know,
the density of these vehicles go up
that they can exchange more information
in terms of what they're planning to do
and start influencing how they interact with each other,
as well as, you know,
potentially sharing some observations, right?
To help with, you know,
if you have enough density of these vehicles where,
you know, one car might be seeing something
that another is relevant to another car
that is very dynamic.
You know, it's not part of kind of your updating
your static prior of the map of the world,
but it's more of a dynamic information
that could be relevant to the decisions
that another car is making real time.
So you can see them exchanging that information
and you can build on that.
But again, I see that as an advantage,
but it's, you know, not a requirement.
- So what about the human in the loop?
So when I got a chance to drive with a ride in a Waymo,
you know, there's customer service.
So like there is somebody that's able to dynamically
like tune in and help you out.
What role does the human play in that picture?
That's a fascinating, like, you know,
the idea of teleoperation,
be able to remotely control a vehicle.
So here, what we're talking about is like,
like frictionless, like a human being able to,
in a frictionless way, sort of help you out.
I don't know if they're able to actually control the vehicle.
Is that something you could talk to?
- Yes. - Okay.
- To be clear, we don't do teleoperation.
I got to believe in teleoperation for a reason
is that's not what we have in our cars.
We do, as you mentioned, have, you know,
version of customer support, you know,
we call it life health.
In fact, we find it that it's very important
for our rider experience, especially if it's your first trip,
you've never been in a fully driverless ride
or only Waymo vehicle, you get in, there's nobody there.
And so you can imagine having all kinds of questions
in your head, like how this thing works.
So we've put a lot of thought into kind of guiding
our riders, our customers through that experience,
§07Transcript · 51:48–59:49
especially for the first time,
they get some information on the phone.
If the fully driverless vehicle is used
to service their trip, when you get into the car,
we have an in-car screen and audio that kind of guides them
and explains what to expect.
They also have a button that they can push
that will connect them to, you know,
a real life human being that they can talk to, right,
about this whole process.
So that's one aspect of it.
There is, you know, I should mention that there is
another function that humans provide to our cars,
but it's not teleoperation.
You can think of it a little bit more like, you know,
fleet assistance, kind of like, you know,
traffic control that you have, where our cars,
again, they're responsible on their own
for making all of the decisions,
all of the driving decisions
that don't require connectivity.
They, you know, anything that is safety or latency critical
is done, you know, purely autonomously by onboard,
our onboard system.
But there are situations where, you know,
if connectivity is available,
and a car encounters a particularly challenging situation,
you can imagine like a super hairy scene of an accident,
the cars will do their best.
They will recognize that it's an off-nominal situation.
They will, you know, do their best to come up, you know,
with the right interpretation,
the best course of action in that scenario.
But if connectivity is available,
they can ask for confirmation from, you know,
a human-mode human assistant
to kind of confirm those actions,
and, you know, perhaps provide a little bit
of kind of contextual information and guidance.
- So October 8th was when you're talking about the,
was Waymo launched the fully self,
the public version of its fully driverless,
that's the right term, I think, service in Phoenix.
Is that October 8th?
- That's right.
It was the introduction of fully driverless
rider-only vehicles into our public Waymo One service.
- Okay, so that's amazing.
So it's like anybody can get into Waymo in Phoenix?
- That's right.
So we previously had early people
in our early rider program
taking fully driverless rides in Phoenix.
And just this, a little while ago,
we opened on October 8th,
we opened that mode of operation to the public.
So I can download the app and go on the ride.
There is a lot more demand right now for that service
than we have capacity.
So we're kind of managing that,
but that's exactly the way you described it.
- Yeah, well, that's interesting.
So there's more demand than you can handle.
Like what has been the reception so far?
Like what, I mean, okay, so, you know,
that's, this is a product, right?
That's a whole nother discussion of like
how compelling of a product it is.
Great, but it's also like one of the most
kind of transformational technologies of the 21st century.
So it's also like a tourist attraction.
Like it's fun to, you know, to be a part of it.
So it'd be interesting to see like, what do people say?
What do people, what have been the feedback so far?
- You know, still early days,
but so far the feedback has been incredible,
incredibly positive.
They, you know, we asked them for feedback during the ride.
We asked them for feedback after the ride
as part of their trip.
You know, we asked them some questions.
We asked them to, you know,
rate the performance of our driver.
Most by far, you know, most of our drivers
give us five stars in our app,
which is absolutely great to see.
And you know, that's, and we're,
they're also giving us feedback on, you know,
things we can improve.
And, you know, that's one of the main reasons
we're doing this with Phoenix.
And, you know, over the last couple of years
and every day today,
we are just learning a tremendous amount of new stuff
from our users.
There's no substitute for actually doing the real thing,
actually having a fully driverless product
out there in the field with, you know,
users that are actually, you know,
paying us money to get from point A to point B.
- So this is a legitimate, like,
this is a paid service.
- That's right.
- And the idea is you use the app to go from point A
to point B, and then what are the A's?
What are the, what's the freedom of the,
of the starting and ending places?
- It's an area of geography where that service is enabled.
It's a decent size of geography of territory.
It's actually larger than, you know,
the size of San Francisco.
And, you know, within that, you have full freedom
of, you know, selecting where you want to go.
You know, of course there are some,
and you, on your app, you get a map,
you tell the car where you want to be picked up,
you know, where you want the car to pull over
and pick you up, and then you tell it
where you want to be dropped off, right?
And of course there are some exclusions, right?
You don't want to be, you know,
where in terms of where the car is allowed to pull over,
right, so, you know, that you can't do,
but, you know, besides that, it's-
- Amazing.
- It's not like a fixed, just would be very,
I guess, I don't know,
maybe that's what's the question behind your question,
but it's not a, you know, preset set of-
- Yes, I guess, so within the geographic constraints
with that, within that area, anywhere else,
it can be picked up and dropped off anywhere.
- That's right, and, you know, people use them
on like all kinds of trips.
They, we have, and we have an incredible spectrum of riders.
I think the youngest, actually have car seats in them,
and we have, you know, people taking their kids on rides.
I think the youngest riders we had on cars
are, you know, one or two years old, you know,
and the full spectrum of use cases.
People, you can take them to, you know, schools,
to, you know, go grocery shopping,
to restaurants, to bars, you know, run errands,
you know, go shopping, et cetera, et cetera.
You can go to your office, right?
Like the full spectrum of use cases,
and people, you're gonna use them in their daily lives
to get around, and we see all kinds of, you know,
really interesting use cases,
and that's what's providing us
incredibly valuable experience
that we then, you know, use to improve our product.
- So as somebody who's been on, done a few long rants
with Joe Rogan and others about the toxicity
of the internet and the comments,
and the negativity in the comments,
I'm fascinated by feedback.
I believe that most people are good and kind and intelligent
and can provide, like, even in disagreement,
really fascinating ideas.
So on a product side, it's fascinating to me,
like, how do you get the richest possible user feedback,
like, to improve?
What are the channels that you use
to measure, 'cause like, you're no longer,
that's one of the magical things about autonomous vehicles,
is it's not, like, it's frictionless interaction
with the human, so like, you don't get to,
you know, it's just giving a ride.
So like, how do you get feedback from people
in order to improve?
- Oh, yeah, great question.
Various mechanisms.
So as part of the normal flow, we ask people for feedback.
They, as the car is driving around,
you know, we have on the phone and in the car,
and we have a touchscreen in the car,
you can actually click some buttons
and provide real-time feedback on how the car is doing
and how the car is handling a particular situation,
you know, both positive and negative.
So that's one channel.
We have, as we discussed, customer support or life help,
where, you know, if a customer wants to,
has a question or he has some sort of concern,
they can talk to a person in real time.
So that is another mechanism that gives us feedback.
At the end of a trip, you know,
we also ask them how things went.
They give us comments and, you know, a star rating.
And, you know, if it's, we also, you know,
ask them to explain what, you know,
went well and, you know, what could be improved.
And we have, our riders are providing,
you know, very rich feedback there.
A lot, a large fraction is very passionate
and very excited about this technology.
So we get really good feedback.
We also run UXR studies, right?
You know, specific that are kind of more, you know,
§08Transcript · 59:53–1:08:28
go more in depth and we will run both kind of lateral
and longitudinal studies where we have deeper engagement
with our customers.
You know, we have our user experience research team
tracking over time.
That's when you say about longitudinal, it's cool.
- That's exactly right.
And, you know, that's another really valuable feedback,
source of feedback.
And we're just covering a tremendous amount, right?
People go grocery shopping and they like want to load,
you know, 20 bags of groceries in our cars.
And like that's one workflow that you maybe don't,
you know, think about, you know, getting just right
when you're building the driverless product.
I have people like, you know, who bike as part
of their trip.
So they, you know, bike somewhere,
then they get in our cars, they take apart their bike,
they load into our vehicle, then they go,
and that's, you know, how they, you know,
where we want to pull over and how that, you know,
get in and get out process works,
provides us very useful feedback.
In terms of, you know, what makes a good pickup
and drop off location, we get really valuable feedback.
And in fact, we had to do some really interesting work
with high definition maps and thinking about
walking directions.
And if you imagine you're in a store, right?
In some giant space, and then, you know,
you want to be picked up somewhere.
If you just drop a pin at a current location,
which is maybe in the middle of a shopping mall,
like what's the best location for the car
to come pick you up?
And you can have simple heuristics where you just kind of
take your, you know, you clean in distance
and find the nearest spot where the car can pull over
that's closest to you.
But oftentimes that's not the most convenient one.
You know, I have many anecdotes where that heuristic
breaks in horrible ways.
One example that I often mention is somebody wanted to be,
you know, dropped off in Phoenix and, you know,
we got car picked a location that was close,
I think the closest to their, you know,
where the pin was dropped on the map in terms of,
you know, latitude and longitude.
But it happened to be on the other side of a parking lot
that had this row of cacti and the poor person had to like
walk all around the parking lot to get to where they wanted
to be in 110 degree heat.
So that, you know, that was the bottom.
So then, you know, we took all, take all of these,
all of that feedback from our users and incorporate it
into our system and improve it.
- Yeah, I feel like that's like requires AGI
to solve the problem of like, when you're,
which is a very common case,
when you're in a big space of some kind,
like apartment building, it doesn't matter.
It's just not some large space.
And then you call the, like the Waymo from there, right?
It's like, whatever, it doesn't matter,
a ride share vehicle.
And like, where's the pin supposed to drop?
I feel like that's, you don't think,
I think that requires AGI.
I'm gonna, in order to solve, okay, the alternative,
which I think the Google search engine has taught
is like, there's something really valuable
about the perhaps slightly dumb answer,
but a really powerful one, which is like,
what was done in the past by others?
Like, what was the choice made by others?
That seems to be like, in terms of Google search,
when you have like billions of searches,
that you could see which, like when they recommend
what you might possibly mean,
they suggest based on not some machine learning thing,
which they also do, but like,
on what was successful for others in the past
and finding a thing that they were happy with.
Is that integrated at all with Waymo?
Like, what pickups worked for others?
- It is.
I think you're exactly right.
So there's a real, it's an interesting problem.
Naive solutions have interesting failure modes.
So there's definitely lots of things
that can be done to improve
and both learning from what works,
what doesn't work in actual hail
from getting richer data and getting more information
about the environment and richer maps.
But you're absolutely right that there's something,
I think there's some properties of solutions
that in terms of the effect that they have on users,
some are much, much, much better than others, right?
And predictability and understandability is important.
So you can have maybe something
that is not quite as optimal,
but is very natural and predictable to the user
and kind of works the same way all the time.
And that matters.
That matters a lot for the user experience.
- But to get to the basics,
the pretty fundamental property
is that the car actually arrives where you told it to ride.
Like you can always change it, see it on the map
and you can move it around if you don't like it.
But like that property that the car actually shows up
on the pin is critical,
which where compared to some of the human driven analogs,
I think you can have more predictability.
It's actually the fact,
if I do a little bit of a detour here,
I think the fact that it's your phone and the car,
it's two computers talking to each other
can lead to some really interesting things we can do
in terms of the user interfaces,
both in terms of function,
like the car actually shows up exactly where you told it
you want it to be,
but also some really interesting things
on the user interface,
like as the car is driving,
as you call it and it's on the way to come and pick you up.
And of course you get the position of the car
and the route on the map,
but, and they actually follow that route, of course,
but it can also share some really interesting information
about what it's doing.
So, you know, our cars, as they are coming to pick you up,
if it's come, if a car is coming up to a stop sign,
it will actually show you that like it's there sitting
because it's at a stop sign or a traffic light
will show you that it's got, you know,
sitting at a red light.
So, you know, they're like little things, right?
But it's, I find those little touch,
touches really interesting, really magical.
And it's just, you know,
little things like that that you can do
to kind of delight your users.
- You know, this makes me think of,
there's some products that I just love.
Like there's a company called Rev,
Rev.com where I like for this podcast, for example,
I can just drag and drop a video
and then they do all the captioning.
It's humans doing the captioning,
but they connect, they automate everything
of connecting you to the humans
and they do the captioning and transcription.
It's all effortless.
And it like, I remember when I first started using them,
it was like, life's good.
Like, because it was so painful to figure that out earlier.
The same thing with something called iZotope RX,
this company I use for cleaning up audio,
like the sound cleanup they do,
it's like drag and drop
and it just cleans everything up very nicely.
Another experience like that I had
with Amazon OneClick purchase first time,
I mean, other places do that now,
but just the effortlessness of purchasing,
making it frictionless,
it kind of communicates to me,
like I'm a fan of design, I'm a fan of products,
that you can just create a really pleasant experience.
The simplicity of it, the elegance
just makes you fall in love with it.
So, do you think about this kind of stuff?
I mean, it's exactly what we've been talking about.
It's like the little details
that somehow make you fall in love with the product.
Is that, we went from like urban challenge days
where love was not part of the conversation probably,
and to this point where there's human beings
and you want them to fall in love with the experience.
Is that something you're trying to optimize for,
trying to think about,
like how do you create an experience that people love?
- Oh, absolutely.
I think that's the vision is removing any friction
or complexity from getting our users,
our writers to where they wanna go.
And making that as simple as possible.
And then, beyond that, just transportation,
making things and goods get to their destination
as seamlessly as possible.
I talked about a drag and drop experience
where I kind of express your intent
and then it just magically happens.
And for our writers, that's what we're trying to get to
is you download an app and you click and car shows up.
It's the same car.
§09Transcript · 1:08:29–1:18:24
It's very predictable.
It's a safe and high quality experience.
And then it gets you in a very reliable,
very convenient, frictionless way to where you wanna be.
And along the journey,
I think we also want to do little things
to delight our users.
- Like the ride sharing companies,
because they don't control the experience, I think,
they can't make people fall in love necessarily
with the experience.
Or maybe they haven't put in the effort.
But I think if I were to speak
to the ride sharing experience I currently have,
it's just very convenient.
But there's a lot of room for falling in love with it.
We can speak to sort of car companies.
Car companies do this well.
You can fall in love with a car, right?
And be like a loyal car person, like whatever.
Like I like badass hot rods, I guess '69 Corvette.
And at this point, you can't really...
Cars are so, owning a car is so 20th century, man.
But is there something about the Waymo experience
where you hope that people will fall in love with it?
Is that part of it?
Or is it just about making a convenient ride,
not ride sharing, I don't know what the right term is,
but just a convenient A to B autonomous transport?
Or like, do you want them to fall in love with Waymo?
So maybe elaborate a little bit.
I mean, almost like from a business perspective,
I'm curious, like how...
Do you wanna be in the background invisible?
Or do you want to be like a source of joy
that's in very much in the foreground?
- I want to provide the best,
most enjoyable transportation solution.
And that means building it, building our product
and building our service in a way that people do.
Kind of use in a very seamless, frictionless way
in their day-to-day lives.
And I think that does mean,
in some way falling in love in that product, right?
It just kind of becomes part of your routine.
It comes down in my mind to safety,
predictability of the experience and privacy, I think.
Privacy, I think, aspects of it, right?
Our cars, you get the same car,
you get very predictable behavior.
And that is important.
I think if you're gonna use it in your daily life.
Privacy, I mean, when you're in a car,
you can do other things.
You're spending a bunch, just another space
where you're spending a significant part of your life.
And so not having to share it with other people
who you don't wanna share it with,
I think is a very nice property.
Maybe you wanna take a phone call
or do something else in the vehicle.
And safety on the quality of the driving,
as well as the physical safety of not having to share
that ride is important to a lot of people.
- What about the idea that when there's somebody,
like a human driving and they do a rolling stop
on a stop sign, like sometimes,
you get an Uber or Lyft or whatever, like human driver,
and they can be a little bit aggressive as drivers.
It feels like there is, not all aggression is bad.
Now that may be a wrong, again,
20th century conception of driving.
Maybe it's possible to create a driving experience.
Like if you're in the back, busy doing something,
maybe aggression is not a good thing.
It's a very different kind of experience perhaps.
But it feels like in order to navigate this world,
you need to, how do I phrase this?
You need to kind of bend the rules a little bit,
or at least like test the rules.
I don't know what language politicians use to discuss this,
but whatever language they use,
you like flirt with the rules, I don't know.
But like you sort of have a bit of an aggressive way
of driving that asserts your presence in this world,
thereby making other vehicles
and people respect your presence,
and thereby allowing you to sort of navigate
through intersections in a timely fashion.
I don't know if any of that made sense,
but like how does that fit into the experience
of driving autonomously?
Is that-- - It's a lot of stuff.
This is, you're hitting on a very important point
of a number of behavioral components
and parameters that make your driving feel assertive
and natural and comfortable and predictable.
Now, our cars will follow rules, right?
They will do the safest thing possible in all situations,
let me be clear on that.
But if you think of really, really good drivers,
just think about professional lemon drivers, right?
They will follow the rules.
They're very, very smooth, and yet they're very efficient.
But they're assertive.
They're comfortable for the people in the vehicle.
They're predictable for the other people outside the vehicle
that they share the environment with.
And that's the kind of driver that we want to build.
And you think if maybe there's a sport analogy there,
you can do in very many sports,
the true professionals are very efficient
in their movements, right?
So they don't do like hectic flailing, right?
They're smooth and precise, right?
And they get the best results.
So that's the kind of driver that we want to build.
In terms of aggressiveness, yeah,
you can like roll through the stop signs.
You can do crazy lane changes.
Typically doesn't get you to your destination faster.
Typically not the safest or most predictable,
or most comfortable thing to do.
But there is a way to do both.
And that's what we're trying to build,
the driver that is safe, comfortable,
smooth, and predictable.
- Yeah, that's a really interesting distinction.
I think in the early days of autonomous vehicles,
the vehicles felt cautious as opposed to efficient.
And still probably, but when I rode in the Waymo,
I mean, it was quite assertive.
It moved pretty quickly.
And like, yeah, then he's one of the surprising feelings
was that it actually, it went fast
and it didn't feel like awkwardly cautious
than autonomous vehicle.
So I've also programmed autonomous vehicles
and everything I've ever built was felt awkwardly,
either overly aggressive, okay?
Especially when it was my code,
or like awkwardly cautious is the way I would put it.
And Waymo's vehicle felt like assertive
and I think efficient is like the right terminology here.
They wasn't, and I also like the professional limo driver.
'Cause we often think like, you know,
an Uber driver or a bus driver or a taxi.
This is the funny thing is people think
like taxi drivers are professionals.
I mean, it's like, that's like saying
I'm a professional walker
just because I've been walking all my life.
I think there's an art to it, right?
And if you take it seriously as an art form,
then there's a certain way that mastery looks like.
And it's interesting to think about
what does mastery look like in driving?
And perhaps what we associate with like aggressiveness
is unnecessary, like it's not part
of the experience of driving.
It's like unnecessary fluff that efficiency,
you can be, you can create a good driving experience
within the rules.
That's, I mean, you're the first person to tell me this.
So it's kind of interesting.
I need to think about this,
but that's exactly what it felt like with Waymo.
I kind of had this intuition,
maybe it's the Russian thing, I don't know,
that you have to break the rules in life to get anywhere.
But maybe, maybe it's possible that that's not the case
in driving.
I have to think about that.
But it certainly felt that way on the streets of Phoenix
when I was there in Waymo,
that that was a very pleasant experience
and it wasn't frustrating in that like,
come on, move already kind of feeling.
It wasn't, that wasn't there.
- Yeah, I mean, that's what we're going after.
I don't think you have to pick one.
I think truly good driving,
it gives you both efficiency, a certainness,
but also comfort and predictability and safety.
And that's what fundamental improvements
in the core capabilities truly unlock.
And you can kind of think of it as,
precision and recall trade-off.
You have certain capabilities of your model,
and then it's very easy when you have some curve
of precision and recall, you can move things around
and can choose your operating point
in your training of precision versus recall,
false positives versus false negatives.
But then, and you can tune things on that curve
and be kind of more cautious or more aggressive,
but then aggressive is bad or cautious is bad.
But true capabilities come from actually moving
the whole curve up.
And then you are on a very different plane
§10Transcript · 1:18:28–1:27:55
of those trade-offs.
And that's what we're trying to do here
is to move the whole curve up.
- Before I forget, let's talk about trucks a little bit.
So I also got a chance to check out
some of the Waymo trucks.
I'm not sure if we wanna go too much into that space,
but it's a fascinating one.
So maybe we can mention at least briefly,
Waymo is also now doing autonomous trucking.
And how different, like philosophically and technically
is that whole space of problems?
- It's one of our two big products
and commercial applications of our driver,
ride hailing and deliveries.
We have Waymo One and Waymo Via,
moving people and moving goods.
Trucking is an example of moving goods.
We've been working on trucking since 2017.
It is a very interesting space.
And your question of how different is it?
It has this really nice property
that the first order challenges,
like the science, the hard engineering,
whether it's hardware or onboard software
or off-board software,
all of the systems that you build
for training your ML models,
for evaluating your entire system,
those fundamentals carry over.
The true challenges of driving, perception,
semantic understanding, prediction,
decision making, planning, evaluation,
the simulator, ML infrastructure, those carry over.
Like the data and the application
and kind of the domains might be different,
but the most difficult problems,
all of that carries over between the domains.
So that's very nice.
So that's how we approach it.
We're kind of build investing in the core,
the technical core.
And then there is specialization of that core technology
to different product lines,
to different commercial applications.
So on, just to tease it apart a little bit,
on trucks, so starting with the hardware,
the configuration of the sensors is different.
They're different physically, geometrically,
different vehicles.
So for example, we have two of our main laser
on the trucks on both sides,
so that we have, not have the blind spots.
Whereas on the JLR I-PACE, we have one of it,
sitting at the very top,
but the actual sensors are almost the same
or largely the same.
So all of the investment that over the years
we've put into building our custom lighters,
custom radars, pulling the whole system together,
that carries over very nicely.
Then on the perception side,
the fundamental challenges of seeing,
understanding the world,
whether it's object detection, classification,
tracking, semantic understanding, all that carries over.
Yes, there's some specialization
when you're driving on freeways,
range becomes more important,
the domain is a little bit different,
but again, the fundamentals carry over very, very nicely.
Same, and I guess you get into prediction
or decision-making, right?
The fundamentals of what it takes to predict
what other people are going to do,
to find the long tail, to improve your system
in that long tail of behavior prediction and response,
that carries over, right?
And so on and so on.
So, I mean, that's pretty exciting.
By the way, does Waymo VIA include
using the smaller vehicles for transportation goods?
That's an interesting distinction.
So, I would say there's three interesting
modes of operation.
So, one is moving humans, one is moving goods,
and one is like moving nothing, zero occupancy,
meaning like you're going to the destination,
you're empty vehicle.
I mean, it's-
- The third is the less of it,
that's the entirety of it,
it's the less exciting from the commercial perspective.
(both laughing)
- Well, I mean, in terms of like,
if you think about what's inside a vehicle as it's moving,
because it does, some significant fraction
of the vehicle's movement has to be empty.
I mean, it's kind of fascinating,
maybe just on that small point,
is there different control and like,
policies that are applied for zero occupancy vehicle?
So, a vehicle with nothing in it,
or is it just move as if there is a person inside,
what was with some subtle differences?
- As a first order approximation, there are no differences.
And if you think about, you know, safety and, you know,
comfort and quality of driving, only part of it,
has to do with the people or the goods
inside of the vehicle, right?
But you don't wanna be, you know,
you wanna drive smoothly, as we discussed,
not for the purely for the benefit of, you know,
whatever you have inside the car, right?
It's also for the benefit of the people outside
kind of feeding, fitting naturally and predictably
into that whole environment, right?
So, you know, yes, there are some second order things
you can do, you can change your route,
and, you know, optimize maybe kind of your fleet,
things at the fleet scale,
and you would take into account whether, you know,
some of your cars are actually, you know,
serving a useful trip, whether with people or with goods,
whereas, you know, other cars are, you know,
driving completely empty, you know,
to that next valuable trip that they're going to provide,
but that those are mostly second order effects.
- Okay, cool.
So Phoenix is an incredible place,
and what you've announced in Phoenix is,
it's kind of amazing, but, you know,
that's just like one city.
How do you take over the world?
I mean, I'm asking for a friend.
- One step at a time.
- One city at a time.
- Is that the cartoon pinky in the brain?
Okay.
- But, you know, gradually is a true answer.
- So I think the heart of your question is, you know, what--
- Can you ask a better question than I asked?
- You're asking a great question.
- Answer that one.
- I'm, you know, just gonna, you know,
phrase it in the terms that I want to answer.
- Answer it, perfect.
That's exactly right, brilliant.
Please.
- No, you know, where are we today?
And, you know, what happens next?
And what does it take to go beyond Phoenix?
And what does it take to get this technology
to more places and more people around the world, right?
- So our next big area of focus
is exactly that, larger scale commercialization
and just, you know, scaling up.
If I think about, you know, the main,
and, you know, Phoenix gives us that platform
and gives us that foundation upon which we can build.
And it's, there are a few really challenging aspects
of this whole problem that you have to pull together
in order to, you know, build the technology,
in order to deploy it into the field,
to go from a driverless car to a fleet of cars
that are providing a service,
and then all the way to, you know, commercialization.
So, and, you know, this is what we have in Phoenix.
We've taken the technology from a proof point
to an actual deployment,
and have taken our driver, you know, from, you know,
one car to a fleet that can provide a service.
Beyond that, if I think about what it will take
to scale up and, you know, deploy in, you know,
more places with more customers,
I tend to think about three main dimensions,
three main axes of scale.
One is the core technology, you know,
the hardware and software core capabilities of our driver.
The second dimension is evaluation and deployment.
And the third one is just the, you know,
product, commercial, and operational excellence.
So you can talk a bit about where we are along, you know,
each one of those three dimensions,
about where we are today and, you know,
what has, what will happen next.
On, you know, the core technology,
on, you know, the hardware and software, you know,
together, comprised of driver,
we, you know, obviously have that foundation
that is providing fully driverless trips to our customers
as we speak, in fact.
And we've learned a tremendous amount from that.
So now what we're doing is we are incorporating
all those lessons into some pretty fundamental improvements
in our core technology, both on the hardware side
and on the software side,
to build a more general, more robust solution
that then will enable us to massively scale,
you know, beyond Phoenix.
§11Transcript · 1:27:57–1:36:48
So on the hardware side,
all of those lessons are now incorporated
into this fifth generation hardware platform
that is, you know, being deployed right now.
And that's the platform, the fourth generation,
the thing that we have right now driving in Phoenix,
it's good enough to operate fully driverless,
you know, night and day, you know,
various speeds and various conditions.
But the fifth generation is the platform
upon which we want to go to massive scale.
We've really made qualitative improvements
in terms of the capability of the system,
the simplicity of the architecture,
the reliability of the redundancy.
It is designed to be manufacturable at very large scale
and, you know, provides the right unit economics.
So that's the next big step for us on the hardware side.
- That's already there for scale, the version five.
- That's right.
- And is that a coincidence
or should we look into a conspiracy theory
that it's the same version as the Pixel phone?
(laughs)
Is that what's the hardware?
- I can neither confirm nor deny, Lux.
- All right, cool.
So, sorry.
So that's the, okay, that's that axis.
What else?
- So similarly, you know,
the hardware is a very discreet jump,
but similar to how we're making that change
from the fourth generation hardware to the fifth,
we're making similar improvements on the software side
to make it more robust and more general
and allow us to kind of quickly scale beyond Phoenix.
So that's the first dimension of core technology.
The second dimension is evaluation and deployment.
Now, how do you measure your system?
How do you evaluate it?
How do you build a release and deployment process
where, you know, with confidence,
you can regularly release new versions
of your driver into a fleet?
How do you get good at it so that it is not, you know,
a huge tax on your researchers and engineers that, you know,
so you can, how do you build all of these, you know,
processes, the frameworks, the simulation, the evaluation,
the data science, the validation,
so that, you know, people can focus on improving the system
and kind of the releases just go out the door
and get deployed across the fleet.
So we've gotten really good at that in Phoenix.
That's been a tremendously difficult problem,
but that's what we have in Phoenix right now
that gives us that foundation.
And now we're working on kind of incorporating
all the lessons that we've learned
to make it more efficient, to go to new places,
you know, and scale up and just kind of, you know,
stamp things out.
So that's that second dimension of evaluation and deployment.
And the third dimension is product, commercial,
and operational excellence, right?
And again, Phoenix there is providing
an incredibly valuable platform.
You know, that's why we're doing things end-to-end
in Phoenix, we're learning as, you know,
we discussed a little earlier today,
tremendous amount of really valuable lessons
from our users getting really incredible feedback.
And we'll continue to iterate on that
and incorporate all those lessons
into making our product, you know,
even better and more convenient for our users.
- So you're converting this whole process of Phoenix
in Phoenix into something that could be copied
and pasted elsewhere.
So like, perhaps you didn't think of it that way
when you were doing the experimentation Phoenix,
but so how long did, basically,
you can correct me, but you've,
I mean, it's still early days,
but you've taken the full journey in Phoenix, right?
As you were saying of like what it takes
to basically automate, I mean,
it's not the entirety of Phoenix, right?
But I imagine it can encompass the entirety of Phoenix
at some near-term date,
but that's not even perhaps important,
like as long as it's a large enough geographic area.
So what, how copy-pastable is that process currently?
Is that process currently?
And how, like, you know,
like when you copy and paste in Google Docs, I think,
no, or in Word, you can like apply source formatting
or apply destination formatting.
So when you copy and paste the Phoenix into like,
say, Boston, how do you apply the destination formatting?
Like how much of the core of the entire process
of bringing an actual public transportation,
autonomous transportation service to a city
is there in Phoenix that you understand enough
to copy and paste into Boston or wherever?
- So we're not quite there yet.
We're not at a point where we're kind of massively
copy and pasting all over the place,
but Phoenix, what we did in Phoenix,
and we very intentionally have chosen Phoenix
as our first full deployment area,
you know, exactly for that reason,
to kind of tease the problem apart,
look at each dimension,
focus on the fundamentals of complexity
and de-risking those dimensions,
and then bringing the entire thing together
to get all the way,
force ourselves to learn all those hard lessons
on technology, hardware and software,
on the evaluation deployment,
on operating a service, operating a business,
using actually serving our customers,
all the way so that we're fully informed
about the most difficult, most important challenges
to get us to that next step of massive copy and pasting,
as you said.
And that's what we're doing right now.
We're incorporating all those things that we learned
into that next system that then will allow us
to kind of copy and paste all over the place
and to massively scale to more users and more locations.
I mean, you know, I just talked a little bit about,
what does that mean along those different dimensions?
So on the hardware side, for example,
again, it's that switch from the fourth
to the fifth generation.
And the fifth generation is designed
to kind of have that property.
- Can you say what other cities you're thinking about?
Like I'm thinking about,
sorry, we're in San Francisco now.
I thought I want to move to San Francisco,
but I'm thinking about moving to Austin.
I don't know why.
People are not being very nice about San Francisco currently.
Maybe it's in vogue right now.
But Austin seems, I visited there and it was,
I was in a Walmart.
It's funny, these moments like turn your life.
There's this very nice woman with kind eyes,
just like stopped and said,
"You look so handsome in that tie, honey," to me.
This has never happened to me in my life,
but just the sweetness of this woman
is something I've never experienced,
certainly in the streets of Boston,
but even in San Francisco where people wouldn't,
that's just not how they speak or think.
I don't know.
There's a warmth to Austin that love.
And since Waymo does have a little bit of a history there,
is that a possibility?
- Is this your version of asking the question of like,
you know, Dimitri, I know you can't share
your commercial and deployment roadmap,
but I'm thinking about moving to San Francisco, Austin,
like in a blink twice, if you think I should move to.
- Yeah, that's true.
That's true.
You got me.
We've been testing all over the place.
I think we've been testing more than 25 cities.
We drive in San Francisco, we drive in Michigan for snow.
We are doing significant amount of testing in the Bay Area,
including San Francisco.
- But just not like,
'cause we're talking about the various different thing,
which is like a full on large geographic area,
public service.
You can't share.
Okay.
What about Moscow?
Is that, when is that happening?
Take on Yandex.
I'm not paying attention to those folks.
They're doing, you know, there's a lot of fun.
I mean, maybe as a way of a question,
you didn't speak to sort of like policy
or like, is there tricky things with government and so on?
Is there other friction that you've encountered
except sort of technological friction
of solving this very difficult problem?
Is there other stuff that you have to overcome
when deploying a public service in a city?
That's interesting.
- It's very important.
So we put significant effort in creating those partnerships
and those relationships with governments at all levels,
local governments, municipalities,
§12Transcript · 1:36:50–1:45:48
state level, federal level.
We've been engaged in very deep conversations
from the earliest days of our projects
whenever at all of these levels,
whenever we go to test or operate in a new area,
we always lead with a conversation
with the local officials.
But the result of that investment is that,
no, it's not challenges we have to overcome,
but it is a very important
that we continue to have this conversation.
- Yeah, I love politicians too.
Okay, so Mr. Elon Musk said that LIDAR is a crutch.
What are your thoughts?
- I wouldn't characterize it exactly that way.
I know, I think LIDAR is very important.
It is a key sensor that we use just like other modalities.
As we discussed, our cars use cameras, LIDARs and radars.
They are all very important.
They are at the kind of the physical level.
They are very different.
They have very different physical characteristics.
Cameras are passive, LIDARs and radars are active,
and use different wavelengths.
So that means they complement each other very nicely.
And together, combined, they can be used
to build a much safer and much more capable system.
So to me, it's more of a question,
why the heck would you handicap yourself
and not use one or more of those sensing modalities
when they undoubtedly just make your system
more capable and safer?
Now, what might make sense for one product
or one business might not make sense for another one.
So if you're talking about driver assist technologies,
you make certain design decisions
and you make certain trade-offs,
and you make different ones if you're building a driver
that you deploy in fully driverless vehicles.
And LIDAR specifically, when this question comes up,
typically the criticisms that I hear
are the counterpoints that cost and aesthetics.
And I don't find either of those, honestly, very compelling.
So on the cost side,
there's nothing fundamentally prohibitive
about the cost of LIDARs.
Radars used to be very expensive
before people made certain advances in technology
and started to manufacture them at massive scale
and deploy them in vehicles, similar with LIDARs.
And this is where the LIDARs that we have on our cars,
especially the fifth generation,
we've been able to make some pretty qualitative
discontinuous jumps in terms of the fundamental technology
that allow us to manufacture those things
at very significant scale and at a fraction of the cost
of both our previous generation,
as well as a fraction of the cost
of what might be available on the market
off the shelf right now.
And that improvement will continue.
So I think cost is not a real issue.
Second one is aesthetics.
I don't think that's a real issue either.
- Beauty is in the eye of the beholder.
You can make LIDAR sexy again.
- I think you're exactly right.
I think it is sexy.
Like honestly, I think form is the function.
- Well, okay.
You know, I was actually,
somebody brought this up to me.
I mean, all forms of LIDAR,
even like the ones that are like big,
you can make look, I mean, you can make look beautiful.
Like there's no sense in which
you can't integrate it into design.
Like there's all kinds of awesome designs.
I don't think small and humble is beautiful.
It could be like, you know, brutalism
or like it could be like harsh corners.
I mean, like I said, like hot rods.
Like I don't necessarily like,
like, oh man, I'm gonna start so much controversy with this.
I don't like Porsches.
Okay.
The Porsche 911, like everyone says,
oh, it's the most beautiful.
No, no.
It's like a baby car.
It doesn't make any sense.
But everyone, it's beauty is in the eye of the beholder.
You're already looking at me like,
what is this kid talking about?
- I'm happy to talk about-
- You're digging your own hole.
- The form and function and my take on the beauty
of the hardware that we put on our vehicles.
You know, I will not comment on your Porsche monologue.
- Okay.
All right.
So, but aesthetics, fine.
But there's an underlying like philosophical question
behind the kind of LIDAR question is like,
how much of the problem can be solved
with computer vision, with machine learning?
So I think without sort of disagreements and so on,
it's nice to put it on the spectrum
because Waymo's doing a lot of machine learning as well.
It's interesting to think how much of driving,
if we look at five years, 10 years, 50 years down the road,
what can be learned in almost more and more
and more end to end way.
If we look at what Tesla is doing
as a machine learning problem,
they're doing a multitask learning thing
where it's just, they break up driving
into a bunch of learning tasks
and they have one single neural network
and they're just collecting huge amounts of data
that's training that.
I've recently hung out with George Hotz.
I don't know if you know George.
(laughs)
I love him so much.
He's just an entertaining human being.
We were off mic talking about Hunter S. Thompson.
He's the Hunter S. Thompson of the time I was driving.
Okay.
So he, I didn't realize this with Kama AI,
but they're like really trying to do end to end.
Like looking at the machine learning problem,
they're really not doing multitask learning,
but it's computing the drivable area
as a machine learning task
and hoping that like down the line,
this level two system, this driver assistance
will eventually lead to allowing you
to have a fully autonomous vehicle.
Okay.
There's an underlying deep philosophical question there,
technical question of how much of driving can be learned.
So LiDAR is an effective tool today
for actually deploying a successful service in Phoenix,
right, that's safe, that's reliable, et cetera, et cetera.
But the question,
and I'm not saying you can't do machine learning on LiDAR,
but the question is that like how much of driving
can be learned eventually?
Can we do fully autonomous that's learned?
- Yeah.
You know, learning is all over the place
and plays a key role in every part of our system.
I, as you said, I would decouple the sensing modalities
from the ML and the software parts of it.
LiDAR, radar, cameras, it's all machine learning.
All of the object detection classification, of course,
like that's what these modern deep nets
and con nets are very good at.
You feed them raw data, massive amounts of raw data.
And that's actually what our custom build LiDARs
and radars are really good at.
And radars, they don't just give you point estimates
of objects in space,
they give you raw, like physical observations.
And then you take all of that raw information,
you know, there's colors of the pixels,
whether it's LiDARs returns,
it's some auxiliary information,
it's not just distance, right?
And angle and distance is much richer information
that you get from those returns,
plus really rich information from the radars.
You fuse it all together and you feed it
into those massive ML models that then, you know,
lead to the best results in terms of, you know,
object detection, classification, you know, state estimation.
- So there's a, sorry to interrupt, but there is a fusion.
I mean, that's something that people didn't do
for a very long time,
which is like at the sensor fusion level, I guess,
like early on fusing the information together,
whether so that the sensory information
that the vehicle receives from the different modalities
or even from different cameras is combined
before it is fed into the machine learning models.
- Yeah, so I think this is one of the trends.
You're seeing more of that, you mentioned end-to-end,
there's different interpretation of end-to-end.
There is kind of the purest interpretation
of I'm gonna like have one model
that goes from raw sensor data to like, you know,
steering torque and, you know, gas brakes.
That, you know, that's too much.
I don't think that's the right way to do it.
There's more, you know, smaller versions of end-to-end
where you're kind of doing more end-to-end learning
or core training or deep propagation
of kind of signals back and forth
across the different stages of your system.
§13Transcript · 1:45:50–1:53:58
There's, you know, really good ways.
It gets into some fairly complex design choices
where on one hand you want modularity
and the composability of your system.
But on the other hand,
you don't wanna create interfaces that are too narrow
or too brittle, too engineered,
where you're giving up on the generality of a solution
or you're unable to properly propagate signal,
you know, reach signal forward and losses and, you know,
back so you can optimize the whole system jointly.
So I would decouple.
And I guess what you're seeing in terms of the fusion
of the sensing data from different modalities,
as well as kind of fusion in the temporal level,
going more from, you know, frame by frame,
where, you know, you would have one net
that would do frame by frame detection in camera.
And then, you know, something that does frame by frame
and lighter and then radar,
and then you fuse it, you know,
in a weaker engineered way later.
Like the field over the last decade has been evolving
in more kind of joint fusion, more end-to-end models
that are solving some of these tasks, you know, jointly.
And there's tremendous power in that.
And, you know, that's the progression
that kind of our stack has been on as well.
Now, to your, you know,
so I would decouple the sensing
and how that information is used
from the role of ML in the entire stack.
And, you know, I guess it's,
there's trade-offs in modularity
and how do you inject inductive bias into your system?
Right, this is, there's tremendous power
in being able to do that.
So, you know, we have, there's no part of our system
that is not heavily, that does not heavily, you know,
leverage data-driven development or, you know,
state-of-the-art ML.
But there's mapping, there's a simulator,
or there's perception, you know, object level,
you know, perception, whether it's semantic understanding,
prediction, decision-making, you know, so forth and so on.
It's, and of course, object detection and classification,
like you're finding pedestrians and cars and cyclists
and, you know, cones and signs and vegetation
and being very good at estimating
kind of detection classification and state estimation,
there's just stable stakes.
Like that's step zero of this whole stack.
You can be incredibly good at that,
whether you use cameras or light as a radar,
but that's just, you know, that's stable stakes.
That's just step zero.
Beyond that, you get into the really interesting challenges
of semantic understanding, the perception level.
You get into scene level reasoning.
You get into very deep problems
that have to do with prediction and joint prediction
and interaction, so on and so on,
between all of the actors in the environment,
pedestrians, cyclists, other cars,
and you get into decision-making, right?
So how do you build a lot of systems?
So we leverage ML very heavily in all of these components.
I do believe that the best results you achieve
by kind of using a hybrid approach
and having different types of ML,
having different models with different degrees
of inductive bias that you can have,
and combining kind of model-free approaches
with some model-based approaches
and some rule-based, physics-based systems.
So, you know, one example I can give you is traffic lights.
There's a problem of the detection of traffic light state,
and obviously that's a great problem
for computer vision confidence.
That's their bread and butter, right?
That's how you build that.
But then the interpretation of a traffic light,
that you're gonna need to learn that, right?
Red, you don't need to build some complex ML model
that infers with some precision and recall
that red means stop.
It's a very clear engineered signal
with very clear semantics, right?
So you wanna induce that bias.
Like how you induce that bias and that,
whether it's a constraint or a cost function in your stack,
but it is important to be able to inject
that clear semantic signal into your stack.
And that's what we do.
But then the question of like,
and that's when you apply it to yourself,
when you are making decisions,
whether you wanna stop for a red light or not.
But if you think about how other people treat traffic lights,
we're back to the ML version of that.
As you know, they're supposed to stop for a red light,
but that doesn't mean they will.
So then you're back in the like very heavy ML domain
where you're picking up on like very subtle keys about,
you know, that have to do with the behavior of objects
and pedestrians, cyclists, cars,
and the whole thing, you know,
entire configuration of the scene
that allow you to make accurate predictions
on whether they will in fact stop or run a red light.
- So it sounds like already for Waymo,
like machine learning is a huge part of the stack.
So it's a huge part of like, not just,
so obviously the first level zero or whatever you said,
which is like just the object detection
and things that, you know,
with know that machine learning can do,
but also starting to do prediction behavior and so on
to model what other parties in the scene,
entities in the scene are gonna do.
So machine learning is more and more
playing a role in that as well.
- Of course, absolutely.
I think we've been going back to the earliest days,
like, you know, the DARPA Urban,
the DARPA Grand Challenge,
and team was leveraging, you know, machine learning.
I was like pre, you know, ImageNet,
and it was a very different type of ML,
but, and I think actually it was before my time,
but the Stanford team on during the Grand Challenge
had a very interesting machine learned system
that would, you know, use lighter and camera,
we've been driving in the desert,
and it, we had built the model
where it would kind of extend the range
of free space reasoning.
So we get a clear signal from lighter,
and then it had a model that said,
"Hey, like this stuff in camera
kind of sort of looks like this stuff in lighter.
And I know this stuff that I've seen in lighter,
I'm very confident that it's free space.
So let me extend that free space zone
into the camera range
that would allow the vehicle to drive faster."
And then we've been building on top of that
and kind of staying and pushing the state of the art in ML,
in all kinds of different ML over the years.
And in fact, from the earliest days,
I think, you know, 2010 is probably the year
where Google, maybe 2011 probably,
got pretty heavily involved in machine learning,
kind of deep nuts.
And at that time, it was probably the only company
that was very heavily investing in kind of state of the art ML
and self-driving cars, right?
And they go hand in hand.
And we've been on that journey ever since.
We're doing, pushing a lot of these areas
in terms of research at Waymo,
and we collaborate very heavily
with the researchers in Alphabet.
And I call all kinds of ML,
supervised ML, unsupervised ML,
published some interesting research papers in the space,
especially recently.
It's just super active. - Super active learning as well.
- Yeah, so super, super active.
And of course there's kind of the more mature stuff
like, you know, ConvNets for object detection,
but there's some really interesting,
really active work that's happening
and kind of more, you know, and bigger models
and models that have more structure to them,
you know, not just large bitmaps
and reason about temporal sequences.
And some of the interesting breakthroughs
that we've seen in language models, right?
You know, transformers, you know, GPT-3 inference.
There's some really interesting applications
of some of the core breakthroughs
to those problems of, you know, behavior prediction,
as well as, you know, decision-making and planning, right?
You can think about it, kind of the behavior,
how, you know, the path, the trajectories,
the how people drive,
and they have kind of a share
a lot of the fundamental structure, you know, this problem.
There's, you know, sequential nature.
There's a lot of structure in this representation.
There is a strong locality, kind of like in sentences,
you know, words that follow each other,
they're strongly connected,
but there are also kind of larger contexts
that doesn't have that locality.
And you also see that in driving, right?
What, you know, is happening in the scene as a whole
has very strong implications on, you know,
the kind of the next step in that sequence
§14Transcript · 1:54:00–2:02:41
where whether you're predicting
what other people are going to do,
whether you're making your own decisions,
or whether in the simulator,
you're building generative models of, you know,
humans walking, cyclists riding, and other cars driving.
- Oh, that's all really fascinating.
Like how it's fascinating to think that
transformer models and all the breakthroughs in language
and NLP that might be applicable to like driving
at the higher level, at the behavior level.
That's kind of fascinating.
Let me ask about pesky little creatures
called pedestrians and cyclists.
They seem, so humans are a problem.
If we can get rid of them, I would.
But unfortunately, they're also a source of joy
and love and beauty, so let's keep them around.
- They're also our customers.
- Oh, for your perspective, yes, yes, for sure.
They're a source of money, very good.
But I don't even know where I was going.
Oh yes, pedestrians and cyclists.
I, you know, they're a fascinating injection
into the system of uncertainty,
of like a game theoretic dance of what to do.
And also they have perceptions of their own
and they can tweet about your product.
So you don't want to run them over.
- From that perspective, I mean, I don't know,
I'm joking a lot, but I think in seriousness,
like, you know, pedestrians are a complicated,
computer vision problem, a complicated behavioral problem.
Is there something interesting you could say
about what you've learned
from a machine learning perspective,
from also an autonomous vehicle and a product perspective
about just interacting with the humans in this world?
- Yeah, just, you know, to state on record,
we care deeply about the safety of pedestrians,
you know, even the ones that don't have Twitter accounts.
- Thank you.
(laughs)
All right, all right, cool.
Not me.
But yes, I'm glad somebody does.
Okay.
- But you know, in all seriousness,
safety of vulnerable road users,
pedestrians or cyclists is one of our highest priorities.
We do a tremendous amount of testing and validation
and put a very significant emphasis
on the capabilities of our systems
that have to do with safety
around those unprotected vulnerable road users.
You know, cars, as we discussed earlier in Phoenix,
we have completely empty cars,
completely driverless cars,
driving in this very large area.
And you know, some people use them to go to school,
so they will drive through school zones, right?
So kids are kind of the very special class
of those vulnerable road users, right?
You wanna be super, super safe
and super, super cautious around those.
So we take it very, very, very seriously.
And you know, what does it take to be good at it?
An incredible amount of performance
across your whole stack.
You know, it starts with hardware.
And again, you wanna use all sensing modalities
available to you.
Imagine driving on a residential road at night
and kind of making a turn
and you don't have headlights covering some part
of the space and like, you know, a kid might run out.
And you know, lighters are amazing at that.
They see just as well in complete darkness
as they do during the day, right?
So just again, it gives you that extra,
you know, margin in terms of capability
and performance and safety and quality.
And in fact, we oftentimes, in these kinds of situations,
we have our system detect something,
in some cases even earlier than our trained operators
in the car might do, right?
Especially in conditions like very dark nights.
So it starts with sensing.
Then, you know, perception has to be incredibly good.
And you have to be very, very good
at kind of detecting pedestrians
in all kinds of situations
and all kinds of environments,
including people in weird poses,
people kind of running around
and being partially occluded.
So, you know, that's step number one.
Then you have to have very high accuracy
and very low latency in terms of your reactions
to what these actors might do, right?
And we've put a tremendous amount of engineering
and tremendous amount of validation
in to make sure our system performs properly.
And oftentimes it does require a very strong reaction
to do the safe thing.
And we actually see a lot of cases like that.
It's the long tail of really rare,
you know, really crazy events
that contribute to the safety around pedestrians.
Like one example that comes to mind
that we actually got happened in Phoenix,
where we were driving along
and I think it was a 45 mile per hour road.
So you have pretty high speed traffic
and there was a sidewalk next to it.
And there was a cyclist on the sidewalk.
And as we were in the right lane,
right next to the sidewalk, it was a multi-lane road.
So as we got close to the cyclist on the sidewalk,
it was a woman, she tripped and fell,
just fell right into the path of our vehicle.
And our car, you know, this was actually with a test driver.
Our test drivers did exactly the right thing.
They kind of reacted and came to a stop.
It requires both very strong steering
and strong application of the brake.
And then we simulated what our system would have done
in that situation and it did exactly the same thing.
And that speaks to all of those components
of really good state estimation and tracking.
And like imagine a person on a bike
and they're falling over
and they're doing that right in front of you.
So you have to be really like, things are changing.
The appearance of that whole thing is changing.
And a person goes one way, they're falling on the road,
they're being flat on the ground in front of you.
The bike goes flying the other direction.
Like the two objects that used to be one
are now are splitting apart.
And the car has to like detect all of that.
Like milliseconds matter.
And it's not good enough to just brake.
You have to like steer and brake
and there's traffic around you.
So like it all has to come together.
And it was really great to see in this case
and other cases like that,
that we're actually seeing in the wild
that our system is performing exactly the way
that we would have liked
and is able to avoid collisions like this.
- It's such an exciting space for robotics.
Like in that split second
to make decisions of life and death.
I don't know.
The stakes are high in a sense,
but it's also beautiful that
for somebody who loves artificial intelligence,
the possibility that an AI system
might be able to save a human life.
That's kind of exciting as a problem, like to wake up.
It's terrifying probably for an engineer to wake up
and to think about, but it's also exciting
'cause it's in your hands.
Let me try to ask a question
that's often brought up about autonomous vehicles.
And it might be fun to see
if you have anything interesting to say,
which is about the trolley problem.
So a trolley problem is a interesting philosophical construct
that highlights, and there's many others like it,
of the difficult ethical decisions
that we humans have before us
in this complicated world.
So specifically is the choice between
if you were forced to choose to kill a group X of people
versus a group Y of people, like one person.
If you did nothing, you would kill one person,
but if you would kill five people,
and if you decide to swerve out of the way,
you would only kill one person.
Do you do nothing or you choose to do something?
And you can construct all kinds of sort of
ethical experiments of this kind that,
I think at least on a positive note,
inspire you to think about, like introspect
what are the physics of our morality?
And there's usually not good answers there.
I think people love it
'cause it's just an exciting thing to think about.
I think people who build autonomous vehicles
usually roll their eyes
because this is not,
this one as constructed,
this like literally never comes up in reality.
You never have to choose between killing
(laughs)
one or like one of two groups of people.
But I wonder if you can speak to,
§15Transcript · 2:02:45–2:10:40
is there something interesting
to you as an engineer of autonomous vehicles
that's within the trolley problem,
or maybe more generally,
are there difficult ethical decisions
that you find that a algorithm must make?
- On the specific version of the trolley problem,
which one would you do?
If you're driving?
- The question itself is a profound question
because we humans ourselves cannot answer it.
And that's the very point.
I would kill both.
(laughs)
- Humans, I think you're exactly right
in that humans are not particularly good.
I think the kind of phrase is like,
what would a computer do?
But humans are not very good.
And actually oftentimes I think that freezing
and kind of not doing anything
because like you've taken a few extra milliseconds
to just process,
and then you end up like doing the worst
of possible outcomes, right?
So I do think that as you've pointed out,
it can be a bit of a distraction
and it can be a bit of a kind of a red herring.
I think it's an interesting discussion
in the realm of philosophy, right?
But in terms of what,
how that affects the actual engineering
and deployment of self-driving vehicles,
it's not how you go about building a system, right?
We've talked about how you engineer a system,
how you go about evaluating the different components
and the safety of the entire thing.
How do you kind of inject the various model-based,
safety-based,
or I'm gonna say like,
yes, you reason at parts of the system,
you reason about the probability of a collision,
the severity of that collision, right?
And that is incorporated,
and you have to properly reason about the uncertainty
that flows through the system, right?
So those factors definitely play a role
in how the cars then behave,
but they tend to be more of like the emergent behavior.
And what you see, like you're absolutely right,
that these clear theoretical problems that they,
you don't have a car that in system,
and really kind of being back to our previous discussion,
of like, which one do you choose?
Well, oftentimes you made a mistake earlier,
like you shouldn't be in that situation
in the first place, right?
And in reality, the system comes up.
If you build a very good, safe and capable driver,
you have enough clues in the environment
that you drive defensively,
so you don't put yourself in that situation, right?
And again, if you go back to that analogy
of precision and recall,
like, okay, you can make a very hard trade-off,
but neither answer is really good.
But what instead you focus on
is kind of moving the whole curve up,
and then you focus on building the right capability
and the right defensive driving
so that you don't put yourself in a situation like this.
- I don't know if you have a good answer for this,
but people love it when I ask this question about books.
Are there books in your life
that you've enjoyed, philosophical, fiction, technical,
that had a big impact on you as an engineer
or as a human being?
You know, everything from science fiction
to a favorite textbook.
Is there three books that stand out that you can think of?
- Three books.
So I would, you know, that impacted me,
I would say,
this one is, you probably know it well,
but not generally well known,
I think in the US or kind of internationally,
"The Master and Margarita."
It's one of, actually, my favorite books.
It is by a Russian,
it's a novel by Russian author Mikhail Bulgakov.
And it's just, it's a great book.
And it's one of those books
that you can like reread your entire life,
and it's very accessible.
You can read it as a kid.
And like, it's, you know, the plot is interesting.
It's, you know, the devil, you know,
visiting the Soviet Union.
But it, like, you read it,
reread it at different stages of your life,
and you enjoy it for different, very different reasons.
And you keep finding like deeper and deeper meaning.
And, you know, it kind of affected,
you know, it had a,
definitely had an imprint on me,
mostly from the,
probably kind of the cultural stylistic aspect.
Like it makes you,
it's one of those books that, you know,
is good and makes you think,
but also has like this really, you know,
silly, quirky, dark sense of, you know, humor.
- It captures the Russian soul
more than perhaps many other books.
On that like slight note, just out of curiosity,
one of the saddest things is I've read that book
in English.
Did you by chance read it in English or in Russian?
- In Russian, only in Russian.
And I actually, that is a question I had.
Kind of posed to myself every once in a while.
Like I wonder how well it translates,
if it translates at all.
And there's the language aspect of it,
and then there's the cultural aspect.
So I, and actually I'm not sure if, you know,
either of those would work well in English.
- Now I forget their names,
but so when the COVID lifts a little bit,
I'm traveling to Paris
for several reasons.
One is just, I've never been to Paris.
I want to go to Paris,
but there's the most famous translators
of Dostoevsky, Tolstoy, of most of Russian literature
live there.
There's a couple, they're famous, a man and a woman.
And I'm going to sort of have a series of conversations
with them.
And in preparation for that,
I'm starting to read Dostoevsky in Russian.
So I'm really embarrassed to say that I've read this,
everything I've read in Russian literature
of like serious depth has been in English,
even though I can also read, I mean, obviously in Russian,
but for some reason it seemed,
in the optimization of life,
it seemed the improper decision to do it,
to read in Russian.
Like, you know, like I don't need to opt,
I need to think in English, not in Russian,
but now I'm changing my mind on that.
And so the question of how well it translate
is a really fundamental one.
Like it, even with Dostoevsky.
So from what I understand, Dostoevsky translates easier.
Others don't as much.
Obviously the poetry doesn't translate as well.
I'm also the music, big fan of Vladimir Vysotsky.
He doesn't obviously translate well.
People have tried.
But mastermind, I don't know.
I don't know about that one.
I just know it in English, you know,
it's fun as hell in English.
So, but it's a curious question
and I wanna study it rigorously
from both the machine learning aspect
and also because I want to do a couple of interviews
in Russia that I'm still unsure
of how to properly conduct an interview
across a language barrier.
It's a fascinating question that ultimately communicates
to an American audience.
There's a few Russian people that I think
are truly special human beings.
And I feel like I sometimes encounter this
with some incredible scientists
and maybe you encounter this as well
at some point in your life that it feels like
because of the language barrier,
their ideas are lost to history.
It's a sad thing.
I think about like Chinese scientists or even authors
that like, that we don't in English speaking world
don't get to appreciate some like the depth of the culture
because it's lost in translation.
And I feel like I would love to show that to the world.
Like I'm just some idiot, but because I have this,
like at least some semblance of skill in speaking Russian,
I feel like, and I know how to record stuff
on a video camera.
I feel like I wanna catch like Grigori Pearlman,
who's a mathematician.
I'm not sure if you're familiar with him.
I wanna talk to him, like he's a fascinating mind
and to bring him to a wider audience in English speaking,
it'll be fascinating.
But that requires to be rigorous about this question
of how well Bulgakov translates.
§16Transcript · 2:10:44–2:19:33
I mean, I know it's a silly concept,
but it's a fundamental one because how do you translate?
And that's the thing that Google Translate is also facing
as a more machine learning problem.
But I wonder as a more bigger problem for AI,
how do we capture the magic that's there in the language?
- I think that's really interesting,
really challenging problem.
If you do read it, Master and Margarita in English,
sorry, in Russian, I'd be curious to get your opinion.
And I think part of it is language,
but part of it's just centuries of culture
that the cultures are different.
So it's hard to connect that.
- Okay, so that was my first one, right?
You had two more.
The second one, I would probably pick the science fiction
by the Strogowski brothers.
I know it's up there with Isaac Asimov
and Ray Bradbury and company.
The Strogowski brothers kind of appealed more to me.
I think it made more of an impression on me growing up.
- Can you, I apologize if I'm showing my complete ignorance.
I'm so weak on sci-fi.
What did they write?
- Oh, "Roadside Picnic."
"Hard to Be a God."
"Beetle in an Ant Hill."
"Monday Starts on Saturday."
Like it's not just science fiction.
It's also like has very interesting,
interpersonal and societal questions.
And some of the language is just completely hilarious.
- (speaking in foreign language)
- That's the one. - That's right.
- Oh, interesting.
"Monday Starts on Saturday."
So I need to read, okay.
- Oh boy.
You put that in the category of science fiction?
- That one is, I mean, this was more of a silly,
humorous work.
I mean, there is kind of--
- But it's profound too, right?
- Science fiction, right, is about this research institute.
It has deep parallels to like serious research,
but the setting of course is that they're working on magic.
And there's a lot of, so I, and that's their style, right?
They go, and other books are very different, right?
"Hard to Be a God," right?
It's about kind of this higher society being injected
into this primitive world and how they operate there.
Some of the very deep ethical questions there, right?
And like they've got this full spectrum.
Some is more about kind of more adventure style.
But like I enjoy all of their books.
There's probably a couple, actually one I think
that they considered their most important work.
I think it's "The Snail on a Hill."
I don't know exactly how it translates.
I tried reading a couple of times.
I still don't get it, but everything else I fully enjoyed.
And like for one of my birthdays as a kid,
I got like their entire collection,
like occupied a giant shelf in my room.
And then I'll like over the holidays, I just like,
my parents couldn't drag me out of the room
and I read the whole thing cover to cover
and I really enjoyed it.
- And that's the one more, for the third one,
maybe a little bit darker, but you know,
comes to mind is Orwell's "1984."
And you asked what made an impression on me
and the books that people should read.
That one I think falls in the category of both.
Definitely it's one of those books that you read
and you just kind of put it down
and you stay in space for a while.
That kind of work.
I think there's lessons there people should not ignore.
And nowadays with everything that's happening in the world,
I can't help it, but have my mind jump to some parallels
with what Orwell described.
And like there's this whole concept of double think
and ignoring logic and holding completely contradictory
opinions in your mind and not have that not bother you
and sticking to the party line at all costs.
Like there's something there.
- If anything, 2020 has taught me,
and I'm a huge fan of Animal Farm,
which is a kind of friendly, is a friend of "1984"
by Orwell.
It's kind of another thought experiment
of how our society may go in directions
that we wouldn't like it to go.
But if anything that's been kind of heartbreaking
to an optimist about 2020 is that society's kind of fragile.
Like we have this, this is a special little experiment
we have going on and it's not unbreakable.
Like we should be careful to like preserve
whatever the special thing we have going on.
I mean, I think "1984" and these books, "Brave New World",
they're helpful in thinking like stuff can go wrong
in non-obvious ways.
And it's like, it's up to us to preserve it.
And it's like, it's a responsibility.
It's been weighing heavy on me.
'Cause like, for some reason,
like more than my mom follows me on Twitter.
And I feel like I have like now somehow responsibility
to this world.
And it dawned on me that like me and millions of others
had like the little ants that maintain this little colony.
Right?
So we have a responsibility not to be,
I don't know what the right analogy is,
but put a flamethrower to the place.
We wanna not do that.
And there's interesting, complicated ways of doing that
as "1984" shows.
It could be through bureaucracy,
it could be through incompetence,
it could be through misinformation,
it could be through division and toxicity.
I'm a huge believer in like that love
will be the somehow the solution.
So love and robots.
(laughing)
- Love and robots, yeah.
I think you're exactly right.
Unfortunately, I think it's less of a flamethrower
type of an answer.
I think it's more of a,
in many cases can be more of a slow boil.
And that's the danger.
- Let me ask, it's a fun thing to make
a world-class roboticist, engineer and leader
uncomfortable with a ridiculous question about life.
What is the meaning of life, Dmitry,
from a robotics and a human perspective?
- You only have a couple minutes,
or one minute to answer.
So.
(laughing)
- I don't know if that makes it more difficult
or easier, actually.
Yeah, you know, I'm very tempted to quote
one of the stories by Isaac Asimov, actually.
Actually titled, appropriately titled,
"The Last Question," a short story,
where the plot is that humans build this supercomputer,
you know, this AI intelligence,
and once it gets powerful enough,
they pose this question to it.
How can the entropy in the universe be reduced?
Right, so the computer replies,
hang on, as of yet, insufficient information
to give a meaningful answer, right?
And then, you know, thousands of years go by
and they keep posing the same question.
The computer gets more and more powerful
and keeps giving the same answer.
As of yet, insufficient information
to give a meaningful answer,
or something along those lines, right?
And then, you know, it keeps happening
and happening, you fast forward,
like millions of years into the future
and billions of years, and like at some point,
it's just the only entity in the universe.
It's like it's absorbed all humanity
and all knowledge in the universe,
and it keeps posing the same question to itself.
And, you know, finally, it gets to the point
where it is able to answer that question.
But of course, at that point, you know,
there's the heat death of the universe has occurred,
and that's the only entity,
and there's nobody else to provide that answer to.
So the only thing it can do is to, you know,
answer it by demonstration.
So it like, you know, recreates the Big Bang, right?
And resets the clock, right?
(both laughing)
But I can try to give kind of a different version
of the answer, you know, maybe not on the behalf
of all humanity.
I think that might be a little presumptuous
for me to speak about the meaning of life
on the behalf of all humans,
but at least, you know, personally, it changes, right?
I think if you think about kind of what gives,
you know, you and your life meaning and purpose
and kind of what drives you,
it seems to change over time, right?
And the lifespan of, you know, kind of your existence,
you know, when you just enter this world, right?
It's all about kind of new experiences, right?
You get like new smells, new sounds, new emotions, right?
And like, that's what's driving you, right?
§17Transcript · 2:19:35–2:23:03
You're experiencing new, amazing things, right?
And that's magical, right?
That's pretty awesome, right?
That gives you kind of a meaning.
Then, you know, you get a little bit older,
you start more intentionally learning about things, right?
I guess actually before you start intentionally learning,
probably fun.
Fun is a thing that gives you kind of meaning and purpose
and the thing you optimize for, right?
And like fun is good.
Then you get, you know, start learning.
And I guess that this joy of comprehension
and discovery is another thing that, you know,
gives you meaning and purpose and drives you, right?
Then, you know, you learn enough stuff
and you wanna give some of it back, right?
And so impact and contributions back to technology
or society, people, you know, local or more globally
becomes a new thing that drives a lot of kind
of your behavior and is something that gives you purpose
and that you derive positive feedback from, right?
You know, then you go and so on and so forth.
You go through various stages of life.
If you have kids, like that definitely changes
your perspective on things.
You know, I have three that definitely flips some bits
in your head in terms of kind of what you care about
and what you optimize for and, you know, what matters,
what doesn't matter, right?
So, you know, and so on and so forth, right?
And it seems to me that, you know, it's all of those things.
And as you kind of go through life, you know,
you want these to be additive, right?
New experiences, fun, learning, impact.
Like you wanna, you know, be accumulating.
I don't wanna, you know, stop having fun
or experiencing new things.
And I think it's important that, you know,
just kind of becomes additive as opposed
to a replacement or subtraction.
But, you know, those few as far as I got,
but, you know, ask me in a few years,
I might have one or two more to add to the list.
- And before you know it, time is up,
just like it is for this conversation.
But hopefully it was a fun ride.
It was a huge honor to meet you.
As you know, I've been a fan of yours
and a fan of Google self-driving car
and Waymo for a long time.
I can't wait.
I mean, it's one of the most exciting.
If we look back in the 21st century,
I truly believe it will be one of the most exciting things
we descendants of apes have created on this earth.
So I'm a huge fan and I can't wait to see what you do next.
Thanks so much for talking to me.
- Thanks.
Thanks for having me.
And it's also a huge fan.
Doing work on this and I really enjoy this.
Thank you.
- Thanks for listening to this conversation
with Dmitry Dolgov.
And thank you to our sponsors.
Trial Labs, a company that helps businesses
apply machine learning to solve real world problems.
Blinkist, an app I use for reading
through summaries of books.
BetterHelp, online therapy with a licensed professional
and Cash App, the app I use to send money to friends.
Please check out these sponsors in the description
to get a discount and to support this podcast.
If you enjoy this thing, subscribe on YouTube,
review it with 5 Stars on Apple Podcasts,
follow on Spotify, support on Patreon
or connect with me on Twitter @LexFriedman.
And now let me leave you with some words from Isaac Asimov.
Science can amuse and fascinate us all,
but it is engineering that changes the world.
Thank you for listening and hope to see you next time.
(upbeat music)
(upbeat music)