← Lex Fridman

Brian Kernighan: UNIX, C, AWK, AMPL, and Go Programming

第109期2020-07-18English

嘉宾:Brian Kernighan

收听本集00:00 / 1:43:37
阅读设置
按你的习惯阅读
正文字号
下载播客精读
把这一篇,带走慢慢读

精读待补充

下载功能还未开放

§01Transcript · 00:00–08:40

00:00The following is a conversation with Brian Kernighan,

00:03a professor of computer science at Princeton University.

00:07He was a key figure in the computer science community

00:10in the early Unix days, alongside Unix creators,

00:13Ken Thompson and Dennis Ritchie.

00:16He co-authored the C programming language

00:18with Dennis Ritchie, the creator of C,

00:21and has written a lot of books on programming,

00:24computers and life, including "The Practice of Programming,"

00:28"The Go Programming Language,"

00:30and his latest, "Unix, a History and a Memoir."

00:34He co-created AWK, the text processing language

00:36used by Linux folks like myself.

00:39He co-designed Ample, an algebraic modeling language

00:43that I personally love and have used a lot in my life

00:47for large-scale optimization.

00:49I think I can keep going for a long time

00:51with his creations and accomplishments,

00:54which is funny because given all that,

00:56he's one of the most humble and kind people

00:59I've spoken to on this podcast.

01:01Quick summary of the ads, two new sponsors,

01:04the amazing self-cooling 8Sleep mattress

01:09and Raycon earbuds.

01:13Please consider supporting the podcast

01:15by going to 8sleep.com/lex

01:19and going to buyraycon.com/lex.

01:23Click the links, buy the stuff.

01:25It really is the best way to support this podcast

01:27and the journey I'm on.

01:29If you enjoy this thing, subscribe on YouTube,

01:32review it with Firestarz and Apple Podcasts,

01:34support it on Patreon,

01:35or connect with me on Twitter @LexFriedman.

01:38As usual, I'll do a few minutes of ads now

01:41and never any ads in the middle

01:43that can break the flow of the conversation.

01:45This show is sponsored by 8Sleep

01:49and its incredible Pod Pro mattress

01:51that you can check out at 8sleep.com/lex

01:54to get $200 off.

01:57The mattress controls temperature with an app

02:00and can cool down to as low as 55 degrees.

02:03Research shows the temperature has a big impact

02:06on the quality of our sleep.

02:09Anecdotally, it's been a game changer for me.

02:11I love it.

02:12The Pod Pro is packed with sensors

02:14that track heart rate, heart rate variability

02:17and respiratory rate,

02:18showing it all on their app once you wake up.

02:21Plus, if you have a partner,

02:23you can control the temperature of each side of the bed.

02:26I don't happen to have one,

02:28but the 8Sleep app reminds me

02:30that I should probably get on that.

02:32So ladies, if a temperature controlled mattress

02:34isn't a good reason to apply,

02:36I don't know what is.

02:38The app's health metrics are amazing,

02:41but the cooling alone is honestly worth the money.

02:44As some of you know, I don't always sleep,

02:47but when I do, I choose the 8Sleep Pod Pro mattress.

02:51Check it out at 8sleep.com/lex to get $200 off.

02:56This show is also sponsored by Raycon earbuds.

03:01Get them at buyraycon.com/lex.

03:06They've quickly become my main method

03:07of listening to podcasts, audio books, and music.

03:10When I run, do the pushups and pull-ups

03:13that I've begun to hate at this point,

03:15or just living life.

03:17In fact, I often listen to brown noise with these

03:20when I'm thinking deeply about something.

03:22It helps me focus the mind.

03:24They're super comfortable, pair easily,

03:26great sound, great bass, six hours of playtime.

03:30In fact, for fun, I have one of the earbuds in now,

03:33and I'm listening to Europa by Santana,

03:36probably one of my favorite guitar songs.

03:39It kind of makes me feel like I'm in a music video.

03:41So they told me to say that a bunch of celebrities

03:44use these, like Snoop Dogg, Melissa Etheridge, and Cardi B.

03:50I don't even know who Cardi B is,

03:52but her earbud game is on point.

03:55To mention celebrities I actually care about,

03:58I'm sure if Richard Feynman was still with us,

04:01he'd be listening to the Joe Rogan experience

04:03with Raycon earbuds.

04:06Get them at buyraycon.com/lex.

04:09It's how they know I sent you,

04:11and increases the chance that he'll support this podcast

04:13in the future.

04:14So for all of the sponsors, click all of the links.

04:17It really helps this podcast.

04:20And now, here's my conversation with Brian Kernighan.

04:24Unix started being developed 50 years ago,

04:28maybe more than 50 years ago.

04:30Can you tell the story, like you describe in your new book,

04:33of how Unix was created?

04:35- Ha, if I can remember that far back,

04:38it was some while ago.

04:39So I think the gist of it is that at Bell Labs in 1969,

04:45there were a group of people who had just finished

04:48working on the Multics project,

04:49which was itself a follow-on to CTSS.

04:54So we can go back sort of an infinite regress in time,

04:57but the CTSS was a very, very, very nice time-sharing system.

05:01It was very nice to use.

05:02I actually used it as that summer I spent

05:04in Cambridge in 1966.

05:06- What was the hardware there?

05:08So what's the operating system?

05:09What's the hardware there?

05:10What's the CTSS look like?

05:12- So CTSS looked like,

05:14kind of like a standard time-sharing system.

05:17Certainly at the time, it was the only time-sharing,

05:18if no-

05:19- Let's go back to the basic.

05:20What's a time-sharing system?

05:22- Okay, in the beginning was the word,

05:23and the word was itself.

05:24- And then there was time-sharing systems.

05:27- Yeah, if we go back into, let's call it the 1950s

05:29and early 1960s, most computing was done

05:33on very big computers, physically big,

05:35although not terribly powerful by today's standards,

05:38that were maintained in very large rooms,

05:42and you used things like punch cards

05:45to write your programs on and talk to them.

05:47So you would take a deck of cards,

05:49write your program on it, send it over a counter,

05:51hand it to an operator, and some while later,

05:54back would come something that said,

05:55oh, you made a mistake, and then you'd recycle.

05:58And so it was very, very slow.

05:59So the idea of time-sharing was that you take

06:02basically that same computer, but connect to it

06:06with something that looked like an electric typewriter.

06:09That could be a long distance away, it could be close,

06:11but fundamentally what the operating system did

06:14was to give each person who was connected to it

06:18and wanting to do something a small slice of time

06:21to do a particular job.

06:24So I might be editing a file, so I would be typing,

06:28and every time I hit a keystroke,

06:29the operating system would wake up and said,

06:30oh, he typed character, let me remember that,

06:33and then it'd go back to doing something else.

06:34So it'd be going around and around a group of people

06:38who were trying to get something done,

06:39giving each a small slice of time,

06:43and giving them each the illusion

06:45that they pretty much had the whole machine to themselves,

06:47and hence time-sharing, that is sharing

06:49the computing time resource of the computer

06:52among a number of people who were doing it.

06:54- Without the individual people being aware

06:56that there's others, in a sense, the illusion,

06:59the feelings that the machine is your own.

07:02- Pretty much that was the idea, yes.

07:04You had, if it were well done, and if it were fast enough,

07:08and other people weren't doing too much,

07:09you did have the illusion

07:11that you had the whole machine to yourself,

07:13and it was very much better than the punch card model.

07:16And so CTSS, the compatible time-sharing system,

07:19was, I think, arguably the first of these.

07:22It was done, I guess, technically in '64,

07:25or something like that.

07:26It ran on an IBM 7094, slightly modified

07:30to have twice as much memory as the norm.

07:32It had two banks of 32K words instead of one.

07:37So, (laughs)

07:38- 32K words, yeah.

07:40- Each word was 36 bits, so call it,

07:42about 150 kilobytes times two.

07:46So, by today's standards, that's down in the noise.

07:49But at the time, that was a lot of memory,

07:51and memory was expensive.

07:53So, CTSS was just a wonderful environment to work on.

07:56It was done by people at MIT,

07:58led by Fernando Corbato, Corby,

08:02who died just earlier this year,

08:04and a bunch of other folks.

08:06And so I spent the summer of '66 working on that.

08:09Had a great time, met a lot of really nice people,

08:12and indirectly knew of people at Bell Labs

08:17who were also working on a follow-on to CTSS

08:22that was called Multics.

08:24So, Multics was meant to be the system

08:26that would do everything that CTSS did,

08:27but do it better for a larger population.

08:30All the usual stuff.

08:31- Now, the actual time-sharing, the scheduling,

08:35how much, what's the algorithm

08:37that performs the scheduling?

08:39What's that look like?

08:39How much magic is there?

08:40What are the metrics?

§02Transcript · 08:42–17:21

08:42How does it all work in the beginning?

08:44- So, the answer is I don't have a clue.

08:46I think the basic idea was nothing more

08:48than who all wants to get something done.

08:50Suppose that things are very quiet

08:52in the middle of the night,

08:53then I get all the time that I want.

08:55Suppose that you and I are contending at high noon

08:58for something like that,

08:59then probably the simplest algorithm is a round-robin,

09:02one that gives you a bit of time, gives me a bit of time.

09:05And then we could adapt to that.

09:07Like, what are you trying to do?

09:08Are you text editing or are you compiling or something?

09:12And then we might adjust the scheduler

09:13according to things like that.

09:15- So, okay, so Multics was trying to just do some of the,

09:18clean it up a little bit.

09:20- Well, it was meant to be much more than that.

09:22So, Multics was the multiplexed information

09:24and computing service,

09:25and it was meant to be a very large thing

09:27that would provide computing utility,

09:29something that where you could actually think of it

09:32as just a plug-in-the-wall service.

09:35Sort of like cloud computing today, same idea,

09:38but 50 odd years earlier.

09:40And so what Multics offered

09:43was a richer operating system environment,

09:46a piece of hardware that was better designed

09:48for doing the kind of sharing of resources,

09:53and presumably lots of other things.

09:55- Do you think people at that time had the dream

09:58of what cloud computing is starting to become now,

10:01which is computing is everywhere,

10:03that you can just plug in almost,

10:05and you never know how the magic works.

10:09You just kind of plug in, add your little computation

10:11that you need to perform, and it does it.

10:13Was that the dream?

10:14- I don't know where that was the dream.

10:16I wasn't part of it at that point.

10:17Remember, I was an intern for a summer.

10:19But my sense is, given that it was over 50 years ago,

10:23yeah, they had that idea

10:24that it was an information utility,

10:26that it was something where

10:27if you had a computing task to do,

10:29you could just go and do it.

10:31Now, I'm betting that they didn't have the same view

10:35of computing for the masses, let's call it,

10:38the idea that your grandmother would be shopping on Amazon.

10:43I don't think that was part of it.

10:45But if your grandmother were a programmer,

10:47it might be very easy for her to go

10:49and use this kind of utility.

10:51- What was your dream of computers at that time?

10:53What did you see as the future of computers?

10:55'Cause you have predicted what computers are today.

10:59- Oh, short answer, absolutely not.

11:01I have no clue.

11:02I'm not sure I had a dream.

11:03It was a dream job in the sense that

11:05I really enjoyed what I was doing.

11:07I was surrounded by really, really nice people.

11:10Cambridge is a very fine city to live in in the summer,

11:12less so in the winter when it snows,

11:14but in the summer, it was a delightful time.

11:16And so I really enjoyed all of that stuff

11:19and I learned things.

11:20And I think the good fortune of being there for summer

11:25led me then to get a summer job at Bell Labs

11:27the following summer.

11:28And that was quite useful for the future.

11:31- So Bell Labs is this magical, legendary place.

11:35So first of all, where is Bell Labs?

11:39And can you start talking about that journey

11:44towards Unix at Bell Labs?

11:46- Yeah, so Bell Labs is physically scattered around,

11:50at the time, scattered around New Jersey.

11:52The primary location was in a town called Murray Hill,

11:54or a location called Murray Hill.

11:56It was actually across the boundary

11:58between two small towns in New Jersey

12:00called New Providence and Berkeley Heights.

12:03Think of it as about 15, 20 miles straight west

12:05of New York City, and therefore about an hour north

12:08of here in Princeton.

12:10And at that time, it had, make up a number,

12:15three or 4,000 people there, many of whom had PhDs

12:17and mostly doing physical sciences,

12:20chemistry, physics, materials kinds of things,

12:24but very strong math and rapidly growing interest

12:29in computing as people realized you could do things

12:31with computers that you might not have been able

12:33to do before.

12:35You could replace labs with computers

12:37that had worked on models of what was going on.

12:40So that was the essence of Bell Labs.

12:44And again, I wasn't a permanent employee there.

12:46That was another internship.

12:47I got lucky in internships.

12:49- I mean, if you could just linger on it a little bit,

12:52what was in the air there?

12:55'Cause some of the, the number of Nobel Prizes,

12:57the number of Turing Awards,

12:58and just legendary computer scientists

13:00that come from there, inventions including,

13:04developments including Unix, it's just, it's unbelievable.

13:07So was there something special about that place?

13:11- Oh, I think there was very definitely something special.

13:14I mentioned the number of people.

13:15It's a very large number of people, very highly skilled,

13:19and working in an environment where there was always

13:21something interesting to work on,

13:23because the goal of Bell Labs,

13:25which was a small part of AT&T,

13:27which provided basically the country's phone service.

13:30The goal of AT&T was to provide service for everybody.

13:33And the goal of Bell Labs was to try and make that service

13:36keep getting better, so improving service.

13:39And that meant doing research on a lot of different things,

13:43physical devices like the transistor,

13:46or fiber optical cables, or microwave systems,

13:50all of these things the labs worked on.

13:53And it was kind of just the beginning of real boom times

13:56in computing as well, 'cause when I was there,

13:59I went there first in '66.

14:01So computing was at that point fairly young,

14:04and so people were discovering that you could do

14:06lots of things with computers.

14:08- So how was Unix born?

14:10- So Multics, in spite of having an enormous number

14:14of really good ideas, lots of good people working on it,

14:16fundamentally didn't live up, at least in the short run,

14:20and I think ultimately really ever,

14:22to its goal of being this information utility.

14:25It was too expensive, and certainly what was promised

14:29was delivered much too late.

14:31And so in roughly the beginning of 1969,

14:34Bell Labs pulled out of the project.

14:37The project at that point had included MIT,

14:41Bell Labs, and General Electric.

14:44General Electric made computers,

14:45so General Electric was the hardware operation.

14:48So Bell Labs, realizing this wasn't going anywhere

14:50on a timescale they cared about, pulled out of the project.

14:54And this left several people with an acquired taste

14:59for really, really nice computing environments,

15:01but no computing environment.

15:03And so they started thinking about what could you do

15:06if you were gonna design a new operating system

15:09that would provide the same kind of comfortable computing

15:12as CTSS had, but also the facilities of something

15:16like Multics sort of brought forward.

15:17And so they did a lot of paper design stuff,

15:21and at the same time, Ken Thompson found

15:23what is characterized as a little-used PDP-7,

15:27where he started to do experiments with file systems,

15:31just how do you store information on a computer

15:33in an efficient way, and then this famous story

15:36that his wife went away to California for three weeks

15:39taking their one-year-old son, and three weeks,

15:43and he sat down and wrote an operating system,

15:45which ultimately became Unix.

15:47So software productivity was good in those days.

15:50- So PDP, what's a PDP-7?

15:52So it's a piece of hardware?

15:53- Yeah, it's a piece of hardware.

15:54It was one of early machines made

15:56by Digital Equipment Corporation, DEC,

15:59and it was a mini-computer, so-called.

16:03It had, I would have to look up the numbers exactly,

16:07but it had a very small amount of memory,

16:09maybe 16K, 16-bit words or something like that,

16:13relatively slow, probably not super expensive,

16:17maybe, again, making this up, I'd have to look it up,

16:19$100,000 or something like that.

16:21- Which is not super expensive in those days, right?

16:24- It was expensive.

16:25It was enough that you and I probably

16:26wouldn't be able to buy one,

16:27but a modest group of people could get together.

16:30But in any case, it came out, if I recall, in 1964,

16:34so by 1969, it was getting a little obsolete,

16:38and that's why it was little-used.

16:41- If you can sort of comment, what do you think

16:43it's like to write an operating system like that?

16:45So that process that Ken went through in three weeks,

16:48because you were, I mean, you're a part of that process.

16:52You contributed a lot to Enix's early development.

16:57So what do you think it takes to do that first step,

17:01that first kind of, from design to reality on the PDP?

17:05- Well, let me correct one thing.

17:07I had nothing to do with it, so I did not write it.

17:10I have never written operating system code.

17:12And so I don't know.

17:16Now, an operating system is simply code,

17:18and this first one wasn't very big,

17:21but it's something that lets you run processes of some,

§03Transcript · 17:24–25:39

17:24lets you execute some kind of code that has been written.

17:27It lets you store information for periods of time

17:30so that it doesn't go away when you turn the power off

17:33or reboot or something like that.

17:35And there's kind of a core set of tools

17:38that are technically not part of an operating system,

17:40but you probably need them.

17:42In this case, Ken wrote an assembler for the PDP-7

17:46that worked.

17:46He did a text editor so that he could actually create text.

17:49He had the file system stuff that he had been working on,

17:52and then the rest of it was just a way to load things,

17:55executable code from the file system into the memory,

17:58give it control, and then recover control

18:01when it was finished or in some other way quit.

18:04- What was the code written in,

18:06primarily the programming language?

18:08Was it in assembly?

18:09- Yeah, PDP-7 assembler that Ken created.

18:12These things were assembly language until probably the,

18:17call it 1973 or '74, something like that.

18:20- Forgive me if it's a dumb question,

18:23but it feels like a daunting task

18:25to write any kind of complex system in assembly.

18:28- Absolutely.

18:29- It feels like impossible to do any kind of

18:32what we think of as software engineering with assembly

18:36'cause to work on a big picture sort of.

18:40- I think it's hard.

18:41It's been a long time since I wrote assembly language.

18:43It is absolutely true that in assembly language,

18:45if you make a mistake, nobody tells you.

18:47There are no training wheels whatsoever.

18:49And so stuff doesn't work.

18:51Now what?

18:51- There's no debuggers.

18:53- Well, there could be debuggers,

18:54but that's the same problem, right?

18:56How do you actually get something

18:58that will help you debug it?

19:00So part of it is an ability to see the big picture.

19:05Now these systems were not big

19:07in the sense that today's pictures are.

19:08So the big picture was in some sense more manageable.

19:11I mean, then realistically, there's an enormous variation

19:15in the capabilities of programmers.

19:17And Ken Thompson, who did that first one,

19:20is kind of the singularity in my experience of programmers

19:24with no disrespect to you or even to me.

19:27He's in several leagues removed.

19:31- I know there's levels.

19:33It's a fascinating thing that there are unique stars

19:37in particular in the programming space

19:39and at a particular time.

19:41The time matters too, the timing

19:42of when that person comes along.

19:44And a wife does have to leave.

19:47There's this weird timing that happens

19:49that and then all of a sudden

19:50something beautiful is created.

19:52I mean, how does it make you feel

19:53that there's a system that was created in three weeks

19:58or maybe you can even say on a whim,

20:01but not really, but of course quickly,

20:03that is now, you could think of most of the computers

20:07in the world run on a Unix-like system.

20:10- Right.

20:11(laughs)

20:12- How do you, if you kind of zoom

20:15from the alien perspective,

20:16if you're just observing Earth,

20:18that all of a sudden these computers took over the world

20:21and they started from this little initial seed of Unix,

20:24how does that make you feel?

20:26- It's quite surprising.

20:27And you asked earlier about prediction.

20:30The answer is no.

20:31There's no way you could predict that kind of evolution.

20:33And I don't know whether it was inevitable

20:37or just a whole sequence of blind luck.

20:39I suspect more of the latter.

20:40And so I look at it and think, gee, that's kind of neat.

20:45(laughs)

20:47I think the real question is what does Ken think about that?

20:51'Cause he's the guy arguably from whom it really came.

20:55Tremendous contributions from Dennis Ritchie

20:57and then others around in that Bell Labs environment.

21:00But if you had to pick a single person, that would be Ken.

21:04- So you've written a new book,

21:06Unix, a history and a memoir.

21:08Are there some memorable human stories,

21:10funny or profound from that time that just kind of stand out?

21:14- Oh, there's a lot of them in a sense.

21:15And again, it's a question of can you resurrect them

21:18in real time? - Memory?

21:19(laughs)

21:20- Memory fails.

21:21But I think part of it was that Bell Labs at the time

21:25was a very special kind of place to work

21:27because there were a lot of interesting people

21:29and the environment was very, very open and free.

21:31It was a very cooperative environment,

21:33very friendly environment.

21:34And so if you had an interesting problem,

21:36you go and talk to somebody

21:37and they might help you with the solution.

21:39And it was a kind of a fun environment too

21:43in which people did strange things

21:46and often tweaking the bureaucracy in one way or another.

21:51- So rebellious in certain kinds of ways.

21:54- In some ways, yeah, absolutely.

21:56I think most people didn't take too kindly

21:58to the bureaucracy and I'm sure the bureaucracy

22:01put up with an enormous amount

22:03that they didn't really want to.

22:05- So maybe to linger on it a little bit,

22:09do you have a sense of what the philosophy

22:11that characterizes Unix is, the design?

22:15Not just the initial, but just carry through the years,

22:18just being there, being around it.

22:20What's the fundamental philosophy behind the system?

22:23- I think one aspect of the fundamental philosophy

22:25was to provide an environment that made it easy to write

22:29or easier, productive to write programs.

22:31So it was meant as a programmer environment.

22:33It wasn't meant specifically as something

22:36to do some other kind of job.

22:38For example, it was used extensively for word processing,

22:41but it wasn't designed as a word processing system.

22:43It was used extensively for lab control,

22:45but it wasn't designed for that.

22:47It was used extensively as a front end

22:49for big other systems, big dumb systems,

22:52but it wasn't designed for that.

22:53It was meant to be an environment

22:55where it was really easy to write programs.

22:57So the programmers could be highly productive.

23:00And part of that was to be a community.

23:03And there's some observation from Dennis Ritchie,

23:05I think at the end of the book,

23:06it says that from his standpoint,

23:09the real goal was to create a community

23:11where people could work as programmers on a system.

23:16I think in that sense, certainly for many, many years,

23:19it succeeded quite well at that.

23:22And part of that is the technical aspects

23:25of because it made it really easy to write programs,

23:27people did write interesting programs.

23:29Those programs tended to be used by other programmers.

23:31And so it was kind of a virtuous circle

23:34of more and more stuff coming out

23:36that was really good for programmers.

23:39- And you were part of that community of programmers.

23:41So what was it like writing programs on that early Unix?

23:45- It was a blast, it really was.

23:47You know, I liked to program.

23:51I'm not a terribly good programmer,

23:52but it was a lot of fun to write code.

23:55And in the early days,

23:56there was an enormous amount of what you would,

23:58today I suppose, call low-hanging fruit.

24:00People hadn't done things before,

24:02and this was this new environment,

24:04and the whole combination of nice tools

24:07and very responsive system and tremendous colleagues

24:11made it possible to write code.

24:13You could have an idea in the morning,

24:16you could do an experiment with it,

24:19you could have something limping along that night

24:21or the next day, and people would react to it,

24:23and they would say, "Oh, that's wonderful,

24:25"but you're really screwed up here."

24:27And the feedback loop was then very, very short and tight.

24:31And so a lot of things got developed fairly quickly

24:34that in many cases still exist today.

24:39And I think that was part of what made it fun,

24:43because programming itself is fun.

24:44It's puzzle solving in a variety of ways,

24:46but I think it's even more fun when you do something

24:49that somebody else then uses.

24:52Even if they whine about it not working,

24:54the fact that they used it is part of the reward mechanism.

24:58- And what was the method of interaction,

25:00the communication, that feedback loop?

25:03I mean, this is before the internet.

25:05- Certainly before the internet.

25:07It was mostly physical right there,

25:11somebody would come into your office and say something.

25:13- So these places are all close by,

25:15like offices are nearby,

25:16so you're really lively in interaction.

25:18- Yeah, yeah, no, Bell Labs was fundamentally

25:20one giant building, and most of the people

25:23who were involved in this Unix stuff

25:24were in two or three quarters, and there was a room,

25:27oh, how big was it?

25:29Probably, call it 50 feet by 50 feet,

25:33make up a number of that, which had some access

25:37to computers there as well as in offices,

25:39and people hung out there, and it had a coffee machine.

§04Transcript · 25:42–34:52

25:42And so it was mostly very physical.

25:46We did use email, of course,

25:48but it was fundamentally, for a long time,

25:52all on one machine, so there was no need for internet.

25:56- It's fascinating to think about what computing

25:58would be today without Bell Labs.

26:00'Cause it seems so many,

26:02people being in the vicinity of each other,

26:06that sort of getting that quick feedback,

26:08working together, so many brilliant people.

26:11I don't know where else that could have existed

26:13in the world, I mean, given how that came together.

26:16(laughs)

26:17Yeah, how does that make you feel,

26:19that little element of history?

26:23- Well, I think that's very nice,

26:24but in a sense, it's survivor bias,

26:26and if it hadn't happened at Bell Labs,

26:29there were other places that were doing

26:30really interesting work as well.

26:32Xerox PARC is perhaps the most obvious one.

26:34Xerox PARC contributed an enormous amount of good material,

26:38and many of the things we take for granted today

26:40in the same way came from Xerox PARC experience.

26:43I don't think they capitalized in the long run as much.

26:46Their parent company was perhaps not as lucky

26:49in capitalizing on this, who knows?

26:52But that's certainly another place

26:54where there was a tremendous amount of influence.

26:58There were a lot of good university activities.

27:00MIT was obviously no slouch in this kind of thing,

27:03and others as well.

27:07- So Unix turned out to be open source

27:10because of the various ways that AT&T operated,

27:13and sort of it had to, the focus was on telephones.

27:17- I think that's a mischaracterization in a sense.

27:21It absolutely was not open source.

27:23It was very definitely proprietary, licensed,

27:27but it was licensed freely to universities

27:30in source code form for many years,

27:33and because of that, generations of university students

27:37and their faculty people grew up knowing about Unix,

27:41and there was enough expertise in the community

27:44that it then became possible for people

27:46to kind of go off in their own direction

27:48and build something that looked Unix-like.

27:50The Berkeley version of Unix started

27:54with that licensed code and gradually picked up

27:58enough of its own code contributions,

28:01notably from people like Bill Joy,

28:04that eventually it was able to become

28:07completely free of any AT&T code.

28:10Now, there was an enormous amount of legal jockeying

28:13around this in the late, early to late '80s, early '90s,

28:17something like that, and then not,

28:22I guess the open source movement might have started

28:26when Richard Stallman started to think about this

28:28in the late '80s, and by 1991, when Torvalds decided

28:32he was going to do a Unix-like operating system,

28:37there was enough expertise in the community

28:40that first he had a target, he could see what to do,

28:44because the kind of the Unix system call interface

28:47and the tools and so on were there,

28:49and so he was able to build an operating system

28:53that at this point, when you say Unix,

28:56in many cases what you're really thinking is Linux.

28:58- Linux, yeah.

28:59- It's funny that from my distant perception,

29:02I felt that Unix was open source

29:04without actually knowing it, but what you're really saying,

29:08it was just freely licensed.

29:11- It was freely licensed.

29:12- So it felt open source, because universities

29:15are not trying to make money, so it felt open source

29:18in the sense that you can get access if you wanted.

29:20- Right, and a very, very, very large number

29:23of universities had the license, and they were able

29:24to talk to all the other universities who had the license,

29:27and so technically not open, technically belonging to AT&T,

29:32pragmatically pretty open.

29:34- And so there's a ripple effect that all the faculty

29:37and the students, then they all grew up,

29:38and then they went throughout the world

29:41and permeated in that kind of way.

29:45So what kind of features do you think make

29:49for a good operating system?

29:51If you take the lessons of Unix, you said,

29:57make it easy for programmers, that seems to be

30:00an important one, but also Unix turned out

30:05to be exceptionally robust and efficient.

30:08- Right.

30:09- So is that an accident when you focus on the programmer,

30:12or is that a natural outcome?

30:14- I think part of the reason for efficiency

30:17was that it began on extremely modest hardware,

30:21very, very, very tiny, and so you couldn't get carried away,

30:23you couldn't do a lot of complicated things

30:26because you just didn't have the resources,

30:30either processor speed or memory,

30:32and so that enforced a certain minimality of mechanisms,

30:37and maybe a search for generalizations

30:40so that you would find one mechanism

30:41that served for a lot of different things

30:43rather than having lots of different special cases.

30:45I think the file system in Unix is a good example

30:48of that file system interface in its fundamental form

30:51is extremely straightforward,

30:53and that means that you can write code

30:55very, very effectively for the file system,

30:58and then one of those generalizations

31:02is that, gee, that file system interface

31:04works for all kinds of other things as well,

31:06and so in particular, the idea of reading and writing

31:09to devices is the same as reading and writing

31:11to a disk that has a file system,

31:14and then that gets carried further in other parts.

31:17The world processes become, in effect,

31:21files in a file system,

31:23and the Plan 9 operating system,

31:25which came along, I guess, in the late '80s

31:27or something like that, took a lot of those ideas

31:30from the original Unix and tried to push

31:32the generalization even further

31:34so that in Plan 9, a lot of different resources

31:37are file systems.

31:38They all share that interface,

31:40so that would be one example

31:42where finding the right model of how to do something

31:45means that an awful lot of things become simpler,

31:48and it means, therefore, that more people can do useful,

31:50interesting things with them

31:51without having to think as hard about it.

31:54- So you said you're not a very good programmer.

31:56- True.

31:58- You're the world's most modest human being, okay,

32:01but you'll continue saying that.

32:03I understand how this works,

32:04but you do radiate a sort of love for programming,

32:07so let me ask, do you think programming

32:10is more an art or a science?

32:13Is it creativity or kind of rigor?

32:16- I think it's some of each.

32:18It's some combination.

32:20Some of the art is figuring out what it is

32:22that you really wanna do, what should that program be,

32:25what would make a good program,

32:27and that's some understanding of what the task is,

32:30what the people who might use this program want,

32:33and I think that's art in many respects.

32:37The science part is trying to figure out how to do it well,

32:40and some of that is real computer science-y stuff,

32:45like what algorithm should we use at some point,

32:48mostly in the sense of being careful to use algorithms

32:52that will actually work properly, scale properly,

32:56avoiding quadratic algorithms

32:57when a linear algorithm should be the right thing,

33:01that kind of more formal view of it.

33:04Same thing for data structures,

33:06but also it's, I think, an engineering field as well,

33:10and engineering's not quite the same as science

33:12because in engineering you're working with constraints.

33:15You have to figure out not only,

33:18so what is a good algorithm for this kind of thing,

33:20but what's the most appropriate algorithm

33:22given the amount of time we have to compute,

33:26the amount of time we have to program,

33:27what's likely to happen in the future with maintenance,

33:30who's gonna pick this up in the future,

33:33all of those kind of things that if you're an engineer,

33:35you get to worry about,

33:37whereas if you think of yourself as a scientist,

33:39well, you can maybe push them over the horizon in a way,

33:42and if you're an artist, what's that?

33:44(laughing)

33:45- So just on your own personal level,

33:48what's your process like of writing a program,

33:50say a small and large sort of tinkering with stuff?

33:55Do you just start coding right away

33:57and just kind of evolve iteratively with a loose notion,

34:02or do you plan on a sheet of paper first

34:05and then kind of design what they teach you

34:09in the kind of software engineering courses in undergrad

34:12or something like that?

34:13What's your process like?

34:14- It's certainly much more the informal incremental.

34:17First, I don't write big programs at this point.

34:20It's been a long time since I wrote a program

34:21that was more than, I call it a few hundred or more lines,

34:25something like that.

34:27Many of the programs I write are experiments

34:29for either something I'm curious about

34:31or often for something that I wanna talk about in a class,

34:34and so those necessarily tend to be relatively small.

34:38A lot of the kind of code I write these days

34:41tends to be for sort of exploratory data analysis

34:44where I've got some collection of data

34:46and I wanna try and figure out

34:47what on earth is going on in it,

34:49and for that, those programs tend to be very small.

34:52Sometimes you're not even programming,

§05Transcript · 34:53–43:10

34:53you're just using existing tools like counting things,

34:57or sometimes you're writing ox scripts

35:00because two or three lines will tell you something

35:02about a piece of data, and then when it gets bigger,

35:05well, then I will probably write something in Python

35:08because that scales better,

35:10up to, call it a few hundred lines or something like that,

35:14and it's been a long time since I wrote programs

35:16that were much more than that.

35:18- Speaking of data exploration and awk,

35:21first, what is awk?

35:23- So awk is a scripting language

35:25that was done by myself, Al Aho, and Peter Weinberger.

35:30We did that originally in the late '70s.

35:32It was a language that was meant to make it really easy

35:34to do quick and dirty tasks like counting things

35:39or selecting interesting information

35:42from basically all text files,

35:45rearranging it in some way or summarizing it.

35:48- It runs a command on each line of a file.

35:51I mean, it's still exceptionally widely used today.

35:55- Oh, absolutely, yeah.

35:56- It's so simple and elegant,

35:58sort of the way to explore data.

36:01Turns out you can just write a script

36:03that does something seemingly trivial on a single line,

36:07and that giving you that slice of the data

36:09somehow reveals something fundamental about the data,

36:13and that keeps, that seems to work still.

36:17- Yeah, it's very good for that kind of thing,

36:19and that's sort of what it was meant for.

36:21I think what we didn't appreciate

36:22was that the model was actually quite good

36:24for a lot of data processing kinds of tasks,

36:27and that it's kept going as long as it has,

36:30'cause at this point it's over 40 years old,

36:32and it's still, I think, a useful tool.

36:35And well, this is paternal interest, I guess,

36:38but I think in terms of programming languages,

36:41you get the most bang for the buck by learning AWK,

36:44and it doesn't scale to big programs,

36:46but it does pretty darn well on these little things

36:49where you just wanna see all the somethings in something.

36:52So yeah, I find, I probably write more AWK

36:56than anything else at this point.

36:58- So what kind of stuff do you love about AWK?

37:01Like, is there, if you can comment on sort of things

37:05that give you joy when you can, in a simple program,

37:10reveal something about the data,

37:11is there something that stands out from particular features?

37:14- I think it's mostly the selection of default behaviors,

37:19that you sort of hinted at it a moment ago.

37:21What AWK does is to read through a set of files,

37:24and then within each file,

37:26it reads through each of the lines,

37:28and then on each of the lines,

37:29it has a set of patterns that it looks for,

37:33that's your AWK program,

37:34and if one of the patterns matches,

37:36there is a corresponding action that you might perform,

37:39and so it's kind of a quadruply nested loop

37:43or something like that,

37:45and that's all completely automatic.

37:46You don't have to say anything about it,

37:48you just write the pattern and the action,

37:49and then run the data by it,

37:52and so that paradigm for programming

37:54is a very natural and effective one,

37:56and I think we captured that reasonably well in AWK,

38:00and it does other things for free as well.

38:01It splits the data into fields,

38:04so that on each line,

38:05there's fields separated by white space or something,

38:07and so it does that for free,

38:08you don't have to say anything about it,

38:11and it collects information as it goes along,

38:13like what line are we on,

38:15how many fields are there on this line?

38:18So lots of things that just make it so that a program

38:20which in another language, let's say Python,

38:24would be five, 10, 20 lines,

38:26in AWK it's one or two lines.

38:28- And so because it's one or two lines,

38:29you can do it on the shell,

38:31you don't have to open up another whole thing,

38:33you can just do it right there

38:35in the interaction with the operator directly.

38:37Is there other shell commands that you love over the years,

38:43like you really enjoy using?

38:46- Oh, grep. - Grep?

38:47- Grep's the only one.

38:49Yeah, grep does everything.

38:51- So grep is a kind of a, what is it,

38:53a simpler version of AWK, I would say?

38:55- In some sense, yeah, right, because--

38:58- What is grep?

38:58- So grep is, it basically searches the input

39:01for particular patterns, regular expressions,

39:03technically, of a certain class,

39:05and it has that same paradigm that AWK does,

39:08it's a pattern action thing,

39:10it reads through all the files

39:11and then all the lines in each file,

39:13but it has a single pattern,

39:15which is the regular expression you're looking for,

39:17and a single action printed, if it matches.

39:20So in that sense, it's a much simpler version

39:22and you could write grep and AWK as a one-liner,

39:25and I use grep probably more than anything else

39:30at this point, just because it's so convenient and natural.

39:35- Why do you think, it's such a powerful tool, grep and AWK,

39:38why do you think operating systems like Windows,

39:41for example, don't have it?

39:44Sort of, you can, of course, I use, which is amazing now,

39:48there's Windows for Linux,

39:51which you could basically use all the fun stuff

39:54like AWK and grep inside of Windows,

39:57but Windows naturally,

39:58sort of as part of the graphical interface,

40:00the simplicity of grep,

40:02sort of searching through a bunch of files

40:04and just popping up naturally.

40:06Why don't you think that,

40:08why do you think that's unique

40:09to the Unix and Linux environment?

40:11- I don't know, it's not strictly unique,

40:14but it's certainly focused there,

40:16and I think some of it's the weight of history

40:19that Windows came from MS-DOS,

40:22MS-DOS was a pretty pathetic operating system,

40:24although common on an unboundedly large number of machines,

40:29but somewhere in roughly the '90s,

40:32Windows became a graphical system,

40:34and I think Microsoft spent a lot of their energy

40:37on making that graphical interface what it is,

40:41and that's a different model of computing.

40:44It's a model of computing that,

40:45where you point and click

40:47and sort of experiment with menus,

40:49it's a model of computing works rather well

40:52for people who are not programmers,

40:54who just wanna get something done,

40:56whereas teaching something like the command line

40:59to non-programmers turns out

41:00to sometimes be an uphill struggle,

41:02and so I think Microsoft probably was right

41:04in what they did.

41:06Now you mentioned Whistle or whatever it's called,

41:08the Linux.

41:09- Whistle, I wonder what it's pronounced,

41:11W-S-L is what,

41:12I've never actually pronounced the Whistle, I like it.

41:13- I have no idea.

41:14(laughing)

41:15- Whistle.

41:16- And things like that for a long,

41:17a Cygwin, for example,

41:18which is a wonderful collection

41:20of take all your favorite tools from Unix and Linux

41:23and just make them work perfectly on Windows,

41:25and so that's something that's been going on

41:27for at least 20 years, if not longer,

41:29and I use that on my one remaining Windows machine

41:33routinely because if you're doing something

41:37that is batch computing,

41:39suitable for command line,

41:41that's the right way to do it

41:42because the Windows equivalents are,

41:44if nothing else, not familiar to me.

41:46- But I would definitely recommend to people

41:49to if they don't use Cygwin to try Whistle.

41:52- Yes.

41:53- I've been so excited that I could use BASH,

41:57that you bash, write scripts quickly in Windows,

42:00it's changed my life.

42:03Okay, what's your perfect programming setup?

42:06What computer, what operating system,

42:08what keyboard, what editor?

42:10- Yeah, perfect is too strong a word.

42:13It's way too strong a word.

42:15What I use by default, I have a,

42:18at this point, a 13-inch MacBook Air,

42:20which I use because it's kind of a reasonable balance

42:24of the various things I need.

42:25I can carry it around.

42:26It's got enough computing horsepower,

42:28screen's big enough, keyboard's okay,

42:30and so I basically do most of my computing on that.

42:34I have a big iMac in my office

42:37that I use from time to time as well,

42:39especially when I need a big screen,

42:41but otherwise, it tends not to be used that much.

42:44- Editor.

42:48- I use mostly SAM, which is an editor

42:51that Rob Pike wrote long ago at Bell Labs.

42:55- Did that, sorry to interrupt, does that precede VI?

42:58Does that precede Emacs?

43:00- It post-dates both VI and Emacs.

43:03It is derived from Rob's experience with ED and VI.

43:10- ED.

§06Transcript · 43:10–50:59

43:10- That's the original Unix editor.

43:14- Oh, wow.

43:15- Dated probably before you were born.

43:18- So what's, actually, what's the history of editors?

43:23Can you briefly, 'cause it's such a,

43:26I use Emacs, I'm sorry to say.

43:28So sorry to come out with that,

43:30but what's the kind of interplay there?

43:33So SAM, yeah, 'cause VI.

43:35- So in ancient times, like,

43:37call it the first time-sharing systems,

43:39going back to what we were talking about,

43:40there were editors, there was an editor on CTSS

43:43that I don't even remember what it was called.

43:46It might've been edit, where you could type text,

43:49program text, and it would do something,

43:52or document text.

43:53- You could save the text.

43:54- And save it, you could edit it,

43:57usual thing that you would get in an editor.

44:00And Ken Thompson wrote an editor called QED,

44:03which was very, very powerful,

44:06but these were all totally a command-based,

44:08they were not mouse or cursor-based,

44:10because it was before mice, and even before cursors,

44:13'cause they were running on terminals

44:15that printed on paper, okay?

44:17No CRT-type displays, let alone LEDs.

44:20And so then when Unix came along,

44:24Ken took QED and stripped it way, way, way, way down,

44:28and that became an editor that he called ED.

44:31And it was very simple, but it was a line-oriented editor.

44:33And so you could load a file,

44:36and then you could talk about the lines,

44:37one through the last line,

44:39and you could print ranges of lines,

44:41you could add text, you could delete text,

44:44you could change text,

44:44or you could do a substitute command

44:46that would change things within a line,

44:47or within groups of lines.

44:49- So you could work on parts of a file, essentially.

44:51- Yeah, you could work on any part of it,

44:53the whole thing, or whatever,

44:53but it was entirely command-line-based,

44:57and it was entirely on paper, okay?

45:00Paper, and that meant that you changed--

45:02- Actual paper.

45:03- Yeah, right, real paper.

45:04And so if you changed a line,

45:06you had to print that line,

45:07using up another line of paper

45:09to see what the change caused, okay?

45:11- Yeah, it's amazing.

45:12- So when CRT displays came along,

45:17then you could start to use cursor control,

45:19and you could sort of move where you were on the screen in--

45:23- Without reprinting every time.

45:26- Printing, and there were a number of editors there.

45:29The one that I was most familiar with,

45:31and still use, is VI, which was done by Bill Choi.

45:35And so that dates from probably the late '70s, as a guess,

45:40and it took full advantage of the cursor controls.

45:45I suspected Emacs was roughly at the same time,

45:48but I don't know, I've never internalized Emacs.

45:51So I use, at this point, I stopped using ED,

45:54although I still can.

45:56I use VI sometimes, and I use SAM when I can.

46:00- And SAM is available on most systems?

46:02- It is available.

46:04You have to download it yourself

46:05from typically the plan line operating system distribution.

46:08It's been maintained by people there.

46:10And so--

46:12- I'll get home tonight, I'll try it.

46:14It's cool.

46:15It sounds fascinating.

46:17Although my love is with Lisp and Emacs,

46:20I've went into that hippie world of--

46:23(both laughing)

46:25- I think it's a lot of things.

46:26What religion were you brought up with?

46:27- Yeah, that's true.

46:28That's true.

46:29Most of the actual programming I do

46:31is C, C++, and Python,

46:34but my weird sort of, yeah,

46:36my religious upbringing is in Lisp.

46:38So can you take on the impossible task

46:41and give a brief history of programming languages

46:44from your perspective?

46:46- So I guess you could say programming languages

46:48started probably in what, the late '40s

46:51or something like that.

46:51People used to program computers

46:53by basically putting in zeros and ones

46:56using something like switches on a console.

46:59And then, or maybe holes in paper tapes,

47:03something like that.

47:04So extremely tedious, awful, whatever.

47:07And so I think the first programming languages

47:10were relatively crude assembly languages

47:14where people would basically write a program

47:18that would convert mnemonics like add, A-D-D,

47:22into whatever the bit pattern was

47:24that corresponded to an add instruction.

47:26And they would do the clerical work

47:28of figuring out where things were.

47:30So you could put a name on a location in a program

47:32and the assembler would figure out

47:34where that corresponded to

47:36when the thing was all put together

47:37and dropped into memory.

47:39And early on, and this would be the late '40s

47:45and very early '50s,

47:46there were assemblers written for the various machines

47:50that people used.

47:51You may have seen in the paper just a couple of days ago,

47:52Tony Berker died.

47:54He did this thing in Manchester

47:55called AutoCode, a language which I knew only by name.

48:00But it sounds like it was a flavor of assembly language,

48:04sort of a little higher in some ways.

48:06And it replaced a language that Alan Turing wrote,

48:09which you put in zeros and ones,

48:11but you put it in backwards order

48:12because that was a hardware work.

48:14Very strange. - That's right.

48:15Yeah, yeah, that's right, backwards.

48:17- So assembly languages,

48:19and let's call that the early 1950s.

48:22And so every different flavor of computer

48:24has its own assembly language.

48:25So the EdSac had its, and the Manchester had its,

48:28and the IBM, whatever, 7090 or 704 or whatever had its,

48:33and so on.

48:34So everybody had their own assembly language.

48:35- And assembly languages have a few commands, additions,

48:38subtraction, then branching of some kind,

48:41if then type of situation.

48:42- Right, they have exactly, in their simplest form at least,

48:46one assembly language instruction

48:50per instruction in the machine's repertoire.

48:52And so you have to know the machine intimately

48:54to be able to write programs in it.

48:56And if you write an assembly language program

48:58for one kind of machine, and then you say,

49:00"Gee, it's nice, I'd like it in a different machine,"

49:02start over, okay.

49:04So, very bad.

49:05And so what happened in the late '50s

49:08was people realized you could play this game again,

49:11and you could move up a level in writing

49:14or creating languages that were closer

49:16to the way that real people might think about

49:18how to write code.

49:20And there were, I guess, arguably three or four

49:24at that time period.

49:25There was Fortran, which came from IBM,

49:28which was formula translation,

49:29meant to make it easy to do scientific

49:31and engineering computations.

49:32- I didn't know that, formula translation, wow.

49:34- That's what it stood for.

49:35There was COBOL, which is the Common Business Oriented

49:37Language that Grace Hopper and others worked on,

49:40which was aimed at business kinds of tasks.

49:44There was ALGOL, which was mostly meant

49:45to describe algorithmic computations.

49:49I guess you could argue BASIC was in there somewhere.

49:51I think it's just a little later.

49:52And so all of those moved the level up.

49:56And so they were closer to what you and I might think of

49:59as we were trying to write a program.

50:02And they were focused on different domains,

50:05Fortran for formula translation, engineering computations,

50:09let's say COBOL for business, that kind of thing.

50:11- And still used today, at least Fortran probably.

50:14- Oh yeah, COBOL too.

50:16But the deal was that once you moved up that level,

50:19then you, let's call it Fortran, you had a language

50:21that was not tied to a particular kind of hardware

50:25because a different compiler would compile

50:26for a different kind of hardware.

50:27And that meant two things.

50:30It meant you only had to write the program once,

50:32which was very important.

50:33And it meant that you could, in fact,

50:35if you were a random engineer, physicist, whatever,

50:38you could write that program yourself.

50:39You didn't have to hire a programmer to do it for you.

50:42It might not be as good as you'd get with a real programmer,

50:44but it was pretty good.

50:45And so it democratized and made much more broadly available

50:49the ability to write code.

50:51- So it puts the power of programming

50:53into the hands of people like you.

50:54- Yeah, anybody who wants, who is willing to invest

50:57some time in learning a programming language

50:59and is not then tied to a particular kind of computer.

§07Transcript · 51:03–59:36

51:03And then in the '70s, you get system programming languages,

51:06of which C is the survivor.

51:07- What does system programming languages mean?

51:11- Programs that, programming languages

51:14that would take on the kinds of things

51:16that would necessary to write so-called system programs,

51:19things like text editors or assemblers or compilers

51:22or operating systems themselves, those kinds of things.

51:26And Fortran-

51:28- Feature rich, they have to be able to do a lot of stuff,

51:30a lot of memory management, access processes,

51:33all that kind of stuff.

51:34- They have to- - Level of processing.

51:35- It's a different flavor of what they're doing.

51:37They're much more in touch with the actual machine

51:41but in a positive way.

51:42That is, you can talk about memory in a more controlled way.

51:45You can talk about the different data types

51:47that the machine supports and the way they're,

51:50and more ways to structure and organize data.

51:54And so the system programming languages,

51:57there was a lot of effort in that in the,

51:59call it the late '60s, early '70s.

52:02C is, I think, the only real survivor of that.

52:05And then what happens after that,

52:08you get things like object-oriented programming languages

52:12because as you write programs in a language like C,

52:14at some point, scale gets to you

52:16and it's too hard to keep track of the pieces

52:18and there's no guardrails or training wheels

52:21or something like that to prevent you from doing bad things.

52:24So C++ comes out of that tradition.

52:28- And then it took off from there.

52:29I mean, there's also a parallel, slightly parallel track

52:32with a little bit of the functional stuff

52:33with Lisp and so on.

52:35But I guess from that point,

52:37it's just an explosion of languages.

52:38There's the Java story, there's the JavaScript,

52:41there's all the stuff that the cool kids these days

52:44are doing with Rust and all that.

52:46So what's to you, so you wrote a book,

52:51C programming language,

52:53and C is probably one of the most important languages

52:56in the history of programming languages

52:58if you kind of look at impact.

53:01What do you think is the most elegant

53:02or powerful part of C?

53:06Why did it survive?

53:07Why did it have such a long-lasting impact?

53:11- I think it found a sweet spot of expressiveness,

53:16that you could really write things

53:17in a pretty natural way, and efficiency,

53:20which was particularly important

53:22when computers were not nearly as powerful

53:24as they are today.

53:25You're gonna put yourself back 50 years,

53:28almost in terms of what computers could do,

53:31and that's roughly four or five generations,

53:35decades of Moore's law, right?

53:37So expressiveness and efficiency,

53:40and I don't know, perhaps the environment

53:43that it came with as well, which was Unix.

53:46So it meant if you wrote a program,

53:47it could be used on all those computers that ran Unix,

53:50and that was all of those computers

53:51because they were all written in C,

53:53and that way Unix, the operating system itself,

53:56was portable, as were all the tools.

53:58So it all worked together, again,

54:00in one of these things where things fed on each other

54:03in a positive cycle.

54:05- What did it take to write sort of a definitive book,

54:10probably definitive book on all of program,

54:11like it's more definitive to a particular language

54:14than any other book on any other language,

54:16and did two really powerful things,

54:19which is popularized the language,

54:22at least from my perspective, maybe you can correct me,

54:24and second is created a standard

54:27of how this language is supposed to be used and applied.

54:33So what did it take?

54:34Did you have those kinds of ambitions in mind

54:37when working on that?

54:38- Is this some kind of joke?

54:39(laughing)

54:41No, of course not.

54:42- So it's an accident of timing, skill, and just luck.

54:47- A lot of it is, clearly, timing was good.

54:51Now, Dennis and I wrote the book in 1977.

54:54- Dennis Ritchie. - Yeah, right.

54:56And at that point, Unix was starting to spread.

54:58I don't know how many there were,

55:00but it would be dozens to hundreds of Unix systems,

55:03and C was also available on other kinds of computers

55:06that had nothing to do with Unix,

55:08and so the language had some potential.

55:11And there were no other books on C,

55:17and Bell Labs was really the only source for it,

55:20and Dennis, of course, was authoritative

55:22because it was his language,

55:23and he had written the reference manual,

55:26which is a marvelous example

55:28of how to write a reference manual.

55:29Really, really very, very well done.

55:31So I twisted his arm until he agreed to write a book,

55:34and then we wrote a book.

55:35And the virtue, or advantage, at least,

55:38I guess, of going first is that then other people

55:40have to follow you if they're gonna do anything.

55:43And I think it worked well

55:46because Dennis was a superb writer.

55:50I mean, he really, really did,

55:51and the reference manual in that book is his, period.

55:55I had nothing to do with that at all.

55:58So just crystal clear prose, very, very well expressed.

56:02And then he and I,

56:03I wrote most of the expository material,

56:07and then he and I sort of did the usual

56:09ping-ponging back and forth, refining it.

56:13But I spent a lot of time trying to find examples

56:15that would sort of hang together

56:16and that would tell people what they might need to know

56:18at about the right time,

56:20that they should be thinking about needing it.

56:22And I'm not sure it completely succeeded,

56:25but it mostly worked out fairly well.

56:28- What do you think is the power of example?

56:30I mean, you're the creator,

56:34at least one of the first people

56:36to do the Hello World program,

56:38which is like the example.

56:40If aliens discover our civilization

56:43hundreds of years from now,

56:44it'll probably be Hello World programs,

56:46just like a half-broken robot

56:48communicating with them with a Hello World.

56:50So what, and that's a representative example,

56:53so what do you find powerful about examples?

56:57- I think a good example will tell you how to do something

57:01and it will be representative of,

57:03you might not wanna do exactly that,

57:05but you will wanna do something

57:06that's at least in that same general vein.

57:09And so a lot of the examples in the C book

57:13were picked for these very, very simple,

57:15straightforward text processing problems

57:17that were typical of Unix.

57:19I want to read input and write it out again.

57:23There's a copy command.

57:24I wanna read input and do something to it

57:27and write it out again.

57:28There's a grab.

57:28And so that kind of find things

57:31that are representative of what people want to do

57:35and spell those out so that they can then take those

57:39and see the core parts and modify them to their taste.

57:44And I think that a lot of programming books that,

57:48I don't look at programming books

57:51a tremendous amount these days,

57:52but when I do, a lot of them don't do that.

57:54They don't give you examples that are both realistic

57:59and something you might want to do.

58:01Some of them are pure syntax.

58:03Here's how you add three numbers.

58:05Well, come on, I could figure that out.

58:07Tell me how I would get those three numbers

58:09into the computer and how we would do something useful

58:11with them and then how I put them back out again,

58:14neatly formatted.

58:15- And especially if you follow that example,

58:17there is something magical of doing something

58:19that feels useful.

58:21- Yeah, right.

58:21And I think it's the attempt,

58:23and it's absolutely not perfect,

58:25but the attempt in all cases was to get something

58:28that was going to be either directly useful

58:31or would be very representative of useful things

58:35that a programmer might want to do.

58:37But within that vein of fundamentally text processing,

58:41reading text, doing something, writing text.

58:43- So you've also written a book on Go language.

58:47I have to admit, so I worked at Google for a while

58:50and I've never used Go.

58:53- Well, you missed something.

58:54- Well, I know I missed something for sure.

58:56So Go and Rust are two languages that I hear very,

59:01spoken very highly of and I wish I would like to,

59:05well, there's a lot of them.

59:06There's Julia, there's all these incredible modern languages

59:10but if you can comment before,

59:12or maybe comment on what do you find,

59:16where does Go sit in this broad spectrum of languages?

59:19And also how do you yourself feel about this wide range

59:23of powerful, interesting languages

59:26that you may never even get to try to explore

59:30because of time?

59:31- So I think, so Go first comes from that same

59:36Bell Labs tradition in part, not exclusively,

§08Transcript · 59:39–1:08:06

59:39but two of the three creators, Ken Thompson and Rob Pike.

59:42- So literally the people.

59:44- Yeah, the people.

59:45And then with this very, very useful influence

59:49from the European school in particular,

59:51the Klaus Spirth influence through Robert Griesemer

59:55who was, I guess, a second generation down student at ETH.

60:00And so that's an interesting combination of things.

60:03And so some ways Go captures the good parts of C,

60:08it looks sort of like C, it's sometimes characterized

60:11as C for the 21st century.

60:12On the surface, it looks very, very much like C.

60:17But at the same time, it has some interesting

60:20data structuring capabilities.

60:21And then I think the part that I would say

60:25is particularly useful, and again, I'm not a Go expert.

60:29In spite of co-authoring the book,

60:31about 90% of the work was done by Alan Donovan,

60:34my co-author who is a Go expert.

60:36But Go provides a very nice model of concurrency.

60:42It's basically the cooperating,

60:44communicating sequential processes that Tony Hoare

60:47set forth, geez, I don't know, 40 plus years ago.

60:51And Go routines are, to my mind, a very natural way

60:56to talk about parallel computation.

60:59And in the few experiments I've done with them,

61:01they're easy to write and typically it's gonna work

61:04and very efficient as well.

61:07So I think that's one place where Go stands out,

61:09that that model of parallel computation

61:13is very, very easy and nice to work with.

61:16- Just to comment on that, do you think C foresaw,

61:19or the early Unix days foresaw threads

61:22and massively parallel computation?

61:26- I would guess not really.

61:27I mean, maybe it was seen, but not at the level

61:30where it was something you had to do anything about.

61:33For a long time, processors got faster.

61:38And then processors stopped getting faster

61:41because of things like power consumption

61:43and heat generation.

61:46And so what happened instead was that instead

61:49of processors getting faster,

61:50there started to be more of them.

61:52And that's where that parallel thread stuff comes in.

61:55- So if you can comment on all the other languages,

62:01does it break your heart that you'll never get

62:03to explore them?

62:04How do you feel about the full variety?

62:07- It's not break my heart, but I would love

62:10to be able to try more of these languages.

62:12The closest I've come is in a class

62:14that I often teach in the spring here.

62:17It's a programming class.

62:18And I often give, I have one sort of small example

62:23that I will write in as many languages as I possibly can.

62:27I've got it in 20 odd languages at this point.

62:30And that's so I do a minimal experiment with a language,

62:35just to say, okay, I have this trivial task,

62:37which I understand the task, and it should,

62:39it takes 15 lines in awk and not much more

62:42in a variety of other languages.

62:44So how big is it?

62:45How fast does it run?

62:46And what pain did I go through to learn how to do it?

62:49And that's like anic data, right?

62:55It's very, very, very narrowly focused.

62:58- Anic data, I like that term.

62:59So yeah, but still, it's a little sample

63:01'cause you get the, I think the hardest step

63:04of the programming language is probably the first step.

63:06Right, so there you're taking the first step.

63:08- Yeah, and so my experience with some languages

63:13is very positive, like Lua,

63:14a scripting language I had never used.

63:17And I took my little program.

63:19The program is a trivial formatter.

63:21It just takes in lines of text of varying lengths,

63:24and it puts them out in lines

63:26that have no more than 60 characters on each line.

63:28So think of it as just kind of the flow of process

63:31in a browser or something.

63:34So it's a very short program.

63:36And in Lua, I downloaded Lua,

63:39and in an hour I had it working,

63:41never having written Lua in my life,

63:43just going with online documentation.

63:44I did the same thing in Scala,

63:46which you can think of as a flavor of Java,

63:48equally trivial.

63:50I did it in Haskell.

63:52It took me several weeks.

63:53But it did run like a turtle.

63:57And I did it in Fortran 90,

64:03and it was painful, but it worked.

64:06And I tried it in Rust,

64:07and it took me several days to get it working

64:10because the model of memory management

64:12was just a little unfamiliar to me.

64:13And the problem I had with Rust,

64:15and it's back to what we were just talking about,

64:18I couldn't find good, consistent documentation on Rust.

64:21Now, this was several years ago,

64:22and I'm sure things have stabilized,

64:24but at the time, everything in the Rust world

64:26seemed to be changing rapidly.

64:27And so you would find what looked like a working example,

64:30and it wouldn't work with the version

64:32of the language that I had.

64:34So it took longer than it should have.

64:37Rust is a language I would like to get back to,

64:39but probably won't.

64:41I think one of the issues,

64:42you have to have something you want to do.

64:44If you don't have something that is the right combination,

64:47if I want to do it,

64:48and yet I have enough disposable time, whatever,

64:53to make it worth learning a new language at the same time,

64:56it's never gonna happen.

64:58- So what do you think about another language of JavaScript

65:02that's this, well, let me just sort of comment on this.

65:06When I was brought up,

65:08sort of JavaScript was seen as

65:09the probably like the ugliest language possible,

65:15and yet it's quite arguably, quite possibly,

65:18taking over not just the front end

65:20and the back end of the internet,

65:21but possibly in the future taking over everything,

65:24because they've now learned to make it very efficient.

65:27- Yeah.

65:27- And so what do you think about this?

65:29- Yeah, well, I think you've captured it in a lot of ways.

65:32When it first came out,

65:33JavaScript was deemed to be fairly irregular

65:35and an ugly language,

65:36and certainly in the academy,

65:37if you said you were working on JavaScript,

65:39people would ridicule you.

65:40It was just not fit for academics to work on.

65:43I think a lot of that has evolved.

65:45The language itself has evolved,

65:47and certainly the technology of compiling it

65:50is fantastically better than it was.

65:53And so in that sense,

65:54it's absolutely a viable solution on back ends,

65:58as well as the front ends.

66:00Used well, I think it's a pretty good language.

66:03I've written a modest amount of it,

66:06and I've played with JavaScript translators

66:09and things like that.

66:10I'm not a real expert,

66:12and it's hard to keep up even there

66:13with the new things that come along with it.

66:15So I don't know whether it will ever take over the world.

66:21I think not,

66:22but it's certainly an important language

66:26and worth knowing more about.

66:29- Maybe to get your comment on something,

66:32which JavaScript, and actually most languages,

66:35sort of Python,

66:36such a big part of the experience of programming

66:40with those languages includes libraries.

66:42So using, building on top of the code

66:44that other people have built.

66:46I think that's probably different from the experience

66:48that we just talked about from Unix and C days

66:51when you're building stuff from scratch.

66:53What do you think about this world

66:55of essentially leveraging,

66:56building up libraries on top of each other

66:58and leveraging them?

66:59- Yeah, that's a very perceptive kind of question.

67:02One of the reasons programming was fun in the old days

67:06was that you were really building it all yourself.

67:09The number of libraries you had to deal with

67:10was quite small.

67:11Maybe it was printf or the standard library

67:14or something like that.

67:16And that is not the case today.

67:17And if you want to do something in,

67:20you mentioned Python and JavaScript,

67:22and those are the two fine examples

67:24you have to typically download a boatload of other stuff

67:27and you have no idea what you're getting.

67:29Absolutely nothing.

67:31I've been doing some playing with machine learning

67:33over the last couple of days and gee,

67:37something doesn't work.

67:38Well, you pip install this, okay?

67:40And down comes another gazillion megabytes of something

67:44and you have no idea what it was.

67:46And if you're lucky it works.

67:48And if it doesn't work, you have no recourse.

67:51There's absolutely no way you could figure out

67:52which in these thousand different packages.

67:55And I think it's worse in the NPM environment

67:59for JavaScript.

68:00I think there's less discipline, less control there.

68:03- And there's aspects of not just not understanding

68:06how it works, but there's security issues,

§09Transcript · 1:08:08–1:16:43

68:08there's robustness issues.

68:09So you don't want to run a nuclear power plant

68:11using JavaScript essentially.

68:14- Probably not.

68:14- So speaking to the variety of languages,

68:18do you think that variety is good?

68:20Or do you hope, think that over time

68:23we should converge towards one, two, three

68:25programming languages?

68:27That you mentioned to the Bell a lot of days

68:29when people could sort of, the community of it.

68:32And the more languages you have,

68:34the more you separate the communities.

68:36There's the Ruby community, there's the Python community,

68:40there's C++ community.

68:42Do you hope that they'll unite one day

68:45to just one or two languages?

68:47- I certainly don't hope it.

68:48I'm not sure that that's right

68:50'cause I honestly don't think there is one language

68:51that will suffice for all the programming needs

68:54of the world.

68:55Are there too many at this point?

68:56Well, arguably, but I think if you look at the,

69:00that sort of the distribution of how they are used,

69:03there's something called a dozen languages

69:06that probably account for 95% of all programming

69:10at this point.

69:11And that doesn't seem unreasonable.

69:13And then there's another, well, 2000 languages

69:17that are still in use that nobody uses.

69:19And, or at least don't use in any quantity.

69:23But I think new languages are a good idea in many respects

69:25'cause they're often a chance to explore an idea

69:30of how a language might help.

69:32I think that's one of the positive things

69:35about functional languages.

69:36For example, they're a particularly good place

69:38where people have explored ideas

69:42that at the time didn't seem feasible,

69:45but ultimately have wound up

69:47as part of mainstream languages as well.

69:50I mean, you just go back as early as recursion Lisp

69:52and then follow forward.

69:54Functions as first-class citizens

69:57and pattern-based languages, and gee, I don't know,

70:01closures and just on and on and on.

70:04Lambda's interesting ideas that showed up first

70:07in let's call it broadly

70:08the functional programming community

70:10and then find their way into mainstream languages.

70:13- Yeah, it's a playground for rebels.

70:15- Yeah, exactly.

70:16(Lex laughing)

70:18And so I think the languages in the playground themselves

70:21are probably not going to be the mainstream,

70:24at least for some while,

70:25but the ideas that come from there are invaluable.

70:28- So let's go to something that when I found out recently,

70:33so I've known that you've done a million things,

70:36but one of the things I wasn't aware of

70:37that you had a role in AMPL,

70:39and before you interrupt me by minimizing your role in it.

70:44- AMPL is for minimizing functions.

70:46- Yeah, minimizing functions, right, exactly.

70:48Can I just say that the elegance

70:52and abstraction power of AMPL is incredible?

70:57When I first came to it about 10 years ago or so,

71:01can you describe what is the AMPL language?

71:04- Sure, so AMPL is a language for mathematical programming,

71:08technical term, think of it as linear programming,

71:10that is setting up systems of linear equations

71:14that are of some sort of system of constraints,

71:18so that you have a bunch of things

71:20that have to be less than this, greater than that,

71:22or whatever, and you're trying to find a set of values

71:25for some decision variables that will maximize

71:29or minimize some objective function.

71:32So it's a way of solving a particular kind

71:35of optimization problem,

71:38a very formal sort of optimization problem,

71:40but one that's exceptionally useful.

71:42- And it specifies, so there's objective function

71:45constraints and variables that become separate

71:48from the data it operates on.

71:50- Right.

71:50- So that kind of separation allows you

71:54to put on different hats.

71:58One put the hat of an optimization person

72:00and then put another hat of a data person

72:03and dance back and forth,

72:04and also separate the actual solvers,

72:08the optimization systems that do the solving.

72:12Then you can have other people come to the table

72:14and then build their solvers,

72:15whether it's linear or non-linear,

72:17convex, non-convex, that kind of stuff.

72:21So what is the, to you as,

72:26maybe you can comment how you got into that world

72:30and what is the beautiful or interesting idea

72:33to you from the world of optimization?

72:35- Sure.

72:36So I preface it by saying I'm absolutely not an expert

72:39on this and most of the important work in AMPL

72:42comes from my two partners in crime on that,

72:45Bob Forer, who was a professor of,

72:48and in the industrial engineering and management

72:50science department at Northwestern,

72:52and my colleague at Bell Labs, Dave Gay,

72:54who was a numerical analyst and optimization person.

72:59So the deal is linear programming,

73:01preface this by saying.

73:03- Let's stay with linear programming.

73:05- Yeah, linear programming is the simplest example of this.

73:07So linear programming as it's taught in school

73:09is that you have a big matrix, which is always called A,

73:12and you say AX is less than or equal to B.

73:14So B is a set of constraints, X is the decision variables,

73:18and A is how the decision variables are combined

73:22to set up the various constraints.

73:24So A is a matrix and X and B are vectors.

73:28And then there's an objective function,

73:30which is just a sum of a bunch of Xs

73:31and some coefficients on them.

73:33And yet that's the thing you wanna optimize.

73:37The problem is that in the real world,

73:39that matrix A is a very, very, very intricate,

73:43very large and very sparse matrix

73:45where the various components of the model

73:47are distributed among the coefficients

73:50in a way that is totally unobvious to anybody.

73:53And so what you need is some way to express

73:58the original model, which you and I would write,

74:01you know, we'd write mathematics on the board,

74:03the sum of this is greater than the sum

74:04of that kind of thing.

74:06So you need a language to write those kinds of constraints.

74:10And Bob Forer for a long time had been interested

74:13in modeling languages,

74:14languages that made it possible to do this.

74:16There was a modeling language around called GAMS,

74:19the General Algebraic Modeling System,

74:21but it looked very much like Fortran, it was kind of clunky.

74:24And so Bob spent a sabbatical year at Bell Labs in 1984,

74:29and he and, he was in the office across from me,

74:32and it's always geography.

74:35And he and Dave Gay and I started talking

74:38about this kind of thing.

74:39And he wanted to design a language that would make it

74:43so that you could take these algebraic specifications,

74:46you know, summation, sines, oversets,

74:48and that you would write on the board

74:51and convert them into basically this A matrix,

74:55and then pass that off to a solver,

74:58which is an entirely separate thing.

75:00And so we talked about the design of the language.

75:05I don't remember any of the details of this now,

75:07but it's kind of an obvious thing.

75:08You're just writing mathematical expressions

75:11in a Fortran-like, or sorry,

75:13an algebraic but textual-like language.

75:15And I wrote the first version of this AMPL program,

75:21my first C++ program.

75:24(Lex laughs)

75:25And-

75:26- That's written in C++?

75:27- Yeah. - Oh, wow.

75:28- And so I did that fairly quickly.

75:30We wrote, it was, you know, 3,000 lines or something,

75:33so it wasn't very big,

75:34but it sort of showed the feasibility of it,

75:36that you could actually do something

75:37that was easy for people to specify models

75:41and convert it into something that a solver could work with.

75:44At the same time, as you say,

75:45the model and the data are separate things.

75:47So one model would then work

75:50with all kinds of different data

75:51in the same way that lots of programs do the same thing,

75:53but with different data.

75:54- So one of the really nice things

75:55is the specification of the models,

75:58human, just kind of like, as you say,

76:00it's human-readable.

76:01Like I literally, I remember on stuff I worked,

76:04I would send it to colleagues

76:07that I'm pretty sure never programmed in their life,

76:10just to understand what the optimization problem is.

76:15I think, how hard is it to convert that?

76:18You said there's a first prototype in C++,

76:20to convert that into something

76:22that could actually be used by the solver.

76:24- It's not too bad, because most of the solvers

76:26have some mechanism that lets them import a model in a form.

76:30It might be as simple as the matrix itself

76:32in just some representation,

76:35or if you're doing things that are not linear programming,

76:38then there may be some mechanism

76:39that lets you provide things like functions to be called,

76:43or other constraints on the model.

§10Transcript · 1:16:47–1:24:48

76:47So all Ample does is to generate that kind of thing,

76:51and then solver deals with all the hard work.

76:54And then when the solver comes back with numbers,

76:57Ample converts those back into your original form.

77:00So you know how much of each thing you should be buying,

77:03or making, or shipping, or whatever.

77:05So we did that in '84,

77:08and I haven't had a lot to do with it since,

77:11except that we wrote a couple of versions of a book on it.

77:14- Which is one of the greatest books ever written.

77:16I love that book.

77:17I don't know why.

77:20- It's an excellent book.

77:21Bob Forer wrote most of it.

77:22And so it's really, really well done.

77:24He must have been a dynamite teacher.

77:25- And typeset in LaTeX.

77:27- No, no, no, are you kidding?

77:29(laughing)

77:30I remember liking the typography, so I don't know.

77:32- We did it with T-Rof.

77:34- I don't even know what that is.

77:35- Yeah, exactly, you're too young.

77:37- Oh boy.

77:38- I think of T-Rof as a predecessor

77:42to the tech family of things.

77:44It's a formatter that was done at Bell Labs

77:46in this same period of the very early '70s

77:48that predates tech and things like that

77:52by five to 10 years.

77:54- But it was nevertheless, I'm going by memories.

77:58I remember it being beautiful.

78:00- Yeah, it was nicely done.

78:01- Outside of Unix, C, Ogg, Golang,

78:03all the things we talked about,

78:05all the amazing work you've done,

78:07you've also done work in graph theory.

78:09Let me ask this crazy out there question.

78:16If you had to make a bet,

78:17and I had to force you to make a bet,

78:19do you think P equals NP?

78:20(laughing)

78:23- The answer is no, although I'm told that somebody asked

78:26Jeff Dean if that was, under what conditions

78:28P would equal NP, and he said either P is zero or N is one.

78:32Or vice versa, I've forgotten.

78:35This is why Jeff Dean is a lot smarter than I am.

78:38- Yeah.

78:38So, but your intuition is--

78:41- I have no intuition, but I've got a lot of colleagues

78:45who've got intuition, and their betting is no.

78:48- That's the popular bet.

78:51Okay, so what is computational complexity theory,

78:55and do you think these kinds of complexity classes,

78:58especially as you've taught in this modern world,

79:01are still a useful way to understand

79:04the hardness of problems?

79:06- I don't do that stuff.

79:07The last time I touched anything to do with that--

79:09- Many, many years ago.

79:10- Was before it was invented.

79:12'Cause I, it's literally true.

79:14I did my PhD thesis on graph--

79:17- Before big O notation.

79:18- Oh, absolutely, before, I did this in 1968,

79:24and I worked on graph partitioning,

79:25which is this question, you've got a graph

79:27that is a nodes and edges kind of graph,

79:30and the edges have weights,

79:31and you just wanna divide the nodes into two piles

79:34of equal size, so that the number of edges

79:36that goes from one side to the other

79:38is as small as possible.

79:39And we--

79:41- You developed, so that problem is hard?

79:45- Well, as it turns out, I worked with Shen Lin

79:48at Bell Labs on this, and we were never able to come up

79:52with anything that was guaranteed to give the right answer.

79:54We came up with heuristics that worked pretty darn well,

79:57and I peeled off some special cases for my thesis,

80:01but it was just hard.

80:02And that was just about the time that Steve Cook

80:04was showing that there were classes of problems

80:06that appeared to be really hard,

80:08of which graph partitioning was one.

80:10But this, my expertise, such as it was,

80:13totally predates that development.

80:16- Oh, interesting, so the heuristic,

80:18which now, it carries the two of yours names

80:21for the traveling salesman problem,

80:23and for the graph partitioning, that was,

80:26like, how did you, you weren't even thinking

80:28in terms of classes, you were just trying to find--

80:29- There was no such idea.

80:31- A heuristic that kinda does the job pretty well.

80:34- You were trying to find something that did the job,

80:36and there was nothing that you would call,

80:38let's say, a closed form or algorithmic thing

80:41that would give you a guaranteed right answer.

80:44I mean, compare graph partitioning to max flow min cut,

80:48or something like that.

80:50That's the same problem, except there's no constraint

80:52on the number of nodes on one side or the other of the cut.

80:56And that means it's an easy problem,

80:58at least as I understand it.

80:59Whereas the constraint that says

81:01the two have to be constrained in size

81:03makes it a hard problem.

81:05- Yeah, so Robert Frost has that poem

81:07where you have to choose two paths.

81:09So why did you, is there another alternate universe

81:13in which you pursued the Don Knuth path

81:16of algorithm design, sort of--

81:19- Not smart enough.

81:21- Not smart enough.

81:22You're infinitely modest,

81:27but so you pursued your kind of love of programming.

81:30I mean, when you look back to those,

81:33I mean, just looking into that world,

81:35does that just seem like a distant world

81:37of theoretical computer science?

81:40Then is it fundamentally different

81:42from the world of programming?

81:44- I don't know.

81:45I mean, certainly, in all seriousness,

81:47I just didn't have the talent for it.

81:49When I got here as a grad student at Princeton

81:51and I started to think about research

81:53at the end of my, I don't know, first year

81:55or something like that,

81:56I worked briefly with John Hopcroft,

81:59who is absolutely, you know,

82:00you mentioned during award winner, et cetera,

82:02a great guy.

82:04And it became crystal clear

82:05I was not cut out for this stuff, period.

82:08Okay.

82:09And so I moved into things

82:11where I was more cut out for it.

82:13And that tended to be things like writing programs

82:16and then ultimately writing books.

82:19- You've said that in Toronto,

82:22as an undergrad, you did a senior thesis

82:24or a literature survey on artificial intelligence.

82:28This was 1964.

82:30- Correct.

82:32- What was the AI landscape, ideas, dreams at that time?

82:37- I think that was one of the,

82:39well, you've heard of AI winters.

82:40This is whatever the opposite was,

82:41AI summer or something.

82:43It was one of these things

82:44where people thought that,

82:46boy, we could do anything with computers,

82:49that all these hard problems,

82:51computers will solve them.

82:52They will do machine translation.

82:54They will play games like chess.

82:57They will do, you know,

83:00prove theorems in geometry.

83:02There are all kinds of examples like that

83:04where people thought,

83:05boy, we could really do those sorts of things.

83:08And, you know, I read the Kool-Aid in some sense.

83:14There's a wonderful collection of papers

83:16called "Computers and Thought"

83:18that was published in about that era.

83:20And people were very optimistic.

83:22And then of course it turned out that

83:24what people had thought was just a few years down the pike

83:27was more than a few years down the pike.

83:31And some parts of that are more or less now

83:34sort of under control.

83:36We finally do play games like go and chess and so on

83:39better than people do,

83:41but there are others,

83:42and machine translation is a lot better than it used to be,

83:45but that's, you know, 50, close to 60 years of progress

83:49and a lot of evolution in hardware

83:51and a tremendous amount more data

83:52upon which you can build systems

83:56that actually can learn from some of that data.

83:58- And the infrastructure to support developers

84:02working together like an open source movement,

84:05the internet period is also an empowering.

84:08But what lessons do you draw from that,

84:11the opposite of winter, that optimism?

84:13- Well, I guess the lesson is that in the short run,

84:19it's pretty easy to be too pessimistic

84:23or maybe too optimistic,

84:24and in the long run, you probably shouldn't be too pessimistic.

84:27I'm not saying that very well.

84:28It reminds me of this remark from Arthur Clarke,

84:32a science fiction author, who says,

84:34"When some distinguished but elderly person

84:36"says that something is possible, he's probably right.

84:41"And if he says it's impossible, he's almost surely wrong."

84:44But you don't know what the timescale is.

84:45- The timescale is critical, right.

84:48So what are your thoughts on this new summer of AI

§11Transcript · 1:24:52–1:33:47

84:52now in the work with machine learning and neural networks?

84:55You've kind of mentioned that you started to try to explore

84:57and look into this world that seems fundamentally different

85:01from the world of heuristics and algorithms like search,

85:06that it's now purely sort of trying to take

85:08huge amounts of data and learn from that data, right?

85:12Programs from the data.

85:14- Yeah.

85:14Look, I think it's very interesting.

85:17I am incredibly far from an expert.

85:19Most of what I know I've learned from my students,

85:21and they're probably disappointed

85:24in how little I've learned from them.

85:26But I think it has tremendous potential

85:29for certain kinds of things.

85:30I mean, games is one where it obviously has had an effect

85:34on some of the others as well.

85:36I think there's, and this is speaking from

85:39definitely not expertise,

85:40I think there are serious problems in certain kinds

85:43of machine learning, at least,

85:45because what they're learning from

85:47is the data that we give them.

85:49And if the data we give them has something wrong with it,

85:52then what they learn from it is probably wrong too.

85:54And the obvious thing is some kind of bias in the data.

85:59That the data has stuff in it like, I don't know,

86:02women aren't as good as men at something, okay?

86:05That's just flat wrong.

86:07But if it's in the data because of historical treatment,

86:11then that machine learning stuff will propagate that.

86:15And that is a serious worry.

86:17- The positive part of that is what machine learning does

86:22is reveal the bias in the data

86:24and puts a mirror to our own society.

86:27And in so doing, helps us remove the bias,

86:30you know, helps us work on ourselves.

86:33- Puts a mirror to ourselves.

86:35- Yeah, that's an optimistic point of view.

86:37And if it works that way, that would be absolutely great.

86:40And what I don't know is whether it does work that way

86:42or whether the AI mechanisms

86:46or machine learning mechanisms reinforce

86:48and amplify things that have been wrong in the past.

86:52And I don't know, but I think that's a serious thing

86:56that we have to be concerned about.

86:58- Let me ask you an out there question, okay?

87:01I know nobody knows, but what do you think it takes

87:03to build a system of human level intelligence?

87:07That's been the dream from the '60s.

87:09We talk about games, about language,

87:12about image recognition,

87:15but really the dream is to create human level

87:18or superhuman level intelligence.

87:19What do you think it takes to do that?

87:21And are we close?

87:23- I haven't a clue and I don't know, roughly speaking.

87:26I mean, this was Turing--

87:27- I was trying to trick you into a hypothesis.

87:30- Yeah, I mean, Turing talked about this

87:31in his paper on machine intelligence back in,

87:34geez, I don't know, early '50s or something like that.

87:36And he had the idea of the Turing test.

87:38And I don't know whether the Turing test is--

87:40- Is a good test of intelligence.

87:41- I don't know, it's an interesting test.

87:43At least it's in some vague sense objective,

87:45whether you can read anything into the conclusions

87:48is a different story.

87:50- Do you have worries, concerns, excitement

87:55about the future of artificial intelligence?

87:56So there's a lot of people who are worried.

87:58And you can speak broadly than just artificial intelligence.

88:01It's basically computing taking over the world

88:05in various forms.

88:06Are you excited by this future,

88:09this possibility of computing being everywhere?

88:12Or are you worried?

88:13- It's some combination of those.

88:16I think almost all technologies over the long run

88:21are for good, but there's plenty of examples

88:24where they haven't been good,

88:26either over a long run for some people

88:28or over a short run.

88:30And computing is one of those.

88:33And AI within it is gonna be one of those as well.

88:36But computing broadly, I mean,

88:38for just a today example is privacy.

88:41That the use of things like social media and so on means that

88:46and the commercial surveillance means that

88:49there's an enormous amount more known about us by people,

88:54other businesses, government, whatever,

88:56than perhaps one ought to feel comfortable with.

88:59So that's an example.

89:01- So that's an example of a possible negative effect

89:07of computing being everywhere.

89:09It's an interesting one 'cause it could also be a positive

89:14if leveraged correctly.

89:16- There's a big if there.

89:18- So I have a deep interest in human psychology

89:22and humans seem to be very paranoid about this data thing.

89:27But that varies depending on age group.

89:31It seems like the younger folks,

89:32so it's exciting to me to see what society looks like

89:3550 years from now,

89:37that the concerns about privacy might be flipped

89:39on their head based purely on human psychology

89:42versus actual concerns or not.

89:45What do you think about Moore's law?

89:49Well, you said a lot of stuff we've talked,

89:52you talked about programming languages in their design,

89:55in their ideas come from the constraints

89:58in the systems they operate in.

90:00Do you think Moore's law,

90:01the exponential improvement of systems

90:07will continue indefinitely?

90:08There's a mix of opinions on that currently.

90:12Or do you think there'll be a plateau?

90:17- Well, the frivolous answer is no exponential

90:21can go on forever.

90:22You run out of something.

90:26Just as we said, timescale matters.

90:27So if it goes on long enough, that might be all we need.

90:30- Yeah, right, won't matter to us.

90:33So I don't know,

90:34we've seen places where Moore's law has changed.

90:35For example, mentioned earlier,

90:37processors don't get faster anymore,

90:41but you use that same growth of,

90:45the ability to put more things in a given area

90:48to go horizontally instead of vertically as it were.

90:51So you can get more and more processors

90:52or memory or whatever on the same chip.

90:55Is that gonna run into a limitation?

90:57Presumably, because at some point

91:00you get down to the individual atoms.

91:03And so you gotta find some way around that.

91:05Will we find some way around that?

91:07I don't know, I just said that if I say it won't,

91:10I'll be wrong.

91:10So perhaps we will.

91:12- So I just talked to Jim Keller and he says,

91:15so he actually describes,

91:16he argues that the Moore's law will continue

91:18for a long, long time because you mentioned the atom.

91:21We actually have, I think a thousand fold increase,

91:25decreased in transistor size still possible

91:29before we get to the quantum level.

91:32So there's still a lot of possibilities.

91:34He thinks it'll continue indefinitely,

91:36which is an interesting, optimistic viewpoint.

91:40But how do you think the programming languages

91:43will change with this increase?

91:45Whether we hit a wall or not,

91:47what do you think?

91:49Do you think there'll be a fundamental change

91:51in the way programming languages are designed?

91:54- I don't know about that.

91:55I think what will happen is continuation

91:58of what we see in some areas at least,

92:02which is that more programming will be done

92:05by programs than by people.

92:08And that more will be done by sort of declarative

92:12rather than procedural mechanisms where I say,

92:15I want this to happen, you figure out how.

92:19And that is in many cases at this point,

92:24domain of specialized languages for narrow domains,

92:28but you can imagine that broadening out.

92:31And so I don't have to say so much in so much detail,

92:35some collection of software, let's call it languages

92:39or programs or something,

92:41we'll figure out how to do what I want to do.

92:44- Interesting, so increased levels of abstraction.

92:47- Yeah.

92:48- And one day getting to the human level

92:50where we can just use natural language.

92:52- Could be possible.

92:54- So you taught, still teach a course,

92:56Computers in Our World here at Princeton

92:59that introduces computing and programming to non-majors.

93:03Just from that experience, what advice do you have

93:07for people who don't know anything about programming

93:10but are kind of curious about this world

93:12where programming seems to become more and more

93:14of a fundamental skill that people need

93:17to be at least aware of?

93:18- Well, I can recommend a good book.

93:20- What's that?

93:21(laughing)

93:22- The book I wrote for the course.

93:24I think this is one of these questions

93:26of should everybody know how to program?

93:28And I think the answer is probably not,

93:31but I think everybody should at least understand

93:32sort of what it is so that if you say to somebody,

93:35I'm a programmer, they have a notion of what that might be,

93:38or if you say this is a program,

93:40or this was decided by a computer running a program,

93:43that they have some vague intuitive understanding

93:47and accurate understanding of what that might imply.

§12Transcript · 1:33:51–1:42:51

93:51So part of what I'm doing in this course,

93:55which is very definitely for non-technical people,

93:57I mean, a typical person in it is a history

93:59or English major, try and explain how computers work,

94:03how they do their thing, what programming is,

94:06how you write a program,

94:07and how computers talk to each other,

94:11and what do they do when they're talking to each other.

94:14And then I would say nobody, very rarely,

94:19and does anybody in that course go on

94:21to become a real serious programmer,

94:24but at least they've got a somewhat better idea

94:27of what all this stuff is about, not just the programming,

94:29but the technology behind computers and communications.

94:32- Do they try and write a program themselves?

94:35- Oh yeah, yeah, a very small amount.

94:38I introduced them to how machines work

94:40at a level below high-level languages,

94:43so we have a kind of a toy machine

94:45that has a very small repertoire, a dozen instructions,

94:47and they write trivial assembly language programs for that.

94:51- Really? - Yeah.

94:51- Wow, so can you, just, if you were to give a flavor

94:55to people of the programming world, of the computing world,

94:59what are the examples they should go with?

95:01So a little bit of assembly to get a sense

95:04at the lowest level of what the program is really doing?

95:08- Yeah, I mean, in some sense,

95:10there's no such thing as the lowest level,

95:12because you can keep going down,

95:13but that's the place where I drew the line.

95:15So the idea that computers have a fairly small repertoire

95:19of very simple instructions that they can do,

95:21like add and subtract and branch and so on,

95:25as you mentioned earlier,

95:26and that you can write code at that level,

95:31and it will get things done,

95:33and then you have the levels of abstraction

95:35that we get with higher-level languages,

95:37like Fortran or C or whatever,

95:39and that makes it easier to write the code

95:42and less dependent on particular architectures.

95:44And then we talk about a lot of the different kinds

95:48of programs that they use all the time

95:50that they don't probably realize are programs,

95:52like they're running Mac OS on their computers

95:57or maybe Windows, and they're downloading apps

96:00on their phones, and all of those things are programs

96:03that are just what we just talked about,

96:05except at a grand scale.

96:08- Yeah, it's easy to forget that they're actual programs

96:10that people program.

96:11There's engineers that wrote those things.

96:14- Yeah, right, and so in a way,

96:17I'm expecting them to make an enormous conceptual leap

96:20from their five or 10-line toy assembly language thing

96:24that adds two or three numbers to something

96:28that is a browser on their phone or whatever,

96:31but it's really the same thing.

96:33- So if you look in broad strokes at history,

96:37what do you think the world,

96:39how do you think the world changed because of computers?

96:42It's hard to sometimes see the big picture

96:45when you're in it, but I guess I'm asking

96:48if there's something you've noticed over the years

96:51that, like you were mentioning,

96:54the students are more distracted looking at their,

96:56now there's a device to look at.

96:58- Right, well, I think computing has changed

97:00a tremendous amount, obviously,

97:02but I think one aspect of that is the way

97:04that people interact with each other,

97:06both locally and far away, and when I was the age

97:11of those kids, making a phone call to somewhere

97:14was a big deal because it cost serious money,

97:17and this was in the '60s, right?

97:19And today, people don't make phone calls,

97:22they send texts or something like that,

97:25so there's an up and down in what people do.

97:29People think nothing of having correspondence,

97:34regular meetings, video, whatever, with friends

97:37or family or whatever in any other part of the world,

97:40and they don't think about that at all.

97:43And so that's just the communication aspect of it.

97:47- Do you think that brings us closer together,

97:50or does it make us, does it take us away

97:55from the closeness of human-to-human contact?

97:59- I think it depends a lot on all kinds of things.

98:02So I trade mail with my brother and sister in Canada

98:05much more often than I used to talk to them on the phone.

98:08So probably every two or three days,

98:10I get something or send something to them,

98:14whereas 20 years ago, I probably wouldn't have talked

98:18to them on the phone nearly as much.

98:20So in that sense, that's brought my brother

98:22and sister and I closer together.

98:24That's a good thing.

98:25I watch the kids on campus, and they're mostly walking

98:29around with their heads down, fooling with their phones

98:32to the point where I have to duck them.

98:34I don't know that that has brought them closer together

98:39in some ways.

98:40There's sociological research that says people are,

98:43in fact, not as close together as they used to be.

98:46I don't know whether that's really true,

98:47but I can see potential downsides in kids

98:52where you think, "Come on, wake up

98:54"and smell the coffee," or whatever.

98:56- That's right, but if you look at, again,

98:59nobody can predict the future, but are you excited

99:02and I kind of touched this a little bit with AI,

99:04but are you excited by the future in the next 10, 20 years

99:08that computing will bring?

99:11You were there when there was no computers, really,

99:15and now computers are everywhere, all over the world,

99:19in Africa and Asia, and just every person,

99:23almost every person in the world has a device.

99:25So are you hopeful, optimistic about that future?

99:30- It's mixed, if the truth be told.

99:32I mean, I think there are some things about that

99:34that are good.

99:34I think there's the potential for people

99:36to improve their lives all over the place,

99:39and that's obviously good.

99:40And at the same time, at least in the short run,

99:44you can see lots and lots of bad

99:45as people become more tribalistic

99:48or parochial in their interests,

99:50and it's an enormous amount more us and them,

99:53and people are using computers in all kinds of ways

99:55to mislead or misrepresent or flat out lie

99:58about what's going on, and that is affecting politics

100:01locally and, I think, everywhere in the world.

100:04- Yeah, the long-term effect on political systems

100:08and so on, it's who knows.

100:10- Who knows, indeed.

100:11(Lex laughing)

100:14- People now have a voice, which is a powerful thing.

100:18People who are oppressed have a voice,

100:21but also everybody has a voice,

100:24and the chaos that emerges from that is fascinating to watch.

100:26- Yeah, yeah, it's kind of scary.

100:29- If you can go back and relive a moment in your life,

100:33one that made you truly happy outside of family,

100:37or was profoundly transformative,

100:40is there a moment or moments that jump out at you

100:44from memory?

100:45- I don't think specific moments.

100:48I think there were lots and lots and lots of good times

100:50at Bell Labs where you would build something,

100:52and it worked.

100:55Huh, Jason, it worked.

100:56- So the moment it worked.

100:57- Yeah, and somebody used it, and they said,

101:00"Gee, that's neat."

101:01Those kinds of things happened quite often

101:04in that sort of golden era in the '70s

101:07when Unix was young, and there was all this

101:10low-hanging fruit and interesting things to work on,

101:13and a group of people who kind of,

101:16we were all together in this, and if you did something,

101:18they would try it out for you.

101:20And I think that was, in some sense,

101:22a really, really good time.

101:24- And awk was awk an example of that,

101:27that when you built it, then people used it?

101:29- Yeah, absolutely.

101:30- And now millions of people use.

101:32- And all your stupid mistakes are right there

101:34for them to look at, right?

101:36So it's mixed.

101:37- Yeah, it's terrifying and vulnerable,

101:39but it's beautiful 'cause it does have a positive impact

101:42on so, so many people.

101:43So I think there's no better way to end it.

101:47Brian, thank you so much for talking to me.

101:48It was an honor.

101:49- Okay, my pleasure.

101:51Good fun.

101:53- Thank you for listening to this conversation

101:55with Brian Kernighan, and thank you to our sponsors,

101:588sleep Mattress and Raycon Earbuds.

102:02Please consider supporting this podcast

102:05by going to 8sleep.com/lex

102:08and to buy raycon.com/lex.

102:12Click the links, buy the stuff.

102:14These both are amazing products.

102:16It really is the best way to support this podcast

102:19and the journey I'm on.

102:21It's how they know I sent you and increases the chance

102:24that they'll actually support this podcast in the future.

102:27If you enjoy this thing, subscribe on YouTube,

102:30review it with Firestarz and Apple Podcast,

102:32support it on Patreon, or connect with me on Twitter

102:35at Lex Friedman, spelled somehow, miraculously,

102:40without the letter E, just F-R-I-D-M-A-N,

102:44because when we immigrated to this country,

102:46we were not so good at spelling.

102:49And now let me leave you with some words

102:51from Brian Kernighan.

§13Transcript · 1:42:53–1:43:03

102:53Don't comment bad code, rewrite it.

102:56Thank you for listening and hope to see you next time.

103:00(upbeat music)

103:03(upbeat music)

— 读到这里,多懂一点。 —