David Deutsch - AI, America, Fun, & Bayes

Dwarkesh Patel opens by asking Deutsch to explain why he expects AIs to be no more fundamentally intelligent than humans. Deutsch first pins down the terms. He takes the question to be about AGIs, and "fundamentally intelligent" to mean capable, in principle, of every type of cognition humans have: science, art, falling in love, being good or evil.

His argument has two halves. On hardware, he says human brains are Turing-complete, so they can in principle run any computable program. He immediately qualifies "any". Two people talking will both be dead in roughly a hundred years, so the number of conversations they can have is finite, and some computations, such as many traveling salesman problems, could not be finished within the age of the universe. What he means is that humans are not limited in which programs they can run, apart from speed and memory capacity. Both of those can be augmented to the level of any other entity in the universe, because anyone who builds a computer that thinks faster than a brain could use that same technology to speed up human thinking.

On software, the question is whether humans can reach the same kinds of explanations as any other entity. Deutsch notes this is usually raised about extraterrestrials, and also about AGIs that might be to us as we are to ants. He treats the speed and memory part as fixable with more hardware. What remains is whether there are "concepts that we are inherently incapable of comprehending". He attributes the view that there are to Martin Rees, who thinks humans can grasp quantum mechanics and apes cannot, and that extraterrestrials might grasp something beyond it that no add-on hardware could give us.

Deutsch applies the same reasoning to qualia such as love. If specialized hardware were needed for love or mathematical insight, that hardware would still be a part of the brain connected to the rest by neurons passing electrical signals and by chemicals whose concentrations change. An artificial device that computed which signals to send and which chemicals to adjust could do the same job indistinguishably. A person born without that hardware could therefore feel love after such augmentation. On this view, the only relevant limits are speed and memory, which is why AGIs and humans have the same range in the sense he defined.

Patel says the software question is the more interesting one, though he flags that he disagrees that human memory and speed can be expanded arbitrarily and easily, and sets that aside for later. He then asks whether all humans can explain everything the smartest humans can. He cites the National Adult Literacy survey, where he says about 21 to 24 percent of Americans fall in level one, unable to do tasks such as identifying a driver's license expiry date or totaling a bank deposit slip. If such people cannot create the Deutsch–Jozsa algorithm, he asks, does the Theory of Universal Explainers fall apart?

Deutsch replies that some humans are brain-damaged to the point that they cannot do tasks an ape can do, so there is a level of dysfunction below which hardware is the limit. We do not yet know enough about the brain, he says, but a figure as large as 24 percent of the population cannot be hardware, so for those people it must be software. If it were hardware, the fix would be repairing it. If it is software, the issue is not simply wanting to be taught but whether the existing software is conceptually ready for the new program.

He illustrates with Brett Hall, who often says he would like to speak Mandarin but will never want it enough to go through the process of acquiring that program. Nothing in Hall's hardware or software prevents it. Deutsch adds that many of his relatives, a couple of generations ago, were forced to migrate to an alien place and quickly learned languages they never expected to speak. People who cannot read driving licenses could likewise learn, in his phrasing, if they wanted to be educated in the way his ancestors wanted to learn languages.

Asked whether apes could be programmed with a full human intellect, Deutsch answers yes, in principle. It would require intricate changes at the neuron level to transfer a program from a human mind to the ape's. He guesses it is possible because the ape's smaller memory and lack of specialized modules do not matter much, since humans do not use their full capacity anyway. As an example, his piano playing and driving knowledge sit unused during this conversation. He adds that the project would be highly immoral, because it would intentionally create a person inside deficient brain hardware.