Vladimir Vapnik: Statistical Learning

Lex Fridman introduces Vladimir Vapnik as co-inventor of the support vector machine, support vector clustering and VC theory, with many foundational ideas in statistical learning. Born in the Soviet Union, he worked at the Institute of Control Sciences in Moscow, then at AT&T, NEC Labs and Facebook Research in the United States, and is now a professor at Columbia; his work has been cited over 170,000 times.

Asked whether God plays dice, Vapnik says it only looks that way because we do not know some important factors. He distinguishes instrumentalism, which creates theory for prediction, from realism, which tries to understand what God did; a mechanical law can be read either as true always and everywhere or as merely allowing prediction of a moving element's position.

He applies this to machine learning. The instrumentalist view, which he says everyone takes now, holds that the goal is to find a classification rule, an instrument for prediction. The alternative is that the goal is to learn conditional probability: for prediction one needs only the rule, but for understanding one needs conditional probability.

Fridman mentions Eugene Wigner's 1960 paper on the unreasonable effectiveness of mathematics and asks whether math is a tool or poetry. Vapnik cites the view that math is the language God uses, and says mathematical structures know something about reality; looking carefully at the equations that define conditional probability reveals more about reality than fantasy does.