From Philosophy to Math to Engineering
Cross-posted from the MIRI blog.
For centuries, philosophers wondered how we could learn what causes what. Some argued it was impossible, or possible only via experiment. Others kept hacking away at the problem, clarifying ideas like counterfactual and probability and correlation by making them more precise and coherent.
Then, in the 1990s, a breakthrough: Judea Pearl and others showed that, in principle, we can sometimes infer causal relations from data even without experiment, via the mathematical machinery of probabilistic graphical models.
Next, engineers used this mathematical insight to write software that can, in seconds, infer causal relations from a data set of observations.
Across the centuries, researchers had toiled away, pushing our understanding of causality from philosophy to math to engineering.
And so it is with Friendly AI research. Current progress on each sub-problem of Friendly AI lies somewhere on a spectrum from philosophy to math to engineering.
We began with some fuzzy philosophical ideas of what we want from a Friendly AI (FAI). We want it to be benevolent and powerful enough to eliminate suffering, protect us from natural catastrophes, help us explore the universe, and otherwise make life awesome. We want FAI to allow for moral progress, rather than immediately reshape the galaxy according to whatever our current values happen to be. We want FAI to remain beneficent even as it rewrites its core algorithms to become smarter and smarter. And so on.
Small pieces of this philosophical puzzle have been broken off and turned into math, e.g. Pearlian causal analysis and Solomonoff induction. Pearl’s math has since been used to produce causal inference software that can be run on today’s computers, whereas engineers have thus far succeeded in implementing (tractable approximations of) Solomonoff induction only for very limited applications.
Toy versions of two pieces of the “stable self-modification” problem were transformed into math problems in de Blanc (2011) and Yudkowsky & Herreshoff (2013), though this was done to enable further insight via formal analysis, not to assert that these small pieces of the philosophical problem had been solved to the level of math.
Thanks to Patrick LaVictoire and other MIRI workshop participants,1 Douglas Hofstadter’s FAI-relevant philosophical idea of “superrationality” seems to have been, for the most part, successfully transformed into math, and a bit of the engineering work has also been done.
I say “seems” because, while humans are fairly skilled at turning math into feats of practical engineering, we seem to be much less skilled at turning philosophy into math, without leaving anything out. For example, some very sophisticated thinkers have claimed that “Solomonoff induction solves the problem of inductive inference,” or that “Solomonoff has successfully invented a perfect theory of induction.” And indeed, it certainly seems like a truly universal induction procedure. However, it turns out that Solomonoff induction doesn’t fully solve the problem of inductive inference, for relatively subtle reasons.2
Unfortunately, philosophical mistakes like this could be fatal when humanity builds the first self-improving AGI (Yudkowsky 2008).3 FAI-relevant philosophical work is, as Nick Bostrom says, “philosophy with a deadline.”
1 And before them, Moshe Tennenholtz.
2 Yudkowsky plans to write more about how to improve on Solomonoff induction, later.
3 This is a specific instance of a problem Peter Ludlow described like this: “the technological curve is pulling away from the philosophy curve very rapidly and is about to leave it completely behind.”
Patrick Suppes to the left?
I would buy a modapuccino¹ if we lived in a 100km radius!
Modafinil’s version of a cappuccino
Well, I hope you’re in Oxford soon again, João! :)
aah, it’s you! :) We might indeed go for a fika sometime in the near future then. I wonder if you were following the Swedish-transhumanism-FHI squabble on my last post.
Yes! Less Wrong, please give vallinder extra karma, for identifying the more ambiguous cartoon dude. :)
Ah, so another philosopher that I originally thought was a mathematician.
From his wikipedia page:
Sounds like either self-advertisement or a hero-worship, depending on who wrote it.
I’m confused. Why does it sound like that to you? Saying someone has made “significant contributions” doesn’t sound excessively adulatory, especially if you’re talking about someone who is widely regarded as having made significant contributions (as evidenced by, say, his National Medal of Science).
First, I never heard of Patrick Suppes in any of my physics studies. Or at all, until he was mentioned here. So this puts a cap on the significance of his contributions to the “foundations of physics”. Second, the medal is “National Medal of Science for Behavioral and Social Science”, not anything physics-related (unless you consider “subjective probability” physics). Now, my attitude toward “philosophy of science” roughly matches that of Feynman, so I am not going to touch that one. What’s left is “the theory of measurement”, which am not in a position to evaluate, except to say that, from the list of relevant publications it seems like he worked in the decision-theoretic area, rather than any physical measurement. Now, maybe he did make “significant contributions” to decision theory, I cannot tell, except that this forum does not mention him much while discussing decision-theoretic issues.
Not to put him down, he did a lot of interesting work and is likely in the top 0.1% of physics PhDs by various metrics, just not as “foundational” as the wiki article presents him.
I’m not familiar with his writings on the foundations of quantum mechanics, but in addition to his work on causality, the three volumes on measurement he co-authored have also been hugely influential. His intellectual autobiography (pdf) might be worth a look.
I think you might have finally got it. I hope so at least, this google image madness has to stop.
The ‘robust cooperation’ paper is very interesting, thanks! Is there an outline of it for those of us who would like to learn about it but do not have much time to spare?
Well you could read the associated post.
If this isn’t what you are looking for, and you clarify what you are looking for, I could probably write some sort of outline.
http://leiterreports.typepad.com/blog/2013/10/progress-in-philosophy-revisited.html
Wait a minute, there’s a senior philosopher, independently from MIRI/FHI, going around giving talks about how we should invest effort and money on philosophically addressing the most important problems? Why aren’t we funding this? He should be ‘lobbying’ universities’ deans and whatnot.
Also, you just mistakenly pictured Russell while attempting to draw another guy. Left dude IS Russell.
The source material sent to the odesk cartoonist was a photo of Suppes, believe it or not. :)
He migh have used this one http://www.stanford.edu/group/cslipublications/cslipublications/cover/Ferrario-cover-web.jpg ? Seems to be also mirrored.
Out of curiosity, who are the left and right dudes supposed to be?
The one on the left looks a bit like Bertrand Russell.
Left-hand dude looks a bit like Immanuel Kant. (I’d have thought David Hume was a more likely choice, but he didn’t look much like that.)
[EDITED to clarify.]
The attire is much too modern to be Kant.
That’s very true.
Karma points to whoever identifies them correctly!
The guy on the right is more accurately represented than the guy on the left, but the guy on the left may still be guessable based on the subject matter. Hint: the guy on the left is a philosopher known, among other things, for his post-1950 contributions to theory of causality.
(The guy in the middle is Judea Pearl, Ilya’s advisor at UCLA.)
Update: dougclow correctly identified the man on the right, and vallinder correctly identified the man on the left.
Guy on the right is Markus Kalisch.
Not sure about the one on the left—outside chance it’s Bertrand Russell but probably not.
Give the man some points!
Guy on the right is indeed Markus Kalisch.
The guy on the left is not Bertrand Russell.
The man on the left is Hans Reichenbach.
Nope!
Richard Jeffrey!
Nope, Suppes.
That was one of my first guesses, but it doesn’t seem to look much like him.
Perhaps the artist used as reference this photo from the obituary of a different man named Hans Reichenbach.
On the left is Willard Quine.
Nope!
How sure are you? It seems plausible that Quine looked like that at one time, but none of the pictures of him I can find look all that similar. And while Quine did important work on many many things, I wouldn’t put causality high on the list of things he’s known for working on.
Still, I’ve so far failed to find anyone notable in the field who looks much more like that image...
Is the guy on the left Sewall Wright?
Whoops, you said post-1950s.
Also Sewall Wright was no philosopher—he was a biologist. Faced with a problem on the left, he basically correctly formalized and solved it for the linear/Gaussian case. He should be in the middle with Judea and on the right with engineers!
Guy on the left is Max Born?
Nope.
Left is Russell. Haircut.
A fine guess, but no.
Meh. Right is Markus Kalisch?
dougclow narrowly beat you to it.
Mirrored Russell? His hair divides on the right, not left. But the subject matter is Russellian.