Look, if someone came to me and said, “I’m great at LW-style philosophy, and the proof of this is, I can argue there’s no libertarian free will” I would reply “You have not yet done any difficult or worthwhile cognitive work.” It’s like saying you don’t believe in astrology. Well, great, and yes there’s lots of people who disagree with you about that, but there’s a difference between doing grade school arithmetic and doing calculus, and “There is no libertarian free will” is grade school arithmetic. It doesn’t interest me that this philosophical school agrees with me about that. It’s too simple and basic, and part of what I object to in philosophy is that they are still arguing about problems like this instead of moving onto real questions.
I don’t get it. Your comment here doesn’t respond to anything I said in my previous comment. The first sentence of my previous comment is: “I did not put you in the Quinean camp merely because of your agreement about libertarian free will.”
I think Eliezer is suggesting that all the things you’ve mentioned that distinguish Quinean naturalists from other philosophers are similarly basic, and that “LW-style philosophy” takes (what turns out to be) Quinean naturalism as a starting point and then goes on to do things that no one working in mainstream philosophy has thought of.
In other words, that the problem with mainstream philosophy isn’t that it’s all wrong, but that much of it is wrong and that the part that isn’t wrong is mostly not doing anything interesting with its not-wrongness.
(I make no comment on whether all, or some, or none, of that is correct. I’m just hoping to reduce the amount of talking-past-one-another here.)
Eliezer is suggesting that the Quineans are “not doing anything interesting with [their] not-wrongness” after being aware of the field for all of an hour and a half?!
Makes perfect sense to me. Someone comes up to me and says “This person is a brilliant mathematician! She just showed me a proof that there’s no highest prime, and proved Pythagoras’ theorem!” my response would be that that’s still no evidence that she’s made any worthwhile contribution to mathematics. She may have, but there’s little reason to believe it from the original statement.
Seems to me less like that and more like, “this Euclid fellow was brilliant”, followed by a list of things that Euclid proved before anybody else proved. Timing matters here. It’s no coincidence that before Quine came along, the clever Eliezers were not taking Quinean naturalism for granted.
For another analogy, if someone came along and told you, “this Hugh Everett fellow was brilliant! Here, read this paper in which he argues that the wave function never collapses”, would you say, “well, Eliezer already went through that a few years ago; there’s still no evidence that Everett made any worthwhile contribution”?
I did not come to this conclusion on the basis of having read the claim somewhere. Rather, it’s what I gather from having read philosophy from both before and after Quine. If clever men were coming up with Quinean insight left and right before Quine appeared, then we should see a large number of philosophers pre-Quine who make Quine redundant. I don’t recall encountering any of these philosophers whose existence would virtually be assured if I were wrong. But suppose that I am simply ignorant. We still have Quine’s reputation to content with, the wide acknowledgment by major philosophers that he was highly influential. If I were wrong, he should have been lost in a sea of bright young men who anticipated his key insights.
“If clever men were coming up with Quinean insight left and right before Quine appeared, then we should see a large number of philosophers pre-Quine who make Quine redundant.”
Assuming also that those ‘clever men’ were going into philosophy rather than dismissing it as Eliezer has.
Eliezer may say that he dismisses philosophy, but he has nevertheless published a great deal which takes issue with some philosophy, agrees with other philosophy, and most importantly, he has provided a great deal of argumentation in favor of these conclusions which some philosophers agree with and other philosophers disagree with. Whether he believes it or not, Eliezer is doing philosophy, and a lot of it.
So, where are these clever men pre-Qune who dismissed philosophy and then proceeded, as Eliezer has done, to produce reams of it?
There are a few, e.g. E.T. Jaynes, Alfred North Whitehead (“Philosophy begins in wonder. And, at the end, when philosophic thought has done its best, the wonder remains. ”), and Richard Feynman (over and over and over again.)
More generally, though, those ‘clever men’ have tended to ignore philosophy and charge ahead with whatever they’re doing; it’s just that Eliezer’s work has tended to impinge more on philosophy than, say, themodynamics experiments or calculus proofs.
ETA: This didn’t actually address what Constant meant; my apologies.
Well, you did answer the question I asked, so it’s my fault that I didn’t word the question right. It’s practically a philosophical tradition to bury philosophy and then do philosophy on the grave of philosophy. For example the positivists sought to bury metaphysics. The king is dead, long live the king. So, sure, there are many examples of that.
The issue I was interested in was not this, but was whether it is probable that Eliezer independently reproduced Quine’s philosophy. I did not think it was likely. Certain of our ideas really do arise spontaneously among the clever generation after generation, but other ideas do not but are discovered rarely, at which point the ideas may be widely disseminated. I don’t number Quine’s ideas as among those that arise spontaneously, but among those that are rarely discovered and then may be widely disseminated. My evidence for this was Quine’s seeming originality. In response, it was argued that until Quine, the discoverers went on to do something else, which is why it wasn’t until Quine that the ideas were brought to the attention of philosophers. I argued in response that at least some fraction should, like Eliezer, have written about it, and then I asked, so where are these pre-Quine Quines who wrote about it? Only, I worded the question badly, and instead asked, where are the philosophers who dismissed philosophy. Of which there are, of course, many.
It’s hard to trace those causal lines, but here’s one data point: Dennett’s ideas have spread rather widely, and Dennett is an enthusiastic Quinean naturalist, and indeed was a student of Quine. Here’s Dennett:
...Quine’s book, ‘From a Logical Point of View’, which I read in despair in the math library late at night that freshman year, because I was taking a very difficult course in logic. And the next morning I’d read the whole book and I decided to transfer to Harvard to work with him.
Also: Stich, who might be called the ‘founder’ of experimental philosophy, was also an enthusiastic student of Quine’s. And experimental philosophy is the kind of philosophy getting all the major press in the last 10 years, it seems to me.
“still no evidence” is very much different to claims that certain properties do not exist in a given body of work. Absence of evidence (after an hour’s looking, if that) is not evidence of absence.
“Absence of evidence (after an hour’s looking, if that) is not evidence of absence.”
Actually it is. Weak evidence, but evidence nonetheless.
More to the point, if someone makes a claim that a work belonging to reference class X has a property Y, and then presents no evidence that it has that property, and you’ve previously investigated many other members of class X and found them all to have the property not-Y, it’s reasonable to assume that the new work has not-Yness until given evidence otherwise.
If someone comes along and says “this unqualified person on the internet has found a proof that the Second Law of Thermodynamics is wrong! I know all other unqualified people on the internet who’ve said that have been wrong, but I’m going to claim this one is correct, without giving you any evidence for that”, you’d be absolutely reasonable just to say “they’re wrong” without bothering to check.
It appears that Eliezer has come to the conclusion, based on the academic philosophy he’s read, that the reference class “academic philosophers” and the reference class “random nut on the internet” have several properties in common. He may or may not be correct in this conclusion (I’ve read little academic philosophy and wouldn’t want to judge) but his reactions given that premise seem perfectly sensible to me.
I think it’s absurd to equate the claim “Certain philosophers can have ideas useful to LessWrong” with “this unqualified person on the internet has found a proof that the Second Law of Thermodynamics is wrong”, and the fact that you’re framing it as such indicates that you are highly motivated in your argumentation.
As for the hypothetical premise that “the reference class “academic philosophers” and the reference class “random nut on the internet” have several properties in common”, I invite you to look to the top right of this website for the endorsement (of and by) the Future of Humanity institute that does, you guessed it, academic philosophy. Also refer to the numerous occurrences throughout this website of top contributors citing the FHI as a valid outlet for efficient donations towards existential risk mitigation. Is LessWrong suggesting we donate to people with the credibility of ‘random nuts on the internet’? Or is there perhaps some inconsistency which is what the people all over this thread are pointing out?
I actually have no great feelings about the argument either way. I’m using that as an example of a case where given a sufficiently strong prior you would accept Eliezer’s reasoning. I’m also suggesting that Eliezer appears to have that sufficiently strong prior.
Please note that I made no claims about my own thoughts on academic philosophy, and specifically stated that I don’t share that hypothetical premise. But from Eliezer’s own statements, it appears that he does have that pre-existing view of philosophers. And given that he has already formed that view he is being perfectly reasonable in not bothering to change that view without sufficiently strong evidence.
So what you’re actually saying is that given an arbitrary premise held arbitrarily strongly, one can rationally reject an arbitrary amount of evidence. I guess this is true, if trivially so.
What I think you’ve missed is that the premise is not shielded from discussion and can be itself judged, especially on this website which rejects theism for the exact reason of starting from an arbitrary premise.
Yes, I am saying that. However, I’m also saying that from what Eliezer has said, I don’t think his view of academic philosophy is an arbitrary one, but one formed from reading a reasonable amount of philosophy. Nor do I think the amount of evidence that’s been presented is arbitrary—it certainly doesn’t, by itself, convince me that this group of people have much to say, and I’m starting out from a neutral position, not a negative one.
Eliezer’s response does not. It looks like the response of one who feels their baby, LW style philosophy, is under attack. But it isn’t.
Methinks Eliezer needs to spend more time practicing the virtues of scholarship by actually reading much of the philosophy that he is critiquing. His assessments of “naturalistic” philosophy seem like straw men. Furthermore, from a psychological perspective, it seems like Eliezer is trying to defend his previously made commitments to “LW-Style philosophy” at all costs. This is not the mark of true rationality—true rationality admits challenges to previous assumptions.
Okay, so what have they done that I would consider cognitive philosophy? It doesn’t matter how many verbal-type non-dissolved questions we agree on apart from that. I’m taking free will as an exemplar and saying, “But it’s all like that, so far as I’ve been able to tell.”
It doesn’t matter how many verbal-type non-dissolved questions we agree on apart from that. I’m taking free will as an exemplar and saying, “But it’s all like that, so far as I’ve been able to tell.”
I’m not sure what you mean by this. Are you saying that my claim that LW-style philosophy shares many central assumptions with Quinean naturalism in contrast to most of philosophy doesn’t hinge on whether or not I can present a long list of things on which LW-style philosophy and Quinean naturalism agree on, in contrast to most of philosophy?
I suspect that’s not what you’re saying, but then… what do you think it was that I was claiming in the first place?
Or, another way to put it: Which sentence of my original article are you disagreeing with? Do you disagree with my claim that “standard Less Wrong positions on philosophical matters have been standard positions in a movement within mainstream philosophy for half a century”? Or perhaps you disagree with my claim that “Less Wrong-style philosophy is part of a movement within mainstream philosophy to massively reform philosophy in light of recent cognitive science—a movement that has been active for at least two decades”? Or perhaps you disagree with my claim that “Rationalists need not dismiss or avoid philosophy”?
I wonder if you agree with gjm’s suggestion that “LW-style philosophy takes (what turns out to be) Quinean naturalism as a starting point and then goes on to do things that no one working in mainstream philosophy has thought of.” That’s roughly what I said above, though of course I’ll point out that lots of Quinean naturalists have taken Quinean naturalism as a starting point and done things that nobody else thought of. That’s just what it means to make original contributions in the movement.
I’ll be happy to provide examples of “cognitive philosophy” once I’ve got this above bit cleared up. I’ve given examples before (Schroeder 2004; Bishop & Trout 2004; Bickle 2003), but of course I could give more detail.
Are you saying that my claim that LW-style philosophy shares many central assumptions with Quinean naturalism in contrast to most of philosophy doesn’t hinge on whether or not I can present a long list of things on which LW-style philosophy and Quinean naturalism agree on, in contrast to most of philosophy?
I’m saying that the claim that LW-style philosophy shares many assumptions with Quinean naturalism in contrast to most of philosophy is unimportant, thus, presenting the long list of basic assumptions on which LW-style and Quinean naturalism agree is from my perspective irrelevant.
Do you disagree with my claim that “standard Less Wrong positions on philosophical matters have been standard positions in a movement within mainstream philosophy for half a century”?
Yes. What I would consider “standard LW positions” is not “there is no libertarian free will” but rather “the philosophical debate on free will arises from the execution of the following cognitive algorithms X, Y, and Z”. If the latter has been a standard position then I would be quite interested.
Or perhaps you disagree with my claim that “Less Wrong-style philosophy is part of a movement within mainstream philosophy to massively reform philosophy in light of recent cognitive science—a movement that has been active for at least two decades”?
The kind of reforms you quote are extremely basic, along the lines of “OMG there are cognitive biases and they affect philosophers!” not “This is how this specific algorithm generates the following philosophical debate...” If the movement hasn’t progressed to the second stage, then there seems little point in aspiring LW rationalists reading about it.
GJM’s suggestion is correct but the thing which you seem to deny and which I think is true is that LW is at a different stage of doing this sort of philosophy than any Quinean naturalism I have heard of, so that the other Quineans “doing things that nobody else have thought of” don’t seem to be doing commensurate work.
I am not asking for an example of someone who agrees with me that, sure, cognitive philosophy sounds like a great idea, by golly. There’s a difference between saying “Sure, evolution is true!” and doing evolutionary biology.
I’m asking for someone who’s dissolved a philosophical question into a cognitive algorithm, preferably in a way not previously seen before on LW.
Did you read the LW sequence on free will, both the setup and the solution? Apologies if you’ve already previously answered this question, I have a vague feeling that I asked you before and you said yes, but still, just checking.
On the whole, you seem to think that I should be really enthusiastic about finding philosophers who agree with my basic assumptions, because here are these possible valuable allies in academia—why, if we could reframe LW as Quineanism, we’d have a whole support base ready-made!
Whereas I’m thinking, “If you ask what sort of activity these people perform in their daily work, their skills are similar to those of other philosophers and unlike those of people trying to figure out what algorithm a brain is running” and so they can’t be hired to do the sort of work we need without extensive retraining; and since we’re not out to reform academic philosophy, per se, it’s not clear that we need allies in a fight we could just bypass.
I’m saying that the claim that LW-style philosophy shares many assumptions with Quinean naturalism in contrast to most of philosophy is unimportant...
Well, it’s important to my claim that LW-style philosophy fits into the category of Quinean naturalism, which I think is undeniable. You may think Quinean naturalism is obvious, but well… that’s what makes you a Quinean naturalist. Part of the purpose of my post is to place LW-style philosophy in the context of mainstream philosophy, and my list of shared assumptions between LW-style philosophy and Quinean philosophy does just that. That goal by itself wasn’t meant to be very important. But I think it’s a categorization that cuts reality near enough the joints to be useful.
What I would consider “standard LW positions” is not “there is no libertarian free will” but rather “the philosophical debate on free will arises from the execution of the following cognitive algorithms X, Y, and Z”. If the latter has been a standard position then I would be quite interested.
Then we are using the word “standard” in different ways. If I were to ask most people to list some “standard LW positions”, I’m pretty sure they would list things like reductionism, empiricism, the rejection of libertarian free will, atheism, the centrality of cognitive science to epistemology, and so on—long before they list anything like “the philosophical debate on free will arises from the execution of the following cognitive algorithms X, Y, and Z”. I’m not even sure how much consensus that enjoys on Less Wrong. I doubt it is as much a ‘standard’ position on Less Wrong than the other things I mentioned.
But I’m not here to argue about the meaning of the word standard.
Disagreement: dissolved.
Moving on: Yes, I read the free will stuff. ‘How an Algorithm Feels from the Inside’ is one of my all-time favorite Yudkowsky posts.
I’ll have to be more clear on what you think LW is doing that Quinean naturalists are not doing. But really, I don’t even need to wait for that to respond. Even work by philosophers who are not Quinean naturalists can be useful in your very particular line of work—for example in clearing up your CEV article’s conflation of “extrapolating” from means to ends and “extrapolating” from current ends to new ends after reflective equilibrium and other processes have taken place.
Finally, you say that if Quinean naturalism hasn’t progressed from recognizing that biases affect philosophers to showing how a specific algorithm generates a philosophical debate then “there seems little point in aspiring LW rationalists reading about it.”
This claim is, I think, both clearly false as stated and misrepresents the state of Quinean naturalism.
First, on falsity: There are many other useful things for philosophers (including Quinean naturalists) to be doing besides just working with scientists to figure out why our brains produce confused philosophical debates. Since your own philosophical work on Less Wrong has considered far more than just this, I assume you agree. Thus, it is not the case that Quinean naturalists aren’t doing useful work unless they are discovering the cognitive algorithms that generate philosophical debates.
Second, on misrepresentation: Quinean naturalists don’t just discuss the fact that cognitive biases affect philosophers. Quinean naturalists also discuss how to do philosophy amidst the influence of cognitive biases. That very question is a major subject of your writing on Less Wrong, so I doubt you see no value in it. Moreover, Quinean naturalists do sometimes discuss how cognitive algorithms generate philosophical debates. See, for example, Eric Schwitzgebel’srecent work on how introspection works and why it generates philosophical confusions.
It seems you’re not just resisting the classification of LW-style philosophy within the broader category of Quinean naturalism. You’re also resisting the whole idea of seeing value in what mainstream naturalistic philosophers are doing, which I don’t get. How do you think that thought got generated? Reading too much modal logic and not enough Dennett / Bickle / Bishop / Metzinger / Lokhorst / Thagard?
I’m not even trying to say that Eliezer Yudkowsky should read more naturalistic philosophy. I suspect that’s not the best use of your time, especially given your strong aversion to it. But I am saying that the mainstream community has useful insights and clarifications and progress to contribute. You’ve already drawn heavily from the basic insights of Quinean naturalism, whether or not you got them from Quine himself. And you’ve drawn from some of the more advanced insights of people like Judea Pearl and Nick Bostrom.
So I guess I just don’t get what looks to me like a strong aversion in you to rationalists looking through Quinean naturalistic philosophy for useful insights. I don’t understand where that aversion is coming from. If you’re not that familiar with Quinean naturalistic philosophy, why do you assume in advance that it’s a bad idea to read through it for insights?
I’m quite sure they do. Right now I can’t think of a philosopher who is as imposing to me as (the late) E.T. Jaynes is. Unless you count people like Judea Pearl who also do AI research, that is. :)
But that doesn’t mean that mainstream philosophers never make useful and original contributions on all kinds of subjects relevant to Less Wrong and even to friendly AI.
Right now I can’t think of a philosopher who is as imposing to me as (the late) E.T. Jaynes is. Unless you count people like Judea Pearl who also do AI research, that is.
That (Jaynes) is a pretty high standard. But not impossibly high. As candidates, I would mention Jaakko Hintikka, Per Martin-Lof and the late David Lewis. If you are allowed to count economists, then I would also mention game theorists like Aumann, Binmore, and the late John Harsanyi. And if you allow philosophically inclined physicists like Jaynes, there are quite a few folks worth mentioning.
If so, I don’t think he can maintain that position consistently, since he has already benefited from the work of many mainstream philosophers, and continues to do so—for example Bostrom on anthropic reasoning.
So I guess I just don’t get what looks to me like a strong aversion in you to rationalists looking through Quinean naturalistic philosophy for useful insights. I don’t understand where that aversion is coming from.
Actually an expectation that studying this philosophy stuff would be of no use (or even can harm you), which is a more reflectively reliable judgment than mere emotional aversion. Might be incorrect, but can’t be influenced by arguing that aversion is irrelevant (not that you do argue this way, but summarizing the position with use of that word suggests doing that).
Yes, that’s what most Quinean naturalists are doing...
Can I expect a reply to my claim that a central statement of your above comment was both clearly false and misrepresented Quinean naturalism? I hope so. I’m also still curious to hear your response to the specific example I’ve now given several times of how even non-naturalistic philosophy can provide useful insights that bear directly on your work on Friendly AI (the “extrapolation” bit).
As for expecting naturalistic philosophy to teach very bad habits of thought: That has some plausibility. But it is hard to argue about with any precision. What’s the cost/benefit analysis on reading naturalistic philosophy after having undergone significant LW-rationality training? I don’t know.
But I will point out that reading naturalistic philosophy (1) deconverted me from fundamentalist Christianity, (2) led me to reject most of standard analytic philosophy, (3) led me to almost all of the “standard” (in the sense I intended above) LW positions, and (4) got me reading and loving Epistemology and the Psychology of Human Judgment and Good and Real (two philosophy books that could just as well be a series of Less Wrong blog posts) - all before I started regularly reading Less Wrong.
So… it’s not always bad. :)
Also, I suspect your recommendation to not read naturalistic, reductionistic philosophy outside of Less Wrong feels very paternalistic and cultish to me, and I have a negative emotional (and perhaps rational) reaction to the suggestion that people should only get their philosophy from a single community.
Can I expect a reply to my claim that a central statement of your above comment was both clearly false and misrepresented Quinean naturalism?
Reply to charge that it is clearly false: Sorry, it doesn’t look clearly false to me. It seems to me that people can get along just fine knowing only what philosophy they pick up from reading AI books.
Reply to charge that it misrepresented Quinean naturalism: Give me an example of one philosophical question they dissolved into a cognitive algorithm. Please don’t link to a book on Amazon where I click “Surprise me” ten times looking for a dissolution and then give up. Just tell me the question and sketch the algorithm.
The CEV article’s “conflation” is not a convincing example. I was talking about the distinction between terminal and instrumental value way back in 2001, though I made the then-usual error of using nonstandard terminology. I left that distinction out of CEV specifically because (a) I’d seen it generate cognitive errors in people who immediately went funny in the head as soon as they were introduced to the concept of top-level values, and (b) because the original CEV paper wasn’t supposed to go down to the level of detail of ordering expected-consequence updates versus moral-argument-processing updates.
On whether people can benefit from reading philosophy outside of Less Wrong and AI books, we simply disagree.
Your response on misrepresenting Quinean naturalism did not reply to this part: “Quinean naturalists don’t just discuss the fact that cognitive biases affect philosophers. Quinean naturalists also discuss how to do philosophy amidst the influence of cognitive biases. That very question is a major subject of your writing on Less Wrong, so I doubt you see no value in it.”
As for an example of dissolving certain questions into cognitive algorithms, I’m drafting up a post on that right now. (Actually, the current post was written as a dependency for the other post I’m writing.)
On CEV and extrapolation: You seem to agree that the distinction is useful, because you’ve used it yourself elsewhere (you just weren’t going into so much detail in the CEV paper). But that seems to undermine your point that valuable insights are not to be found in mainstream philosophy. Or, maybe that’s not your claim. Maybe your claim is that all the valuable insights of mainstream philosophy happen to have already shown up on Less Wrong and in AI textbooks. Either way, I once again simply disagree.
I doubt that you picked up all the useful philosophy you have put on Less Wrong exclusively from AI books.
I agree about philosophy and actually I feel similar about the LW style rationality, for my value of real work (engineering mostly, with some art and science). Your tricks burden the tree search, and also easily lead to wrong order of branch processing as the ‘biases’ for effective branch processing are either disabled or worst of all negated, before a substitute is devised.
If you want to form a belief about, for example, FAI, it’s all nice that you don’t feel that the morality can result from some simple principles. If you want to build FAI—this branch (the generated morality that we agree with) is much much lower while it’s probability of success, really, isn’t that much worse, as the long, hand wavy argument has many points of possible failure and low reliability. Then, there’s still no immunity against fallacies. The worst form of sunk cost fallacy is disregard for possibility of better solution after the cost has been sunk. That’s what destroys corporations after they sink costs. They don’t even pursue cost-recovery option when it doesn’t coincide with prior effort and only utilizes part of prior effort.
There are many other [Edit: was originally ‘more’] useful things for philosophers (including Quinean naturalists) to be doing besides just working with scientists to figure out why our brains produce confused philosophical debates.
Perhaps. But it is difficult to imagine any less complete problem dissolution being successful at actually shutting down that confused philosophical debate, and thus freeing those first-class minds to actually do those hypothetical useful things.
BTW, by “more” I meant “additional”: I meant that there “are many other useful things for philosophers… to be doing...” I’ve now clarified the wording in the original comment.
It might be useful, if only for gaining status and attention and funding, to connect your work directly to one or several academic fields. To present it as a synthesis of philosophy, computer science, and cognitive science (or some other combination of your choice.) When people ask me what LessWrong is, I generally say something like “It’s philosophy from a computer scientist’s perspective.” Most people can only put a mental label on something when they have a rough idea of what it’s like, and it’s not practical to say, “Well, our work isn’t like anything.”
That doesn’t mean you have to hire philosophers or join a philosophy department; it might not mean that you, personally, have to do anything. But I do think that more people would be interested, and have a smaller inferential distance, if LW ideas were generally presented as related to other disciplines.
Expanding on this, which section of my local Barnes And Noble is your (Eliezer) book going to be in? Philosophy seems like the best fit (aside from the best selling non-fiction) to get new interested readership.
Amazon’s “Books > Nonfiction > Social Sciences” contains things like Malcolm Gladwell and Predictably Irrational, which I think is the audience that Eliezer is targeting.
Just taking the example I happen to know about, Sarah-Jane Leslie works on the meaning of generics. (What do we mean when we say “Tigers have stripes” ? All tigers? Most tigers? Normal tigers? But then how do we account for true statements like “Tigers eat people” when most tigers don’t eat people, or “Peacocks have colorful tails” when female peacocks don’t have colorful tails?) She answers this question directly using evidence from cognitive science. I think it counts as question-dissolving.
When I read your first post here, my mind immediately went to You’re Entitled to Arguments, But Not (That Particular) Proof. I gave you you the benefit of the doubt since you called it a ‘litmus test’ (however arbitrary), but you seem to have anchored on that. If your work is in substantial agreement with an established field in philosophy, that means there are more intelligent people who could become allies, and a store of knowledge from where valuable insights could come. I don’t know why you are looking this particular gift horse in the mouth.
There’s lots of people I think have valuable insights—cognitive scientists, AI researchers, statistical learning experts, mathematicians...
The question is whether high-grade academic philosophy belongs on the scholarship list, not whether scholarship is a virtue. The fact that they have managed to produce a minority school that agrees with Gary Drescher on the extremely basic question of whether there’s libertarian free will (no) and people are made of atoms (yes), does not entitle them to a position next to “Artificial Intelligence: A Modern Approach”.
Physicalism and the rejection of free will are both majority positions in Anglophone philosophy, actually, but I agree that agreement on those points doesn’t put someone on the shelf next to AIMA.
Physicalism and the rejection of free will are both majority positions in Anglophone philosophy
Regarding physicalism, I don’t entirely trust that survey.
Firstly, most of those who call themselves physicalists nevertheless think that qualia exist and are Deeply Mysterious, such that one cannot deduce a priori, from objective physical facts, that Alfred isn’t a zombie or that Alfred and Bob aren’t qualia-inverted with respect to each other.
Secondly, in very recent years − 90s into the new century—I think there’s been a rising tide of antimaterialism. Erstwhile physicalists such as Jaegwon Kim have defected. Anthologies are published with names like “The Waning of Materialism”.
As the survey itself tell us, only 16% accept or lean towards “zombies are inconceivable”.
This is all consistent with my experience in internet debates, where it seems that most upcoming or wannabe philosophers who have any confident opinions on the matter are antimaterialists.
Firstly, most of those who call themselves physicalists nevertheless think that qualia exist and are Deeply Mysterious, such that one cannot deduce a priori, from objective physical facts, that Alfred isn’t a zombie or that Alfred and Bob aren’t qualia-inverted with respect to each other.
[...]
only 16% accept or lean towards “zombies are inconceivable”.
Strictly speaking, I don’t think either of these requires abandonment of physicalism by even a small degree. To say that one can or cannot conceive something is not to directly say anything about reality itself (#). To say that one can or cannot deduce something is, again, not directly to say anything about reality itself (#except in the trivial sense that it says something about what one, i.e. a real person, can or cannot deduce, or can or cannot conceive). Even if you want to argue that it says something about reality itself, however indirectly, it’s not at all obvious that it says this particular thing (i.e. non-physicalism).
In particular, I am well aware of the severe limitations of deduction as a path to knowledge. Being so aware, I am not in the slightest surprised by, or troubled by, the inability to deduce that Alfred isn’t a zombie. I don’t see why I should be troubled. As for what I can conceive—well, I can conceive all sorts of things which have no obvious connection to reality. Why should examination of the limits of my imagination give me any sort of information about whether physicalism is true?
The key question for me is: is the hypothesis of physicalism tenable? I’m not asking for proof, deductive or otherwise. I am asking whether the hypothesis is consistent with the evidence and internally coherent. The fact that someone can conceive of zombies, and therefore conceive that the hypothesis is false, is no disproof of the hypothesis. And similarly, the fact that the hypothesis of physicalism cannot be deduced is no disproof either.
Possibly you should state your hypothesis ahead of time and define what would count (or have counted in the past) as a worthwhile contribution to LW-style rationalism from within the analytic philosophy community.
Then we would have a concrete way to decide the question of whether analytic philosophy has contributed anything in the past, or contributes anything in the future.
It also might turn out in the process of formalising your definition of what counts as a worthwhile contribution that nothing outside of your specific field of AI research counts for you, which would in itself be a worthwhile realisation.
Acknowledging my own biases here, I’m an analytic philosopher who mostly teaches scientific methodology and ethics (with a minor side interest in statistics) and my reaction to perusing the LW content was that there were some very interesting and valuable nuggets here for me to fossick out but that the bulk of the content wasn’t new or non-obvious to me.
Possibly there is so little for you in philosophy that has real novelty or value because there is already enormous overlap between what you do and what is done in the relevant subset of philosophy.
Being a philosopher makes you acutely aware of how deep the intellectual debts of most modern people are to philosophy, and how little awareness of this they have. It’s all too easy to believe that one came to one’s moral viewpoint entirely unassisted and entirely naturally, for example, without being aware that one is actually articulating a mixture of Kant and Bentham’s ideas that you never would have come up with had you lived before Kant and Bentham. Many people who have never heard of Peter Singer take the animal liberation movement for granted, unaware that the term “animal liberation” was coined by a philosopher in 1975 drawing on previous work by philosophers in the 1970s.
Look, if someone came to me and said, “I’m great at LW-style philosophy, and the proof of this is, I can argue there’s no libertarian free will” I would reply “You have not yet done any difficult or worthwhile cognitive work.” It’s like saying you don’t believe in astrology. Well, great, and yes there’s lots of people who disagree with you about that, but there’s a difference between doing grade school arithmetic and doing calculus, and “There is no libertarian free will” is grade school arithmetic. It doesn’t interest me that this philosophical school agrees with me about that. It’s too simple and basic, and part of what I object to in philosophy is that they are still arguing about problems like this instead of moving onto real questions.
Eliezer,
I don’t get it. Your comment here doesn’t respond to anything I said in my previous comment. The first sentence of my previous comment is: “I did not put you in the Quinean camp merely because of your agreement about libertarian free will.”
I think Eliezer is suggesting that all the things you’ve mentioned that distinguish Quinean naturalists from other philosophers are similarly basic, and that “LW-style philosophy” takes (what turns out to be) Quinean naturalism as a starting point and then goes on to do things that no one working in mainstream philosophy has thought of.
In other words, that the problem with mainstream philosophy isn’t that it’s all wrong, but that much of it is wrong and that the part that isn’t wrong is mostly not doing anything interesting with its not-wrongness.
(I make no comment on whether all, or some, or none, of that is correct. I’m just hoping to reduce the amount of talking-past-one-another here.)
Eliezer is suggesting that the Quineans are “not doing anything interesting with [their] not-wrongness” after being aware of the field for all of an hour and a half?!
Makes perfect sense to me. Someone comes up to me and says “This person is a brilliant mathematician! She just showed me a proof that there’s no highest prime, and proved Pythagoras’ theorem!” my response would be that that’s still no evidence that she’s made any worthwhile contribution to mathematics. She may have, but there’s little reason to believe it from the original statement.
Seems to me less like that and more like, “this Euclid fellow was brilliant”, followed by a list of things that Euclid proved before anybody else proved. Timing matters here. It’s no coincidence that before Quine came along, the clever Eliezers were not taking Quinean naturalism for granted.
For another analogy, if someone came along and told you, “this Hugh Everett fellow was brilliant! Here, read this paper in which he argues that the wave function never collapses”, would you say, “well, Eliezer already went through that a few years ago; there’s still no evidence that Everett made any worthwhile contribution”?
“before Quine came along, the clever Eliezers were not taking Quinean naturalism for granted.”
Citation needed.
I did not come to this conclusion on the basis of having read the claim somewhere. Rather, it’s what I gather from having read philosophy from both before and after Quine. If clever men were coming up with Quinean insight left and right before Quine appeared, then we should see a large number of philosophers pre-Quine who make Quine redundant. I don’t recall encountering any of these philosophers whose existence would virtually be assured if I were wrong. But suppose that I am simply ignorant. We still have Quine’s reputation to content with, the wide acknowledgment by major philosophers that he was highly influential. If I were wrong, he should have been lost in a sea of bright young men who anticipated his key insights.
“If clever men were coming up with Quinean insight left and right before Quine appeared, then we should see a large number of philosophers pre-Quine who make Quine redundant.”
Assuming also that those ‘clever men’ were going into philosophy rather than dismissing it as Eliezer has.
Eliezer may say that he dismisses philosophy, but he has nevertheless published a great deal which takes issue with some philosophy, agrees with other philosophy, and most importantly, he has provided a great deal of argumentation in favor of these conclusions which some philosophers agree with and other philosophers disagree with. Whether he believes it or not, Eliezer is doing philosophy, and a lot of it.
So, where are these clever men pre-Qune who dismissed philosophy and then proceeded, as Eliezer has done, to produce reams of it?
There are a few, e.g. E.T. Jaynes, Alfred North Whitehead (“Philosophy begins in wonder. And, at the end, when philosophic thought has done its best, the wonder remains. ”), and Richard Feynman (over and over and over again.)
More generally, though, those ‘clever men’ have tended to ignore philosophy and charge ahead with whatever they’re doing; it’s just that Eliezer’s work has tended to impinge more on philosophy than, say, themodynamics experiments or calculus proofs.
ETA: This didn’t actually address what Constant meant; my apologies.
Well, you did answer the question I asked, so it’s my fault that I didn’t word the question right. It’s practically a philosophical tradition to bury philosophy and then do philosophy on the grave of philosophy. For example the positivists sought to bury metaphysics. The king is dead, long live the king. So, sure, there are many examples of that.
The issue I was interested in was not this, but was whether it is probable that Eliezer independently reproduced Quine’s philosophy. I did not think it was likely. Certain of our ideas really do arise spontaneously among the clever generation after generation, but other ideas do not but are discovered rarely, at which point the ideas may be widely disseminated. I don’t number Quine’s ideas as among those that arise spontaneously, but among those that are rarely discovered and then may be widely disseminated. My evidence for this was Quine’s seeming originality. In response, it was argued that until Quine, the discoverers went on to do something else, which is why it wasn’t until Quine that the ideas were brought to the attention of philosophers. I argued in response that at least some fraction should, like Eliezer, have written about it, and then I asked, so where are these pre-Quine Quines who wrote about it? Only, I worded the question badly, and instead asked, where are the philosophers who dismissed philosophy. Of which there are, of course, many.
It’s hard to trace those causal lines, but here’s one data point: Dennett’s ideas have spread rather widely, and Dennett is an enthusiastic Quinean naturalist, and indeed was a student of Quine. Here’s Dennett:
Also: Stich, who might be called the ‘founder’ of experimental philosophy, was also an enthusiastic student of Quine’s. And experimental philosophy is the kind of philosophy getting all the major press in the last 10 years, it seems to me.
“still no evidence” is very much different to claims that certain properties do not exist in a given body of work. Absence of evidence (after an hour’s looking, if that) is not evidence of absence.
“Absence of evidence (after an hour’s looking, if that) is not evidence of absence.” Actually it is. Weak evidence, but evidence nonetheless.
More to the point, if someone makes a claim that a work belonging to reference class X has a property Y, and then presents no evidence that it has that property, and you’ve previously investigated many other members of class X and found them all to have the property not-Y, it’s reasonable to assume that the new work has not-Yness until given evidence otherwise.
If someone comes along and says “this unqualified person on the internet has found a proof that the Second Law of Thermodynamics is wrong! I know all other unqualified people on the internet who’ve said that have been wrong, but I’m going to claim this one is correct, without giving you any evidence for that”, you’d be absolutely reasonable just to say “they’re wrong” without bothering to check.
It appears that Eliezer has come to the conclusion, based on the academic philosophy he’s read, that the reference class “academic philosophers” and the reference class “random nut on the internet” have several properties in common. He may or may not be correct in this conclusion (I’ve read little academic philosophy and wouldn’t want to judge) but his reactions given that premise seem perfectly sensible to me.
I think it’s absurd to equate the claim “Certain philosophers can have ideas useful to LessWrong” with “this unqualified person on the internet has found a proof that the Second Law of Thermodynamics is wrong”, and the fact that you’re framing it as such indicates that you are highly motivated in your argumentation.
As for the hypothetical premise that “the reference class “academic philosophers” and the reference class “random nut on the internet” have several properties in common”, I invite you to look to the top right of this website for the endorsement (of and by) the Future of Humanity institute that does, you guessed it, academic philosophy. Also refer to the numerous occurrences throughout this website of top contributors citing the FHI as a valid outlet for efficient donations towards existential risk mitigation. Is LessWrong suggesting we donate to people with the credibility of ‘random nuts on the internet’? Or is there perhaps some inconsistency which is what the people all over this thread are pointing out?
I actually have no great feelings about the argument either way. I’m using that as an example of a case where given a sufficiently strong prior you would accept Eliezer’s reasoning. I’m also suggesting that Eliezer appears to have that sufficiently strong prior.
Please note that I made no claims about my own thoughts on academic philosophy, and specifically stated that I don’t share that hypothetical premise. But from Eliezer’s own statements, it appears that he does have that pre-existing view of philosophers. And given that he has already formed that view he is being perfectly reasonable in not bothering to change that view without sufficiently strong evidence.
So what you’re actually saying is that given an arbitrary premise held arbitrarily strongly, one can rationally reject an arbitrary amount of evidence. I guess this is true, if trivially so.
What I think you’ve missed is that the premise is not shielded from discussion and can be itself judged, especially on this website which rejects theism for the exact reason of starting from an arbitrary premise.
(I haven’t downvoted you by the way)
Yes, I am saying that. However, I’m also saying that from what Eliezer has said, I don’t think his view of academic philosophy is an arbitrary one, but one formed from reading a reasonable amount of philosophy. Nor do I think the amount of evidence that’s been presented is arbitrary—it certainly doesn’t, by itself, convince me that this group of people have much to say, and I’m starting out from a neutral position, not a negative one.
I affirm this interpretation.
Eliezer’s response does not. It looks like the response of one who feels their baby, LW style philosophy, is under attack. But it isn’t.
Methinks Eliezer needs to spend more time practicing the virtues of scholarship by actually reading much of the philosophy that he is critiquing. His assessments of “naturalistic” philosophy seem like straw men. Furthermore, from a psychological perspective, it seems like Eliezer is trying to defend his previously made commitments to “LW-Style philosophy” at all costs. This is not the mark of true rationality—true rationality admits challenges to previous assumptions.
Okay, so what have they done that I would consider cognitive philosophy? It doesn’t matter how many verbal-type non-dissolved questions we agree on apart from that. I’m taking free will as an exemplar and saying, “But it’s all like that, so far as I’ve been able to tell.”
I’m not sure what you mean by this. Are you saying that my claim that LW-style philosophy shares many central assumptions with Quinean naturalism in contrast to most of philosophy doesn’t hinge on whether or not I can present a long list of things on which LW-style philosophy and Quinean naturalism agree on, in contrast to most of philosophy?
I suspect that’s not what you’re saying, but then… what do you think it was that I was claiming in the first place?
Or, another way to put it: Which sentence of my original article are you disagreeing with? Do you disagree with my claim that “standard Less Wrong positions on philosophical matters have been standard positions in a movement within mainstream philosophy for half a century”? Or perhaps you disagree with my claim that “Less Wrong-style philosophy is part of a movement within mainstream philosophy to massively reform philosophy in light of recent cognitive science—a movement that has been active for at least two decades”? Or perhaps you disagree with my claim that “Rationalists need not dismiss or avoid philosophy”?
I wonder if you agree with gjm’s suggestion that “LW-style philosophy takes (what turns out to be) Quinean naturalism as a starting point and then goes on to do things that no one working in mainstream philosophy has thought of.” That’s roughly what I said above, though of course I’ll point out that lots of Quinean naturalists have taken Quinean naturalism as a starting point and done things that nobody else thought of. That’s just what it means to make original contributions in the movement.
I’ll be happy to provide examples of “cognitive philosophy” once I’ve got this above bit cleared up. I’ve given examples before (Schroeder 2004; Bishop & Trout 2004; Bickle 2003), but of course I could give more detail.
I’m saying that the claim that LW-style philosophy shares many assumptions with Quinean naturalism in contrast to most of philosophy is unimportant, thus, presenting the long list of basic assumptions on which LW-style and Quinean naturalism agree is from my perspective irrelevant.
Yes. What I would consider “standard LW positions” is not “there is no libertarian free will” but rather “the philosophical debate on free will arises from the execution of the following cognitive algorithms X, Y, and Z”. If the latter has been a standard position then I would be quite interested.
The kind of reforms you quote are extremely basic, along the lines of “OMG there are cognitive biases and they affect philosophers!” not “This is how this specific algorithm generates the following philosophical debate...” If the movement hasn’t progressed to the second stage, then there seems little point in aspiring LW rationalists reading about it.
GJM’s suggestion is correct but the thing which you seem to deny and which I think is true is that LW is at a different stage of doing this sort of philosophy than any Quinean naturalism I have heard of, so that the other Quineans “doing things that nobody else have thought of” don’t seem to be doing commensurate work.
I am not asking for an example of someone who agrees with me that, sure, cognitive philosophy sounds like a great idea, by golly. There’s a difference between saying “Sure, evolution is true!” and doing evolutionary biology.
I’m asking for someone who’s dissolved a philosophical question into a cognitive algorithm, preferably in a way not previously seen before on LW.
Did you read the LW sequence on free will, both the setup and the solution? Apologies if you’ve already previously answered this question, I have a vague feeling that I asked you before and you said yes, but still, just checking.
On the whole, you seem to think that I should be really enthusiastic about finding philosophers who agree with my basic assumptions, because here are these possible valuable allies in academia—why, if we could reframe LW as Quineanism, we’d have a whole support base ready-made!
Whereas I’m thinking, “If you ask what sort of activity these people perform in their daily work, their skills are similar to those of other philosophers and unlike those of people trying to figure out what algorithm a brain is running” and so they can’t be hired to do the sort of work we need without extensive retraining; and since we’re not out to reform academic philosophy, per se, it’s not clear that we need allies in a fight we could just bypass.
Well, it’s important to my claim that LW-style philosophy fits into the category of Quinean naturalism, which I think is undeniable. You may think Quinean naturalism is obvious, but well… that’s what makes you a Quinean naturalist. Part of the purpose of my post is to place LW-style philosophy in the context of mainstream philosophy, and my list of shared assumptions between LW-style philosophy and Quinean philosophy does just that. That goal by itself wasn’t meant to be very important. But I think it’s a categorization that cuts reality near enough the joints to be useful.
Then we are using the word “standard” in different ways. If I were to ask most people to list some “standard LW positions”, I’m pretty sure they would list things like reductionism, empiricism, the rejection of libertarian free will, atheism, the centrality of cognitive science to epistemology, and so on—long before they list anything like “the philosophical debate on free will arises from the execution of the following cognitive algorithms X, Y, and Z”. I’m not even sure how much consensus that enjoys on Less Wrong. I doubt it is as much a ‘standard’ position on Less Wrong than the other things I mentioned.
But I’m not here to argue about the meaning of the word standard.
Disagreement: dissolved.
Moving on: Yes, I read the free will stuff. ‘How an Algorithm Feels from the Inside’ is one of my all-time favorite Yudkowsky posts.
I’ll have to be more clear on what you think LW is doing that Quinean naturalists are not doing. But really, I don’t even need to wait for that to respond. Even work by philosophers who are not Quinean naturalists can be useful in your very particular line of work—for example in clearing up your CEV article’s conflation of “extrapolating” from means to ends and “extrapolating” from current ends to new ends after reflective equilibrium and other processes have taken place.
Finally, you say that if Quinean naturalism hasn’t progressed from recognizing that biases affect philosophers to showing how a specific algorithm generates a philosophical debate then “there seems little point in aspiring LW rationalists reading about it.”
This claim is, I think, both clearly false as stated and misrepresents the state of Quinean naturalism.
First, on falsity: There are many other useful things for philosophers (including Quinean naturalists) to be doing besides just working with scientists to figure out why our brains produce confused philosophical debates. Since your own philosophical work on Less Wrong has considered far more than just this, I assume you agree. Thus, it is not the case that Quinean naturalists aren’t doing useful work unless they are discovering the cognitive algorithms that generate philosophical debates.
Second, on misrepresentation: Quinean naturalists don’t just discuss the fact that cognitive biases affect philosophers. Quinean naturalists also discuss how to do philosophy amidst the influence of cognitive biases. That very question is a major subject of your writing on Less Wrong, so I doubt you see no value in it. Moreover, Quinean naturalists do sometimes discuss how cognitive algorithms generate philosophical debates. See, for example, Eric Schwitzgebel’s recent work on how introspection works and why it generates philosophical confusions.
It seems you’re not just resisting the classification of LW-style philosophy within the broader category of Quinean naturalism. You’re also resisting the whole idea of seeing value in what mainstream naturalistic philosophers are doing, which I don’t get. How do you think that thought got generated? Reading too much modal logic and not enough Dennett / Bickle / Bishop / Metzinger / Lokhorst / Thagard?
I’m not even trying to say that Eliezer Yudkowsky should read more naturalistic philosophy. I suspect that’s not the best use of your time, especially given your strong aversion to it. But I am saying that the mainstream community has useful insights and clarifications and progress to contribute. You’ve already drawn heavily from the basic insights of Quinean naturalism, whether or not you got them from Quine himself. And you’ve drawn from some of the more advanced insights of people like Judea Pearl and Nick Bostrom.
So I guess I just don’t get what looks to me like a strong aversion in you to rationalists looking through Quinean naturalistic philosophy for useful insights. I don’t understand where that aversion is coming from. If you’re not that familiar with Quinean naturalistic philosophy, why do you assume in advance that it’s a bad idea to read through it for insights?
I’m reminded of the “subsequence” of The Level Above Mine, Competent Elites, Above Average AI Scientists, and That Magical Click.
Maybe mainstream philosophers just lack the aura of thousand-year-old rationalist vampires?
I’m quite sure they do. Right now I can’t think of a philosopher who is as imposing to me as (the late) E.T. Jaynes is. Unless you count people like Judea Pearl who also do AI research, that is. :)
But that doesn’t mean that mainstream philosophers never make useful and original contributions on all kinds of subjects relevant to Less Wrong and even to friendly AI.
That (Jaynes) is a pretty high standard. But not impossibly high. As candidates, I would mention Jaakko Hintikka, Per Martin-Lof and the late David Lewis. If you are allowed to count economists, then I would also mention game theorists like Aumann, Binmore, and the late John Harsanyi. And if you allow philosophically inclined physicists like Jaynes, there are quite a few folks worth mentioning.
I’d never heard of Per Martin-Lof, thanks.
I of course am not definitive here, but I strongly suspect that from EY’s perspective it means precisely that.
If so, I don’t think he can maintain that position consistently, since he has already benefited from the work of many mainstream philosophers, and continues to do so—for example Bostrom on anthropic reasoning.
Maybe. But they have a self-deprecating sense of humor. Doesn’t that count for something?
Actually an expectation that studying this philosophy stuff would be of no use (or even can harm you), which is a more reflectively reliable judgment than mere emotional aversion. Might be incorrect, but can’t be influenced by arguing that aversion is irrelevant (not that you do argue this way, but summarizing the position with use of that word suggests doing that).
Thanks for the link to Eric Schwitzgebel; very interesting reading!
Because I expect it to teach very bad habits of thought that will lead people to be unable to do real work. Assume naturalism! Move on! NEXT!
Yes, that’s what most Quinean naturalists are doing...
Can I expect a reply to my claim that a central statement of your above comment was both clearly false and misrepresented Quinean naturalism? I hope so. I’m also still curious to hear your response to the specific example I’ve now given several times of how even non-naturalistic philosophy can provide useful insights that bear directly on your work on Friendly AI (the “extrapolation” bit).
As for expecting naturalistic philosophy to teach very bad habits of thought: That has some plausibility. But it is hard to argue about with any precision. What’s the cost/benefit analysis on reading naturalistic philosophy after having undergone significant LW-rationality training? I don’t know.
But I will point out that reading naturalistic philosophy (1) deconverted me from fundamentalist Christianity, (2) led me to reject most of standard analytic philosophy, (3) led me to almost all of the “standard” (in the sense I intended above) LW positions, and (4) got me reading and loving Epistemology and the Psychology of Human Judgment and Good and Real (two philosophy books that could just as well be a series of Less Wrong blog posts) - all before I started regularly reading Less Wrong.
So… it’s not always bad. :)
Also, I suspect your recommendation to not read naturalistic, reductionistic philosophy outside of Less Wrong feels very paternalistic and cultish to me, and I have a negative emotional (and perhaps rational) reaction to the suggestion that people should only get their philosophy from a single community.
Reply to charge that it is clearly false: Sorry, it doesn’t look clearly false to me. It seems to me that people can get along just fine knowing only what philosophy they pick up from reading AI books.
Reply to charge that it misrepresented Quinean naturalism: Give me an example of one philosophical question they dissolved into a cognitive algorithm. Please don’t link to a book on Amazon where I click “Surprise me” ten times looking for a dissolution and then give up. Just tell me the question and sketch the algorithm.
The CEV article’s “conflation” is not a convincing example. I was talking about the distinction between terminal and instrumental value way back in 2001, though I made the then-usual error of using nonstandard terminology. I left that distinction out of CEV specifically because (a) I’d seen it generate cognitive errors in people who immediately went funny in the head as soon as they were introduced to the concept of top-level values, and (b) because the original CEV paper wasn’t supposed to go down to the level of detail of ordering expected-consequence updates versus moral-argument-processing updates.
Thanks for your reply.
On whether people can benefit from reading philosophy outside of Less Wrong and AI books, we simply disagree.
Your response on misrepresenting Quinean naturalism did not reply to this part: “Quinean naturalists don’t just discuss the fact that cognitive biases affect philosophers. Quinean naturalists also discuss how to do philosophy amidst the influence of cognitive biases. That very question is a major subject of your writing on Less Wrong, so I doubt you see no value in it.”
As for an example of dissolving certain questions into cognitive algorithms, I’m drafting up a post on that right now. (Actually, the current post was written as a dependency for the other post I’m writing.)
On CEV and extrapolation: You seem to agree that the distinction is useful, because you’ve used it yourself elsewhere (you just weren’t going into so much detail in the CEV paper). But that seems to undermine your point that valuable insights are not to be found in mainstream philosophy. Or, maybe that’s not your claim. Maybe your claim is that all the valuable insights of mainstream philosophy happen to have already shown up on Less Wrong and in AI textbooks. Either way, I once again simply disagree.
I doubt that you picked up all the useful philosophy you have put on Less Wrong exclusively from AI books.
I agree about philosophy and actually I feel similar about the LW style rationality, for my value of real work (engineering mostly, with some art and science). Your tricks burden the tree search, and also easily lead to wrong order of branch processing as the ‘biases’ for effective branch processing are either disabled or worst of all negated, before a substitute is devised.
If you want to form a belief about, for example, FAI, it’s all nice that you don’t feel that the morality can result from some simple principles. If you want to build FAI—this branch (the generated morality that we agree with) is much much lower while it’s probability of success, really, isn’t that much worse, as the long, hand wavy argument has many points of possible failure and low reliability. Then, there’s still no immunity against fallacies. The worst form of sunk cost fallacy is disregard for possibility of better solution after the cost has been sunk. That’s what destroys corporations after they sink costs. They don’t even pursue cost-recovery option when it doesn’t coincide with prior effort and only utilizes part of prior effort.
Perhaps. But it is difficult to imagine any less complete problem dissolution being successful at actually shutting down that confused philosophical debate, and thus freeing those first-class minds to actually do those hypothetical useful things.
BTW, by “more” I meant “additional”: I meant that there “are many other useful things for philosophers… to be doing...” I’ve now clarified the wording in the original comment.
It might be useful, if only for gaining status and attention and funding, to connect your work directly to one or several academic fields. To present it as a synthesis of philosophy, computer science, and cognitive science (or some other combination of your choice.) When people ask me what LessWrong is, I generally say something like “It’s philosophy from a computer scientist’s perspective.” Most people can only put a mental label on something when they have a rough idea of what it’s like, and it’s not practical to say, “Well, our work isn’t like anything.”
That doesn’t mean you have to hire philosophers or join a philosophy department; it might not mean that you, personally, have to do anything. But I do think that more people would be interested, and have a smaller inferential distance, if LW ideas were generally presented as related to other disciplines.
Expanding on this, which section of my local Barnes And Noble is your (Eliezer) book going to be in? Philosophy seems like the best fit (aside from the best selling non-fiction) to get new interested readership.
Amazon’s “Books > Nonfiction > Social Sciences” contains things like Malcolm Gladwell and Predictably Irrational, which I think is the audience that Eliezer is targeting.
Just taking the example I happen to know about, Sarah-Jane Leslie works on the meaning of generics. (What do we mean when we say “Tigers have stripes” ? All tigers? Most tigers? Normal tigers? But then how do we account for true statements like “Tigers eat people” when most tigers don’t eat people, or “Peacocks have colorful tails” when female peacocks don’t have colorful tails?) She answers this question directly using evidence from cognitive science. I think it counts as question-dissolving.
When I read your first post here, my mind immediately went to You’re Entitled to Arguments, But Not (That Particular) Proof. I gave you you the benefit of the doubt since you called it a ‘litmus test’ (however arbitrary), but you seem to have anchored on that. If your work is in substantial agreement with an established field in philosophy, that means there are more intelligent people who could become allies, and a store of knowledge from where valuable insights could come. I don’t know why you are looking this particular gift horse in the mouth.
There’s lots of people I think have valuable insights—cognitive scientists, AI researchers, statistical learning experts, mathematicians...
The question is whether high-grade academic philosophy belongs on the scholarship list, not whether scholarship is a virtue. The fact that they have managed to produce a minority school that agrees with Gary Drescher on the extremely basic question of whether there’s libertarian free will (no) and people are made of atoms (yes), does not entitle them to a position next to “Artificial Intelligence: A Modern Approach”.
Physicalism and the rejection of free will are both majority positions in Anglophone philosophy, actually, but I agree that agreement on those points doesn’t put someone on the shelf next to AIMA.
Regarding physicalism, I don’t entirely trust that survey.
Firstly, most of those who call themselves physicalists nevertheless think that qualia exist and are Deeply Mysterious, such that one cannot deduce a priori, from objective physical facts, that Alfred isn’t a zombie or that Alfred and Bob aren’t qualia-inverted with respect to each other.
Secondly, in very recent years − 90s into the new century—I think there’s been a rising tide of antimaterialism. Erstwhile physicalists such as Jaegwon Kim have defected. Anthologies are published with names like “The Waning of Materialism”.
As the survey itself tell us, only 16% accept or lean towards “zombies are inconceivable”.
This is all consistent with my experience in internet debates, where it seems that most upcoming or wannabe philosophers who have any confident opinions on the matter are antimaterialists.
All good points. I take back the claim that physicalism is a majority position; that is under serious doubt.
How sad! :(
[...]
Strictly speaking, I don’t think either of these requires abandonment of physicalism by even a small degree. To say that one can or cannot conceive something is not to directly say anything about reality itself (#). To say that one can or cannot deduce something is, again, not directly to say anything about reality itself (#except in the trivial sense that it says something about what one, i.e. a real person, can or cannot deduce, or can or cannot conceive). Even if you want to argue that it says something about reality itself, however indirectly, it’s not at all obvious that it says this particular thing (i.e. non-physicalism).
In particular, I am well aware of the severe limitations of deduction as a path to knowledge. Being so aware, I am not in the slightest surprised by, or troubled by, the inability to deduce that Alfred isn’t a zombie. I don’t see why I should be troubled. As for what I can conceive—well, I can conceive all sorts of things which have no obvious connection to reality. Why should examination of the limits of my imagination give me any sort of information about whether physicalism is true?
The key question for me is: is the hypothesis of physicalism tenable? I’m not asking for proof, deductive or otherwise. I am asking whether the hypothesis is consistent with the evidence and internally coherent. The fact that someone can conceive of zombies, and therefore conceive that the hypothesis is false, is no disproof of the hypothesis. And similarly, the fact that the hypothesis of physicalism cannot be deduced is no disproof either.
I
Possibly you should state your hypothesis ahead of time and define what would count (or have counted in the past) as a worthwhile contribution to LW-style rationalism from within the analytic philosophy community.
Then we would have a concrete way to decide the question of whether analytic philosophy has contributed anything in the past, or contributes anything in the future.
It also might turn out in the process of formalising your definition of what counts as a worthwhile contribution that nothing outside of your specific field of AI research counts for you, which would in itself be a worthwhile realisation.
Acknowledging my own biases here, I’m an analytic philosopher who mostly teaches scientific methodology and ethics (with a minor side interest in statistics) and my reaction to perusing the LW content was that there were some very interesting and valuable nuggets here for me to fossick out but that the bulk of the content wasn’t new or non-obvious to me.
Possibly there is so little for you in philosophy that has real novelty or value because there is already enormous overlap between what you do and what is done in the relevant subset of philosophy.
Being a philosopher makes you acutely aware of how deep the intellectual debts of most modern people are to philosophy, and how little awareness of this they have. It’s all too easy to believe that one came to one’s moral viewpoint entirely unassisted and entirely naturally, for example, without being aware that one is actually articulating a mixture of Kant and Bentham’s ideas that you never would have come up with had you lived before Kant and Bentham. Many people who have never heard of Peter Singer take the animal liberation movement for granted, unaware that the term “animal liberation” was coined by a philosopher in 1975 drawing on previous work by philosophers in the 1970s.