What you describe as egoist consequentialist’s reasoning is actually reasoning according to causal decision theory, and when you talk about influence of beliefs on consequences, this can be seen as considering a form of precommitment (which allows patching some of CDT’s blind spots). If you use TDT/UDT/ADT instead, the problem goes away, and egoistic consequentialists start cooperating.
I agree that Tesseract’s post needs more familiarity with the decision theory articles (mainly those taking apart Newcomb’s problem, esp. this). However, the complexity of value and metaethics sequences don’t help much. Per an EY post summarizing them I can’t find atm, the only relevant insight here from those articles is that, “Your ethics are part of your values, so your actions should take them into account as well.”
This leaves unanswered the questions of a) why we classify certain parts of our values as “ethics”, and b) whether those ethics are properly a terminal or instrumental value. These are what I tried to address in this article, with my answers being that a) Those are the parts where we intuitively rely on acausal “consequences” (SAMELs in the article), and b) instrumental.
Your article is an excellent one, and makes many of the same points I tried to make here.
Specifically,
...in Dilemma B, an ideal agent will recognize that their decision to pick their favorite ice cream at the expense of another person suggests that others in the same position will do (and have done) likewise, for the same reason.
is the same idea I was trying to express with the ‘cheating student’ example, and then generalized in the final part of the post, and likewise the idea of Parfitian-filtered decision theory seems to be essentially the same as the concept in my post of ideally-rational agents adopting decision theories which make them consciously ignore their goals in order to achieve them better. (And in fact, I was planning to include in my next post how this sort of morality solves problems like Parfit’s Hitchhiker when functionally applied.)
Upon looking back on the replies here (although I have yet to read through all the decision theory posts Vladimir recommended), I realize that I haven’t been convinced that I was wrong—that there’s a flaw in my theory I haven’t seen—only that the community strongly disapproves. Given that your post and mine share many of the same ideas, and yours is at +21 while mine is at −7, I think that the differences are that a. mine was seen as presumptuous (in the vein of the ‘one great idea’), and b. I didn’t communicate clearly enough (partially because I haven’t studied enough terminology) and include answers to enough anticipated objections to overcome the resistance engendered by a. I think I also failed to clearly make the distinction between this as a normative strategy (that is, one I think ideal game-theoretic agents would follow, and a good reason for consciously deciding to be moral) and as a positive description (the reason actual human beings are moral.)
However, I recognize that even though I haven’t yet been convinced of it, there may well be a problem here that I haven’t seen but would if I knew more about decision theory. If you could explain such a problem to me, I would be genuinely grateful—I want to be correct more than I want my current theory to be right.
Okay, on re-reading your post, I can be more specific. I think you make good points (obviously, because of the similarity with my article), and it would probably be well-received if submitted here in early ’09. However, there are cases where you re-treaded ground that has been discussed before without reference to the existing discussions and concepts:
The apparent contradiction in this case results from thinking about beliefs and actions as though they were separate. Arriving at a belief is an action in itself, one which can have effects on utility. One cannot, therefore, arrive at a belief about utility without considering the effects on utility that holding that belief would have. If arriving at the belief “actions are justified by their effect on utility” doesn’t maximize utility, then you shouldn’t arrive at that belief.
Here you’re describing what Wei Dai calls “computational/logical consequences” of a decision in his UDT article.
This rule requires you to hold whatever beliefs will (conditional upon them being held) lead to the best results – even when the actions those beliefs produce don’t, in themselves, maximize utility.
Here you’re describing EY’s TDT algorithm.
The applied morality becomes deontological, in the sense that actions are judged not by their effect on utility but by their adherence to the pre-set principle.
The label of deontological doesn’t quite fit here, as you don’t advocate adhering to a set of categorical “don’t do this” rules (as would be justified in a “running on corrupted hardware” case), but rather, consider a certain type of impact your decision has on the world, which itself determines what rules to follow.
Finally, I think you should have clarified that the relationship between your decision to (not) cheat and others’ decision is not a causal one (though still sufficient to motivate your decision).
I don’t think you deserved −7 (though I didn’t vote you up myself). In particular, I stand by my initial comment that, contra Vladimir, you show sufficient assimilation of the value complexity and meta-ethics sequences. I think a lot of the backlash is just from the presentation—not the format, or writing, but needing to adapt it to the terminology and insights already presented here. And I agree that you’re justified in not being convinced you’re wrong.
Hope that helps.
EDIT: You also might like this recent discussion about real-world Newcomblike problems, which I intend to come back to more rigorously
Very much, thank you. Your feedback has been a great help.
Given that others arrived at some of these conclusions before me, I can see why there would be disapproval—though I can hardly feel disappointed to have independently discovered the same answers. I think I’ll research the various models more thoroughly, refine my wording (I agree with you that using the term ‘deontology’ was a mistake), and eventually make a more complete and more sophisticated second attempt at morality as a decision theory problem.
Thanks for the feedback. Unfortunately, the discussion on my article was dominated by a huge tangent on utility functions (which I talked about, but was done in a way irrelevant to the points I was making). I think the difference was that I plugged my points into the scenarios and literature discussed here. What bothered me about your article was that it did not carefully define the relationship between your decision theory and the ethic you are arguing for, though I will read it again to give a more precise answer.
The idea of complexity of values explains why “happiness” or “selfishness” can’t be expected to capture the whole thing: when you talk about “good”, you mean “good” and not some other concept. To unpack “good”, you have no other option than to list all the things you value, and such list uttered by a human can’t reflect the whole thing accurately anyway.
Metaethics sequence deals with errors of confusing moral reasons and historical explanations: evolution’s goals are not your own and don’t have normative power over your own goals, even if there is a surface similarity and hence some explanatory power.
I agree that those are important things to learn, just not for the topic Tesseract is writing about.
What do you mean? Tesseract makes these exact errors in the post, and those posts explain how not to err there, which makes the posts directly relevant.
Tesseract’s conclusion is hindered by not having read about the interplay between decision theory and values (i.e. how to define a “selfish action”, which consequences to take into consider, etc.), not the complexity of value as such. Tesseract would me making the same errors on decision theory even if human values were not so complex, and decision theory is the focus of the post.
Might not be relevant to “Tesseract’s conclusion”, but is relevant to other little conclusions made in the post along the way, even if they are all independent and don’t damage each other.
However, the complexity of value and metaethics sequences don’t help much.
They may not have much in the way of factual conclusions to operate by, but they are an excellent introduction to how to think about ethics, morality, and what humans want—which is effectively the first and last thirds of this post.
It’s not well-constructed overall, but I wish I had a nickel every time someone’s huge ethical system turned out to be an unconscious example of rebelling within nature, or something that gets stuck on the pebblesorter example.
Right, but reversed stupidity is not intelligence. I mean, he can only get away with the following because he’s left his terms so fuzzy as to be meaningless:
And if, in the meanwhile, it seems to you like I’ve just proved that there is no morality… well, I haven’t proved any such thing. But, meanwhile, just ask yourself if you might want to help people even if there were no morality. If you find that the answer is yes, then you will later discover that you discovered morality.
That is, one would be upset if I said “there is a God, it’s Maxwell’s Equations!” because the concept of God and the concept of universal physical laws are generally distinct. Likewise, saying “well, morality is an inborn or taught bland desire to help others” makes a mockery of the word ‘morality.’
I think your interpretation oversimplifies things. He’s not saying “morality is an inborn or taught bland desire to help others”; he’s rather making the claim (which he defers until later) that what we mean by morality cannot be divorced from contingent human psychology, choices and preferences, and that it’s nonsense to claim “if moral sentiments and principles are contingent on the human brain rather than written into the nature of the universe, then human brains should therefore start acting like their caricatures of ‘immoral’ agents”.
I am not sure what you mean. Do you mean that the way Eliezer espouses thinking about ethics and morality in those sequences is a poor way of thinking about morality? Do you mean that Eliezer’s explanations of that way are poor explanations? Both? Something else?
The methodology is mediocre, and the conclusions are questionable. At the moment I can’t do much besides express distaste; my attempts to articulate alternatives have not gone well so far. But I am thinking about it, and actually just stumbled across something that might be useful.
I’m going to have to disagree with this. The methodology with which Eliezer approaches ethical and moral issues is definitely on par with or exceeding the philosophy of ethics that I’ve studied. I am still unsure whether you mean the methodology he espouses using, or the methods he applied to make the posts.
Your objection and its evident support by the community is noted, and therefore I have deleted the post. I will read further on the decision theory and its implications, as that seems to be a likely cause of error.
However, I have read the meta-ethics sequence, and some of Eliezer’s other posts on morality, and found them unsatisfactory—they seemed to me to presume that morality is something you should have regardless of the reason for it rather than seriously questioning the reasons for possessing it.
On the point of complexity of value, I was attempting to use the term ‘utility’ to describe human preferences, which would necessarily take into account complex values. If you could describe why this doesn’t work well, I would appreciate the correction.
That said, I’m not going to contend here without doing more research first (and thank you for the links), so this will be my last post on the subject.
they seemed to me to presume that morality is something you should have regardless of the reason for it rather than seriously questioning the reasons for possessing it.
One thing to consider: Why do you need a reason to be moral/altruistic but not a reason to be selfish? (Or, if you do need a reason to be selfish, where does the recursion end, when you need to justify every motive in terms of another?)
On the topic of these decision theories, you might get a lot from the second half of Gary Drescher’s book Good and Real. His take isn’t quite the same thing as TDT or UDT, but it’s on the same spectrum, and the presentation is excellent.
Most of your questions are already answered on the site, better then you attempt answering them. Read up on complexity of value, metaethics sequence, decision theory posts (my list) and discussion of Prisoner’s Dilemma in particular.
What you describe as egoist consequentialist’s reasoning is actually reasoning according to causal decision theory, and when you talk about influence of beliefs on consequences, this can be seen as considering a form of precommitment (which allows patching some of CDT’s blind spots). If you use TDT/UDT/ADT instead, the problem goes away, and egoistic consequentialists start cooperating.
I agree that Tesseract’s post needs more familiarity with the decision theory articles (mainly those taking apart Newcomb’s problem, esp. this). However, the complexity of value and metaethics sequences don’t help much. Per an EY post summarizing them I can’t find atm, the only relevant insight here from those articles is that, “Your ethics are part of your values, so your actions should take them into account as well.”
This leaves unanswered the questions of a) why we classify certain parts of our values as “ethics”, and b) whether those ethics are properly a terminal or instrumental value. These are what I tried to address in this article, with my answers being that a) Those are the parts where we intuitively rely on acausal “consequences” (SAMELs in the article), and b) instrumental.
Your article is an excellent one, and makes many of the same points I tried to make here.
Specifically,
is the same idea I was trying to express with the ‘cheating student’ example, and then generalized in the final part of the post, and likewise the idea of Parfitian-filtered decision theory seems to be essentially the same as the concept in my post of ideally-rational agents adopting decision theories which make them consciously ignore their goals in order to achieve them better. (And in fact, I was planning to include in my next post how this sort of morality solves problems like Parfit’s Hitchhiker when functionally applied.)
Upon looking back on the replies here (although I have yet to read through all the decision theory posts Vladimir recommended), I realize that I haven’t been convinced that I was wrong—that there’s a flaw in my theory I haven’t seen—only that the community strongly disapproves. Given that your post and mine share many of the same ideas, and yours is at +21 while mine is at −7, I think that the differences are that a. mine was seen as presumptuous (in the vein of the ‘one great idea’), and b. I didn’t communicate clearly enough (partially because I haven’t studied enough terminology) and include answers to enough anticipated objections to overcome the resistance engendered by a. I think I also failed to clearly make the distinction between this as a normative strategy (that is, one I think ideal game-theoretic agents would follow, and a good reason for consciously deciding to be moral) and as a positive description (the reason actual human beings are moral.)
However, I recognize that even though I haven’t yet been convinced of it, there may well be a problem here that I haven’t seen but would if I knew more about decision theory. If you could explain such a problem to me, I would be genuinely grateful—I want to be correct more than I want my current theory to be right.
Okay, on re-reading your post, I can be more specific. I think you make good points (obviously, because of the similarity with my article), and it would probably be well-received if submitted here in early ’09. However, there are cases where you re-treaded ground that has been discussed before without reference to the existing discussions and concepts:
Here you’re describing what Wei Dai calls “computational/logical consequences” of a decision in his UDT article.
Here you’re describing EY’s TDT algorithm.
The label of deontological doesn’t quite fit here, as you don’t advocate adhering to a set of categorical “don’t do this” rules (as would be justified in a “running on corrupted hardware” case), but rather, consider a certain type of impact your decision has on the world, which itself determines what rules to follow.
Finally, I think you should have clarified that the relationship between your decision to (not) cheat and others’ decision is not a causal one (though still sufficient to motivate your decision).
I don’t think you deserved −7 (though I didn’t vote you up myself). In particular, I stand by my initial comment that, contra Vladimir, you show sufficient assimilation of the value complexity and meta-ethics sequences. I think a lot of the backlash is just from the presentation—not the format, or writing, but needing to adapt it to the terminology and insights already presented here. And I agree that you’re justified in not being convinced you’re wrong.
Hope that helps.
EDIT: You also might like this recent discussion about real-world Newcomblike problems, which I intend to come back to more rigorously
Very much, thank you. Your feedback has been a great help.
Given that others arrived at some of these conclusions before me, I can see why there would be disapproval—though I can hardly feel disappointed to have independently discovered the same answers. I think I’ll research the various models more thoroughly, refine my wording (I agree with you that using the term ‘deontology’ was a mistake), and eventually make a more complete and more sophisticated second attempt at morality as a decision theory problem.
Great, glad to hear it! Looking forward to your next submission on this issue.
Thanks for the feedback. Unfortunately, the discussion on my article was dominated by a huge tangent on utility functions (which I talked about, but was done in a way irrelevant to the points I was making). I think the difference was that I plugged my points into the scenarios and literature discussed here. What bothered me about your article was that it did not carefully define the relationship between your decision theory and the ethic you are arguing for, though I will read it again to give a more precise answer.
The idea of complexity of values explains why “happiness” or “selfishness” can’t be expected to capture the whole thing: when you talk about “good”, you mean “good” and not some other concept. To unpack “good”, you have no other option than to list all the things you value, and such list uttered by a human can’t reflect the whole thing accurately anyway.
Metaethics sequence deals with errors of confusing moral reasons and historical explanations: evolution’s goals are not your own and don’t have normative power over your own goals, even if there is a surface similarity and hence some explanatory power.
I agree that those are important things to learn, just not for the topic Tesseract is writing about.
What do you mean? Tesseract makes these exact errors in the post, and those posts explain how not to err there, which makes the posts directly relevant.
Tesseract’s conclusion is hindered by not having read about the interplay between decision theory and values (i.e. how to define a “selfish action”, which consequences to take into consider, etc.), not the complexity of value as such. Tesseract would me making the same errors on decision theory even if human values were not so complex, and decision theory is the focus of the post.
Might not be relevant to “Tesseract’s conclusion”, but is relevant to other little conclusions made in the post along the way, even if they are all independent and don’t damage each other.
They may not have much in the way of factual conclusions to operate by, but they are an excellent introduction to how to think about ethics, morality, and what humans want—which is effectively the first and last thirds of this post.
Huh? It struck me as pretty poor, actually.
It’s not well-constructed overall, but I wish I had a nickel every time someone’s huge ethical system turned out to be an unconscious example of rebelling within nature, or something that gets stuck on the pebblesorter example.
Right, but reversed stupidity is not intelligence. I mean, he can only get away with the following because he’s left his terms so fuzzy as to be meaningless:
That is, one would be upset if I said “there is a God, it’s Maxwell’s Equations!” because the concept of God and the concept of universal physical laws are generally distinct. Likewise, saying “well, morality is an inborn or taught bland desire to help others” makes a mockery of the word ‘morality.’
I think your interpretation oversimplifies things. He’s not saying “morality is an inborn or taught bland desire to help others”; he’s rather making the claim (which he defers until later) that what we mean by morality cannot be divorced from contingent human psychology, choices and preferences, and that it’s nonsense to claim “if moral sentiments and principles are contingent on the human brain rather than written into the nature of the universe, then human brains should therefore start acting like their caricatures of ‘immoral’ agents”.
I am not sure what you mean. Do you mean that the way Eliezer espouses thinking about ethics and morality in those sequences is a poor way of thinking about morality? Do you mean that Eliezer’s explanations of that way are poor explanations? Both? Something else?
The methodology is mediocre, and the conclusions are questionable. At the moment I can’t do much besides express distaste; my attempts to articulate alternatives have not gone well so far. But I am thinking about it, and actually just stumbled across something that might be useful.
I’m going to have to disagree with this. The methodology with which Eliezer approaches ethical and moral issues is definitely on par with or exceeding the philosophy of ethics that I’ve studied. I am still unsure whether you mean the methodology he espouses using, or the methods he applied to make the posts.
Your objection and its evident support by the community is noted, and therefore I have deleted the post. I will read further on the decision theory and its implications, as that seems to be a likely cause of error.
However, I have read the meta-ethics sequence, and some of Eliezer’s other posts on morality, and found them unsatisfactory—they seemed to me to presume that morality is something you should have regardless of the reason for it rather than seriously questioning the reasons for possessing it.
On the point of complexity of value, I was attempting to use the term ‘utility’ to describe human preferences, which would necessarily take into account complex values. If you could describe why this doesn’t work well, I would appreciate the correction.
That said, I’m not going to contend here without doing more research first (and thank you for the links), so this will be my last post on the subject.
One thing to consider: Why do you need a reason to be moral/altruistic but not a reason to be selfish? (Or, if you do need a reason to be selfish, where does the recursion end, when you need to justify every motive in terms of another?)
On the topic of these decision theories, you might get a lot from the second half of Gary Drescher’s book Good and Real. His take isn’t quite the same thing as TDT or UDT, but it’s on the same spectrum, and the presentation is excellent.