Here’s maybe another point of view on this: consequentialism fundamentally talks about receiving stuff from the universe. An hour climbing a mountain, an hour swimming in the sea, or hey, an hour in the joy experience machine. The endpoint of consequentialism is a sort of amoeba that doesn’t really need a body to overcome obstacles, or a brain to solve problems, all it needs to do is receive and enjoy. To the extent I want life to be also about doing something or being someone, that might be a more natural fit for alternatives to consequentialism—deontology and virtue ethics.
Having a body that does things is part of your values and is easily described in them. I don’t see deontology or virtue ethics as giving any more fundamentally adequate solution to this (beyond the trivial ‘define a deontological rule about …‘, or ‘it is virtuous to do interesting things yourself’, but why not just do that with consequentialism?).
My attempt at interpreting what you mean is that you’re drawing a distinction between morality about world-states vs. morality about process, internal details, experiencing it, ‘yourself’. To give them names, “global”-values (you just want them Done) & “indexical”/’local”-values (preferences about your experiences, what you do, etc.) Global would be reducing suffering, avoiding heat death and whatnot. Local would be that you want to learn physics from the ground up and try to figure out XYZ interesting problem as a challenge by yourself, that you would like to write a book rather than having an AI do it for you, and so on.
I would say that, yes, for Global you should/would have an amorphous blob that doesn’t necessarily care about the process. That’s your (possibly non-sentient) AGI designing a utopia while you run around doing interesting Local things. Yet I don’t see why you think only Global is naturally described in consequentialism.
I intrinsically value having solved hard problems—or rather, I value feeling like I’ve solved hard problems, which is part of overall self-respect, and I also value realness to varying degrees. That I’ve actually done the thing, rather than taken a cocktail of exotic chemicals. We could frame this in a deontological & virtue ethics sense: I have a rule about realness, I want my experiences to be real. / I find it virtuous to solve hard problems, even if in a post-singularity world.
But do I really have a rule about realness? Uh, sort-of? I’d be fine to play a simulation where I forget about the AGI world and am in some fake-scifi game world and solve hard problems. In reality, my value has a lot more edge-cases that will be explored than many deontological rules prefer. My real value isn’t really a rule, it is just sometimes easy to describe it that way. Similar to how “do not lie” or “do not kill” is usually not a true rule.
Like, we could describe my actual value here as a rule, but seems actually more alien to the human mind. My actual value for realness is some complicated function of many aspects of my life, preferences, current mood to some degree, second-order preferences, and so on. Describing that as a rule is extremely reductive.
And ‘realness’ is not adequately described as a complete virtue either. I don’t always prefer realness: if playing a first-person shooter game, I prefer that my enemies are not experiencing realistic levels of pain! So there are intricate trade-offs here as I continue to examine my own values.
Another aspect I’m objecting to mentally when I try to apply those stances is that there’s two ways of interpreting deontology & virtue ethics that I think are common on LW. You can treat them as actual philosophical alternatives to consequentialism, like following the rule “do not lie”. Or you can treat them as essentially fancy words for deontology=>”strong prior for this rule being generally correct and also a good coordination point” and virtue ethics=>”acting according to a good Virtue consistently as a coordination scheme/culture modification scheme and/or because you also think that Virtue is itself a Good”.
Like, there’s a difference between talking about something using the language commonly associated with deontology and actually practicing deontology. I think conflating the two is unfortunate.
The overaching argument here is that consequentialism properly captures a human’s values, and that you can use the basic language of “I keep my word” (deontology flavored) or “I enjoy solving hard problems because they are good to solve” (virtue ethics flavored) without actually operating within those moral theories. You would have the ability to unfold these into the consequentialist statements of whatever form you prefer.
In your reply to cubefox, “respect this person’s wishes” is not a deontological rule. Well, it could be, but I expect your actual values don’t fulfill that. Just because your native internal language suggestively calls it that, doesn’t mean you should shoehorn it into the category of rule!
“play with this toy” still strikes me as natively a heuristic/approximation to the goal of “do things I enjoy”. The interlinking parts of my brain that decided to bring that forward is good at its job, but also dumb because it doesn’t do any higher order thinking. I follow that heuristic only because I expect to enjoy it—the heuristic providing that information. If I had another part of my consideration that pushed me towards considering whether that is a good plan, I might realize that I haven’t actually enjoyed playing with a teddy bear in years despite still feeling nostalgia for that. I’m not sure I see the gap between consequentialism and this. I don’t have the brain capacity to consider every impulse I get, but I do want to consider agents other than AIXI to be a consequentialist.
I think there’s a space in there for a theory of minds, but I expect it would be more mechanistic or descriptive rather than a moral theory. Ala shard theory.
Or, alternatively, even if you don’t buy my view that the majority of my heuristics can be cast as approximations of consequentialist propositions, then deontology/virtue ethics are not natural theories either by your descriptions. They miss a lot of complexity even within their usual remit.
No problem about long reply, I think your arguments are good and give me a lot to think about.
My attempt at interpreting what you mean is that you’re drawing a distinction between morality about world-states vs. morality about process, internal details, experiencing it, ‘yourself’.
I just thought of another possible classification: “zeroth-order consequentialist” (care about doing the action but not because of consequences), “first-order consequentialist” (care about consequences), “second-order consequentialist” (care about someone else being able to choose what to do). I guess you’re right that all of these can be translated into first-order. But by the same token, everything can be translated to zeroth-order. And the translation from second to first feels about as iffy as the translation from first to zeroth. So this still feels fuzzy to me, I’m not sure what is right.
Here’s maybe another point of view on this: consequentialism fundamentally talks about receiving stuff from the universe. An hour climbing a mountain, an hour swimming in the sea, or hey, an hour in the joy experience machine. The endpoint of consequentialism is a sort of amoeba that doesn’t really need a body to overcome obstacles, or a brain to solve problems, all it needs to do is receive and enjoy. To the extent I want life to be also about doing something or being someone, that might be a more natural fit for alternatives to consequentialism—deontology and virtue ethics.
This reply is perhaps a bit too long, oops.
Having a body that does things is part of your values and is easily described in them. I don’t see deontology or virtue ethics as giving any more fundamentally adequate solution to this (beyond the trivial ‘define a deontological rule about …‘, or ‘it is virtuous to do interesting things yourself’, but why not just do that with consequentialism?).
My attempt at interpreting what you mean is that you’re drawing a distinction between morality about world-states vs. morality about process, internal details, experiencing it, ‘yourself’. To give them names, “global”-values (you just want them Done) & “indexical”/’local”-values (preferences about your experiences, what you do, etc.) Global would be reducing suffering, avoiding heat death and whatnot. Local would be that you want to learn physics from the ground up and try to figure out XYZ interesting problem as a challenge by yourself, that you would like to write a book rather than having an AI do it for you, and so on.
I would say that, yes, for Global you should/would have an amorphous blob that doesn’t necessarily care about the process. That’s your (possibly non-sentient) AGI designing a utopia while you run around doing interesting Local things. Yet I don’t see why you think only Global is naturally described in consequentialism.
I intrinsically value having solved hard problems—or rather, I value feeling like I’ve solved hard problems, which is part of overall self-respect, and I also value realness to varying degrees. That I’ve actually done the thing, rather than taken a cocktail of exotic chemicals. We could frame this in a deontological & virtue ethics sense: I have a rule about realness, I want my experiences to be real. / I find it virtuous to solve hard problems, even if in a post-singularity world.
But do I really have a rule about realness? Uh, sort-of? I’d be fine to play a simulation where I forget about the AGI world and am in some fake-scifi game world and solve hard problems. In reality, my value has a lot more edge-cases that will be explored than many deontological rules prefer. My real value isn’t really a rule, it is just sometimes easy to describe it that way. Similar to how “do not lie” or “do not kill” is usually not a true rule.
Like, we could describe my actual value here as a rule, but seems actually more alien to the human mind. My actual value for realness is some complicated function of many aspects of my life, preferences, current mood to some degree, second-order preferences, and so on. Describing that as a rule is extremely reductive.
And ‘realness’ is not adequately described as a complete virtue either. I don’t always prefer realness: if playing a first-person shooter game, I prefer that my enemies are not experiencing realistic levels of pain! So there are intricate trade-offs here as I continue to examine my own values.
Another aspect I’m objecting to mentally when I try to apply those stances is that there’s two ways of interpreting deontology & virtue ethics that I think are common on LW. You can treat them as actual philosophical alternatives to consequentialism, like following the rule “do not lie”. Or you can treat them as essentially fancy words for deontology=>”strong prior for this rule being generally correct and also a good coordination point” and virtue ethics=>”acting according to a good Virtue consistently as a coordination scheme/culture modification scheme and/or because you also think that Virtue is itself a Good”.
Like, there’s a difference between talking about something using the language commonly associated with deontology and actually practicing deontology. I think conflating the two is unfortunate.
The overaching argument here is that consequentialism properly captures a human’s values, and that you can use the basic language of “I keep my word” (deontology flavored) or “I enjoy solving hard problems because they are good to solve” (virtue ethics flavored) without actually operating within those moral theories. You would have the ability to unfold these into the consequentialist statements of whatever form you prefer.
In your reply to cubefox, “respect this person’s wishes” is not a deontological rule. Well, it could be, but I expect your actual values don’t fulfill that. Just because your native internal language suggestively calls it that, doesn’t mean you should shoehorn it into the category of rule!
“play with this toy” still strikes me as natively a heuristic/approximation to the goal of “do things I enjoy”. The interlinking parts of my brain that decided to bring that forward is good at its job, but also dumb because it doesn’t do any higher order thinking. I follow that heuristic only because I expect to enjoy it—the heuristic providing that information. If I had another part of my consideration that pushed me towards considering whether that is a good plan, I might realize that I haven’t actually enjoyed playing with a teddy bear in years despite still feeling nostalgia for that. I’m not sure I see the gap between consequentialism and this. I don’t have the brain capacity to consider every impulse I get, but I do want to consider agents other than AIXI to be a consequentialist.
I think there’s a space in there for a theory of minds, but I expect it would be more mechanistic or descriptive rather than a moral theory. Ala shard theory.
Or, alternatively, even if you don’t buy my view that the majority of my heuristics can be cast as approximations of consequentialist propositions, then deontology/virtue ethics are not natural theories either by your descriptions. They miss a lot of complexity even within their usual remit.
No problem about long reply, I think your arguments are good and give me a lot to think about.
I just thought of another possible classification: “zeroth-order consequentialist” (care about doing the action but not because of consequences), “first-order consequentialist” (care about consequences), “second-order consequentialist” (care about someone else being able to choose what to do). I guess you’re right that all of these can be translated into first-order. But by the same token, everything can be translated to zeroth-order. And the translation from second to first feels about as iffy as the translation from first to zeroth. So this still feels fuzzy to me, I’m not sure what is right.