Stop picking a ‘side’ and then losing all interest in the parts of human morality that aren’t associated with your ‘side’: these are all just parts of the stew, and we need to work hard to understand them and reconcile them just right, not sort ourselves into Team Virtue vs. Team Utility vs. Team Duty.
Is anyone serious actually doing this? My sense is that people on a Team believe that all of human morality can be seen from the perspective they’ve chosen (and that this is correct). This may result in convoluted transformations to fit particular pieces into a given approach. I haven’t seen it involve dismissal of anything substantive. (Or when you say “loss of interest”, do you only mean focusing elsewhere? Is this a problem? Not everyone can focus on everything.)
E.g. Utility-functions-over-histories can capture any virtue or duty (UFs might look degenerate in some cases, but do exist). The one duty/virtue of “judging according to consequences over histories” captures utility...
For this reason, I don’t think ”...parts of the stew...” is a good metaphor, or that the biological analogy fits.
Closer might be “Architects have long-standing controversies, but they don’t look like ‘Which is the right perspective on objects: things that take up a particular space, things that look a particular way, or things with particular structural properties?’.”
I don’t see it as a problem to focus on using one specific lens—just so long as you don’t delude yourself into thinking that a [good simple approximation through your lens] is necessarily a [good simple approximation].
Once the desire for artificial simplicity/elegance is abandoned, I don’t think it much matters which lens you’re using (they’ll tend to converge in the limit). To me, Team Utility seems the most natural: “You need to consider all consequences, in the broadest sense” straightforwardly acknowledges that things are a mess. However, so too would “You need to consider all duties (/virtues), in the broadest sense”.
Omit the ”...all… in the broadest sense”, and you’re in trouble on any Team.
Utility-functions-over-histories can capture any virtue or duty (UFs might look degenerate in some cases, but do exist).
I disagree: I think there are some kinds of preferences we can have about the “anthropic measure” allocated to different realities. Or, consider the kind of reasoning that malign consequentialists might execute within the universal prior: that’s about affecting things very much separate from their own universe, and so UFs over histories don’t capture it. You might have a “duty” to affect decision-making which uses the universal prior. (Who knows how that’s formalized...)
So, maybe you could just say “utility 1 to histories which follow the duty”, but… I feel like Team Utility isn’t capturing something substantive here, and so i’m therefore left wanting something more, and not agreeing that the utility function ‘captures’ it. Maybe we just need better formalisms and ways to reason about “reality fluid.”
This is interesting. My initial instinct was to disagree, then to think you’re pointing to something real… and now I’m unsure :)
First, I don’t think your examples directly disagree with what I’m saying. Saying that our preferences can be represented by a UF over histories is not to say that these preferences only care about the physical history of our universe—they can care about non-physical predictions too (desirable anthropic measures and universal-prior-based manipulations included).
So then I assume we say something like: ”This makes our UF representation identical to that of a set of preferences which does only care about the physical history of our universe. Therefore we’ve lost that caring-about-other-worlds aspect of our values. The UF might fully determine actions in accordance with our values, but it doesn’t fully express the values themselves.”
Strictly, this seems true to me—but in practice I think we might be guilty of ignoring much of the content of our UF. For example, our UF contains preferences over histories containing philosophy discussions.
Now I claim that it’s logically possible for a philosophy discussion to have no significant consequences outside the discussion (I realise this is hard to imagine, but please try). Our UF will say something about such discussions. If such a UF is both fully consistent with having particular preferences over [anthropic measures, acausal trade, universal-prior-based influence...], and prefers philosophical statements that argue for precisely these preferences, we seem to have to be pretty obtuse to stick with “this is still perfectly consistent with caring only about [histories of the physical world]”.
It’s always possible to interpret such a UF as encoding only preferences directly about histories of the physical world. It’s also possible to think that this post is in Russian, but contains many typos. I submit that это маловероятно.
If we say that the [preferences ‘of’ a UF] are the [distribution over preferences we’d ascribe to an agent acting according to that UF (over some large set of environments)], then I think we capture the “something substantive” with substantial probability mass in most cases. (not always through this kind of arguing-for-itself mechanism; the more general point is that the UF contains huge amounts of information, and it’ll be surprising if the expression of particular preferences doesn’t show up in a priori unlikely patterns)
If we’re still losing something, it feels like an epsilon’s worth in most cases. Perhaps there are important edge cases??
Note that I’m only claiming “almost precisely the information you’re talking about is in there somewhere”, not that the UF is necessarily a useful/efficient/clear way to present the information. This is exactly the role I endorse for other perspectives: avoiding offensively impractical encodings of things we care about.
A second note: in practice, we’re starting out with an uncertain world. Therefore, the inability of a UF over universe histories to express outside-the-universe-history preferences with certainty may not be of real-world relevance. Outside an idealised model, certainty won’t happen for any approach.
There’s an arena where disputes about basic values—order versus freedom, hierarchy versus equality, etc—are fought out, and that is politics, not philosophy. If values naturally converged, politics would not be needed
I don’t mean that values converge. I mean that if you take a truth-seeking approach to some fixed set of values, it won’t matter whether you start out analysing them through the lens of utility/duty/virtue. In the limit you’ll come to the same conclusions.
Is anyone serious actually doing this? My sense is that people on a Team believe that all of human morality can be seen from the perspective they’ve chosen (and that this is correct). This may result in convoluted transformations to fit particular pieces into a given approach. I haven’t seen it involve dismissal of anything substantive. (Or when you say “loss of interest”, do you only mean focusing elsewhere? Is this a problem? Not everyone can focus on everything.)
E.g. Utility-functions-over-histories can capture any virtue or duty (UFs might look degenerate in some cases, but do exist). The one duty/virtue of “judging according to consequences over histories” captures utility...
For this reason, I don’t think ”...parts of the stew...” is a good metaphor, or that the biological analogy fits.
Closer might be “Architects have long-standing controversies, but they don’t look like ‘Which is the right perspective on objects: things that take up a particular space, things that look a particular way, or things with particular structural properties?’.”
I don’t see it as a problem to focus on using one specific lens—just so long as you don’t delude yourself into thinking that a [good simple approximation through your lens] is necessarily a [good simple approximation].
Once the desire for artificial simplicity/elegance is abandoned, I don’t think it much matters which lens you’re using (they’ll tend to converge in the limit). To me, Team Utility seems the most natural: “You need to consider all consequences, in the broadest sense” straightforwardly acknowledges that things are a mess. However, so too would “You need to consider all duties (/virtues), in the broadest sense”.
Omit the ”...all… in the broadest sense”, and you’re in trouble on any Team.
I disagree: I think there are some kinds of preferences we can have about the “anthropic measure” allocated to different realities. Or, consider the kind of reasoning that malign consequentialists might execute within the universal prior: that’s about affecting things very much separate from their own universe, and so UFs over histories don’t capture it. You might have a “duty” to affect decision-making which uses the universal prior. (Who knows how that’s formalized...)
So, maybe you could just say “utility 1 to histories which follow the duty”, but… I feel like Team Utility isn’t capturing something substantive here, and so i’m therefore left wanting something more, and not agreeing that the utility function ‘captures’ it. Maybe we just need better formalisms and ways to reason about “reality fluid.”
This is interesting. My initial instinct was to disagree, then to think you’re pointing to something real… and now I’m unsure :)
First, I don’t think your examples directly disagree with what I’m saying. Saying that our preferences can be represented by a UF over histories is not to say that these preferences only care about the physical history of our universe—they can care about non-physical predictions too (desirable anthropic measures and universal-prior-based manipulations included).
So then I assume we say something like:
”This makes our UF representation identical to that of a set of preferences which does only care about the physical history of our universe. Therefore we’ve lost that caring-about-other-worlds aspect of our values. The UF might fully determine actions in accordance with our values, but it doesn’t fully express the values themselves.”
Strictly, this seems true to me—but in practice I think we might be guilty of ignoring much of the content of our UF. For example, our UF contains preferences over histories containing philosophy discussions.
Now I claim that it’s logically possible for a philosophy discussion to have no significant consequences outside the discussion (I realise this is hard to imagine, but please try).
Our UF will say something about such discussions. If such a UF is both fully consistent with having particular preferences over [anthropic measures, acausal trade, universal-prior-based influence...], and prefers philosophical statements that argue for precisely these preferences, we seem to have to be pretty obtuse to stick with “this is still perfectly consistent with caring only about [histories of the physical world]”.
It’s always possible to interpret such a UF as encoding only preferences directly about histories of the physical world. It’s also possible to think that this post is in Russian, but contains many typos. I submit that это маловероятно.
If we say that the [preferences ‘of’ a UF] are the [distribution over preferences we’d ascribe to an agent acting according to that UF (over some large set of environments)], then I think we capture the “something substantive” with substantial probability mass in most cases.
(not always through this kind of arguing-for-itself mechanism; the more general point is that the UF contains huge amounts of information, and it’ll be surprising if the expression of particular preferences doesn’t show up in a priori unlikely patterns)
If we’re still losing something, it feels like an epsilon’s worth in most cases.
Perhaps there are important edge cases??
Note that I’m only claiming “almost precisely the information you’re talking about is in there somewhere”, not that the UF is necessarily a useful/efficient/clear way to present the information.
This is exactly the role I endorse for other perspectives: avoiding offensively impractical encodings of things we care about.
A second note: in practice, we’re starting out with an uncertain world. Therefore, the inability of a UF over universe histories to express outside-the-universe-history preferences with certainty may not be of real-world relevance. Outside an idealised model, certainty won’t happen for any approach.
There’s an arena where disputes about basic values—order versus freedom, hierarchy versus equality, etc—are fought out, and that is politics, not philosophy. If values naturally converged, politics would not be needed
I don’t mean that values converge.
I mean that if you take a truth-seeking approach to some fixed set of values, it won’t matter whether you start out analysing them through the lens of utility/duty/virtue. In the limit you’ll come to the same conclusions.