To me, at least, it seems clear that you should not take the opportunities to reduce your torture sentence. After all, if you repeatedly decide to take them, you will end up with a 0.5 chance of being highly uncomfortable and a 0.5 chance of being tortured for 3^^^^3 years. This seems like a really bad lottery, and worse than the one that lets me have a 0.5 chance of having an okay life.
FWIW, this conclusion is not clear to me. To return to one of my original points: I don’t think you can dodge this objection by arguing from potentially idiosyncratic preferences, even perfectly reasonable ones; rather, you need it to be the case that no rational agent could have different preferences. Either that, or you need to be willing to override otherwise rational individual preferences when making interpersonal tradeoffs.
To be honest, I’m actually not entirely averse to the latter option: having interpersonal trade-offs determined by contingent individual risk-preferences has never seemed especially well-justified to me (particularly if probability is in the mind). But I confess it’s not clear whether that route is open to you, given the motivation for your system as a whole.
More generally, I think the basic property of non-real-valued consistent preference orderings is that they value some things “infinitely more” than others.
FWIW, this conclusion is not clear to me. To return to one of my original points: I don’t think you can dodge this objection by arguing from potentially idiosyncratic preferences, even perfectly reasonable ones; rather, you need it to be the case that no rational agent could have different preferences. Either that, or you need to be willing to override otherwise rational individual preferences when making interpersonal tradeoffs.
Yes, that’s correct. It’s possible that there are some agents with consistent preferences that really would wish to get extraordinarily uncomfortable to avoid the torture. My point was just that this doesn’t seem like it would would be a common thing for agents to want.
Still, it is conceivable that there are at least a few agents out their that would consistently want to opt for the 0.5 chance of being extremely uncomfortable option, and I do suppose it would be best to respect their wishes. This is a problem that I hadn’t previously fully appreciated, so I would like to thank you for brining it up.
Luckily, I think I’ve finally figured out a way to adapt my ethical system to deal with this. That is, the adaptation will allow for agents to choose the extreme-discomfort-from-dust-specks option if that is what they wish for my my ethical system to respect their preferences. To do this, allow for the measure to satisfaction to include infinitesimals. Then, to respect the preferences of such agents, you just need need to pick the right satisfaction measure.
Consider the agent that for which each 50 years of torture causes a linear decrease in their utility function. For simplicity, imagine torture and discomfort are the only things the agent cares about; they have no other preferences; also assume that the agent dislike torture more than it dislikes discomfort, but only be a finite amount. Since the agent’s utility function/satisfaction measure is linear, I suppose being tortured for an eternity would be infinitely worse for the agent than being tortured for a finite amount of time. So, assign satisfaction 0 to the scenario in which the agent is tortured for eternity. And if the agent is instead tortured for n∈R years, let the agent’s satisfaction be 1−nϵ, where ϵ is whatever infinitesimal number you want. If my understanding of infinitesimals is correct, I think this will do what we want it to do in terms having agents using my ethical system respect the agent’s preferences.
Specifically, since being tortured forever would be infinitely worse than being tortured for a finite amount of time, any finite amount of torture would be accepted to decrease the chance of infinite torture. And this is what maximizing this satisfaction measure does: for any lottery, changing the chance of infinite torture has finite affect on expected satisfaction, whereas changing the chance of finite torture only has infinitesimal effect, so so avoiding infinite torture would be prioritized.
Further, among lotteries involving finite amounts of torture, it seems the ethical system using this satisfaction measure continues to do what what it’s supposed to do. For example, consider the choice between the previous two options:
A 0.5 chance of being tortured for 3^^^^3 years and a 0.5 chance of being fine.
A 0.5 chance of 3^^^^3 − 9999999 years of torture and 0.5 chance of being extraordinarily uncomfortable.
If I’m using my infinitesimal math right, the expected satisfaction of taking option 1 would be (0.5∗3↑↑↑↑3ϵ+0.5∗ϵ), and the expected satisfaction of taking option 2 would be 0.5∗(3↑↑↑↑3−9999999)ϵ∗0.5∗mϵ, for some m<<3↑↑↑↑3. Thus, to maximize this agent’s satisfaction measure, my moral system would indeed let the agent give infinite priority to avoiding infinite torture, the ethical system would itself consider the agent to get infinite torture infinitely-worse than getting finite torture, and would treat finite amounts of torture as decreasing satisfaction in a linear manner. And, since the utility measure is still technically bounded, it would still avoid the problem with utility monsters.
(In case it was unclear, ↑ is Knuth’s up-arrow notion, just like “^”.)
FWIW, this conclusion is not clear to me. To return to one of my original points: I don’t think you can dodge this objection by arguing from potentially idiosyncratic preferences, even perfectly reasonable ones; rather, you need it to be the case that no rational agent could have different preferences. Either that, or you need to be willing to override otherwise rational individual preferences when making interpersonal tradeoffs.
To be honest, I’m actually not entirely averse to the latter option: having interpersonal trade-offs determined by contingent individual risk-preferences has never seemed especially well-justified to me (particularly if probability is in the mind). But I confess it’s not clear whether that route is open to you, given the motivation for your system as a whole.
That makes sense, thanks.
Yes, that’s correct. It’s possible that there are some agents with consistent preferences that really would wish to get extraordinarily uncomfortable to avoid the torture. My point was just that this doesn’t seem like it would would be a common thing for agents to want.
Still, it is conceivable that there are at least a few agents out their that would consistently want to opt for the 0.5 chance of being extremely uncomfortable option, and I do suppose it would be best to respect their wishes. This is a problem that I hadn’t previously fully appreciated, so I would like to thank you for brining it up.
Luckily, I think I’ve finally figured out a way to adapt my ethical system to deal with this. That is, the adaptation will allow for agents to choose the extreme-discomfort-from-dust-specks option if that is what they wish for my my ethical system to respect their preferences. To do this, allow for the measure to satisfaction to include infinitesimals. Then, to respect the preferences of such agents, you just need need to pick the right satisfaction measure.
Consider the agent that for which each 50 years of torture causes a linear decrease in their utility function. For simplicity, imagine torture and discomfort are the only things the agent cares about; they have no other preferences; also assume that the agent dislike torture more than it dislikes discomfort, but only be a finite amount. Since the agent’s utility function/satisfaction measure is linear, I suppose being tortured for an eternity would be infinitely worse for the agent than being tortured for a finite amount of time. So, assign satisfaction 0 to the scenario in which the agent is tortured for eternity. And if the agent is instead tortured for n∈R years, let the agent’s satisfaction be 1−nϵ, where ϵ is whatever infinitesimal number you want. If my understanding of infinitesimals is correct, I think this will do what we want it to do in terms having agents using my ethical system respect the agent’s preferences.
Specifically, since being tortured forever would be infinitely worse than being tortured for a finite amount of time, any finite amount of torture would be accepted to decrease the chance of infinite torture. And this is what maximizing this satisfaction measure does: for any lottery, changing the chance of infinite torture has finite affect on expected satisfaction, whereas changing the chance of finite torture only has infinitesimal effect, so so avoiding infinite torture would be prioritized.
Further, among lotteries involving finite amounts of torture, it seems the ethical system using this satisfaction measure continues to do what what it’s supposed to do. For example, consider the choice between the previous two options:
A 0.5 chance of being tortured for 3^^^^3 years and a 0.5 chance of being fine.
A 0.5 chance of 3^^^^3 − 9999999 years of torture and 0.5 chance of being extraordinarily uncomfortable.
If I’m using my infinitesimal math right, the expected satisfaction of taking option 1 would be (0.5∗3↑↑↑↑3ϵ+0.5∗ϵ), and the expected satisfaction of taking option 2 would be 0.5∗(3↑↑↑↑3−9999999)ϵ∗0.5∗mϵ, for some m<<3↑↑↑↑3. Thus, to maximize this agent’s satisfaction measure, my moral system would indeed let the agent give infinite priority to avoiding infinite torture, the ethical system would itself consider the agent to get infinite torture infinitely-worse than getting finite torture, and would treat finite amounts of torture as decreasing satisfaction in a linear manner. And, since the utility measure is still technically bounded, it would still avoid the problem with utility monsters.
(In case it was unclear, ↑ is Knuth’s up-arrow notion, just like “^”.)