Unknown, I’ll bite. While you do point out some extremely counterintuitive consequences of positing that harms aggregate to an asymptote, accepting the dust specks as being worse than the torture is also extremely counterintuitive to most people.
For the moment, I accept the asymptote position, including the mathematical necessity you’ve pointed out.
So far this discussion has focused on harm to persons. But there are other forms of utility and disutility. Here’s the intuition pump I used on myself: the person concept is not so atomic to resist quantification—surely chimpanzees and dogs and fish and such must factor into humane utility calculations, even if they are not full persons. So are we then to prefer a universe with 3^^^3 banana slugs in it and no other life, over our own universe which contains (a much smaller number of) beings capable of greater feelings and thought? Absurd!
Perhaps in most realistic situations, the same experience happening to two different entities should count as almost exactly twice as good or bad as one instance of the experience. But I don’t think we should extend that intuition to these extreme cases with numbers like 3^^^3, else we must consider it an improvement when (say) Eliezer’s buggy AI decides to replace us with an incomprehensible number of slugs, each of which counts as one hundred-thousandth of a person.
At some point, the same experience repeated over and over again just doesn’t count.
Unknown, I’ll bite. While you do point out some extremely counterintuitive consequences of positing that harms aggregate to an asymptote, accepting the dust specks as being worse than the torture is also extremely counterintuitive to most people.
For the moment, I accept the asymptote position, including the mathematical necessity you’ve pointed out.
So far this discussion has focused on harm to persons. But there are other forms of utility and disutility. Here’s the intuition pump I used on myself: the person concept is not so atomic to resist quantification—surely chimpanzees and dogs and fish and such must factor into humane utility calculations, even if they are not full persons. So are we then to prefer a universe with 3^^^3 banana slugs in it and no other life, over our own universe which contains (a much smaller number of) beings capable of greater feelings and thought? Absurd!
Perhaps in most realistic situations, the same experience happening to two different entities should count as almost exactly twice as good or bad as one instance of the experience. But I don’t think we should extend that intuition to these extreme cases with numbers like 3^^^3, else we must consider it an improvement when (say) Eliezer’s buggy AI decides to replace us with an incomprehensible number of slugs, each of which counts as one hundred-thousandth of a person.
At some point, the same experience repeated over and over again just doesn’t count.