Eliezer, to be clear, do you still think that 3^^^3 people having momentary eye irritations—from dust-specs—is worth torturing a single person for 50 years, or is there a possibility that you did the math incorrectly for that example? A proper utilitarian needs to consider the full range of outcomes—and their probabilities—associated with different alternatives. If the momentary eye irritation leads to a greater than 1/3^^^3 probability that someone will have an accident that leads to an outcome worse than 50 years of torture, then the dust specs are preferable. But if the chance of further negative consequences from momentary eye-irritation is so small as to be negligible, then we can consider the cost of the dust specs to be the linear sum of the hedonic loss across all of the people afflicted. The torture, on the other hand, has a significant probability of leading to further negative consequences that could persist across a life-span and impact those who care about that individual. If the tortured individual has a significant probability of committing suicide, then we need to consider all of the potential experiences and accomplishments that the person would have had over the course of their life-time—which could be indefinitely long, depending on how technology progresses—and the impact that the person would have had on others. And finally, as I think you would agree, we wouldn’t want to use an ethical utility function that only considered basic hedonic experience and ignored higher-level meaning. If you merely integrated all of the moments of pleasure/pain across a life-span, you wouldn’t have come close to calculating the value of that life. Music is worth more than the sum of the notes that went into the song. While your basic argument is valid and important, you probably—depending on the details of the argument—came to the wrong conclusion with respect to dust-specs and torture.
Eliezer, to be clear, do you still think that 3^^^3 people having momentary eye irritations—from dust-specs—is worth torturing a single person for 50 years, or is there a possibility that you did the math incorrectly for that example? A proper utilitarian needs to consider the full range of outcomes—and their probabilities—associated with different alternatives. If the momentary eye irritation leads to a greater than 1/3^^^3 probability that someone will have an accident that leads to an outcome worse than 50 years of torture, then the dust specs are preferable. But if the chance of further negative consequences from momentary eye-irritation is so small as to be negligible, then we can consider the cost of the dust specs to be the linear sum of the hedonic loss across all of the people afflicted. The torture, on the other hand, has a significant probability of leading to further negative consequences that could persist across a life-span and impact those who care about that individual. If the tortured individual has a significant probability of committing suicide, then we need to consider all of the potential experiences and accomplishments that the person would have had over the course of their life-time—which could be indefinitely long, depending on how technology progresses—and the impact that the person would have had on others. And finally, as I think you would agree, we wouldn’t want to use an ethical utility function that only considered basic hedonic experience and ignored higher-level meaning. If you merely integrated all of the moments of pleasure/pain across a life-span, you wouldn’t have come close to calculating the value of that life. Music is worth more than the sum of the notes that went into the song. While your basic argument is valid and important, you probably—depending on the details of the argument—came to the wrong conclusion with respect to dust-specs and torture.