As several commenters have pointed out, the original problem does not supply a method for taking limits. Our analysis shows that the problem is ill posed: it has no unique solution unless we take on additional assumptions.
I disagree that the mathematics of original problem is ill-posed, and I think DanielLC made the same point. The point of contention seems to center on the use of infinities in the original problem, which is indeed an issue if they were manipulated as real numbers, but they were not. It is perfectly acceptable and mathematical to have a countably infinite set of objects, and to define a sequence of subsets corresponding to the time evolution of that set. Infinite sets are not defined as the limit of some sequence of finite sets! There is no ambiguity in the mathematics of original problem.*
Because the use of infinity in the original problem is not in the sense of a limit, there is no good reason to think that we should take limits, or that the limits of the solutions to the finite problems should correspond in any way to the solutions of the original problem.
Where there are ambiguities are in the use of the word “utility” and similar concepts as though they were well-defined in this context. And in this sense, I agree that the original problem is ill-posed.
* There are mathematical ambiguities in an unfavorable reading of the original problem, but the following steelman removes them: Biject the people with the natural numbers, and then transfer the nth person on day n.
I added a section called “Deciding how to decide” that (hopefully) deals with this issue appropriately. I also amended the conclusion, and added you as an acknowledgement.
One of the simplest and most intuitive is aggregative utilitarianism, in which we define a utility for each person, add them all together, and make the choice with the larger total utility.
I suggest using the phrase “additive utilitarianism” rather than “aggregative utilitarianism”. It was entirely my fault for saying aggregative utilitarianism in my comment, which was a misnomer; I got it mixed up with aggregative consequentialism. (All flavors of utilitarianism are by definition aggregative because they take into account the utilities from some collection of beings, but not all flavors are additive.)
Note: VincentYu has pointed out in the comments below that VNM utility may be able to deal with the infinites in this problem without taking limits.
Unfortunately, I think that ascribes too much power to VNM utility functions (that term itself is a LessWrongism; elsewhere, they would be called cardinal utility functions or just utility functions). If we had our hands on a VNM utility function, we would be okay (we simply ask it which option it prefers!), but the VNM theorem simply asserts the existence of a utility function given certain basic axioms, and it doesn’t give us the utility function! So, unfortunately, VNM utility also falls flat on its face unless we already know what we prefer. (An important point is that VNM utility functions cannot work with the “utility” described in the problem. It’s an unfortunate historical accident that the word “utility” is overloaded, because VNM utility requires careful handling.)
If we fail to specify the exact type of decision theory we’re using, it is entirely unclear whether taking infinite limits would lead to a self-consistent solution.
If we want to make a decision based on [additive] utility, the infinite problem is ill posed; it has no unique solution unless we take on additional assumptions.
Unfortunately, I think that ascribes too much power to VNM utility functions (that term itself is a LessWrongism; elsewhere, they would be called cardinal utility functions or just utility functions).
I actually don’t recall seeing the usage “VNM utility functions” on less wrong at all, prior to this thread. It may have occurred previously but certainly not with sufficient frequency as to be a ‘lesswrongism’. As you say, the “VNM” is unnecessary in that context since is all the VNM part does is say “it must have a utility function because it adheres to these axioms”.
It is sometimes necessary to explicitly refer to things other than ‘utility functions’ with a ‘VNM’ qualifier. This is largely to pre-empt pedants who, when reading unqualified usage ‘consequentialist’, are not willing to assume that it refers to the only kind of consequentialist that is ever significantly discussed here (those that have utility functions).
VNM utility also falls flat on its face unless we already know what we prefer.
Not quite, but the point stands. The actual requirement is that there is any way to collect any evidence at all about our preferences (or, to be even more general, any way to cause outcomes to be correlated to our preference).
I would like to include this issue in the post, but I want to make sure I understand it first. Tell me if this is right:
It is possible mathematically to represent a countably infinite number of immortal people, as well as the process of moving them between spheres. Further, we should not expect a priori that a problem involving such infinities would have a solution equivalent to those solutions reached by taking infinite limits of an analogous finite problem. Some confusion arises when we introduce the concept of “utility” to determine which of the two choices is better, since utility only serves as a basis on which to make decision for finite problems.
If that’s what you’re saying, I have a couple of questions.
Do you view the paradox as therefore unresolvable as stated, or would you claim that a different resolution is correct?
If I carefully restricted my claim about ill-posedness to the question of which choice is better from a utilitarian sense, would you agree with it?
It is possible mathematically to represent a countably infinite number of immortal people, as well as the process of moving them between spheres. Further, we should not expect a priori that a problem involving such infinities would have a solution equivalent to those solutions reached by taking infinite limits of an analogous finite problem.
That’s an accurate interpretation of my comment.
Some confusion arises when we introduce the concept of “utility” to determine which of the two choices is better, since utility only serves as a basis on which to make decision for finite problems.
I do think that confusion arises in this context from the concept of “utility”, but not because “utility only serves as a basis on which to make decision for finite problems.” The “utility” in the problem is clearly not that of VNM-utility (of which I previously gave a brief explanation) because we not assigning utility to actions, decisions, or choices (a VNM-utility function U would generally have no problem responding to an infinite set of choices, as it simply says: do argmax_{choice}(U(choice))). This severely undermines what we can do with the “utility” in the problem because we are left with the various flavors of aggregative utilitarianism, which suffer from intractable problems even in finite situations! Attempting to extend them to the situation at hand is problematic (and, as Kaj_Sotala remarked, dealing with infinities in aggregative consequentialism is the topic of one of Bostrom’s papers).
Do you view the paradox as therefore unresolvable as stated, or would you claim that a different resolution is correct?
I think that the appearance of the paradox is a consequence of unfamiliarity with infinite sets, and that it is not too surprising that our intuition appears to contradict itself in this context (by presenting each option as better than the other). The contradictory intuitions don’t correspond to a logical contradiction, so the apparent paradox needs no resolution. The actual problem (choosing between the two options) is a matter of preference, just as the choice between strawberry and chocolate is a matter of preference.
2. If I carefully restricted my claim about ill-posedness to the question of which choice is better from a utilitarian sense, would you agree with it?
Absolutely. I think aggregative utilitarianism (as a moral theory) is screwed even in finite scenarios, much less infinite scenarios. (But I also think aggregative utilitarianism is a good but ill-defined general standard for comparing consequences in real life.)
Ok, I think I’ve got it. I’m not familiar with VNM utility, and I’ll make sure to educate myself.
I’m going to edit the post to reflect this issue, but it may take me some time. It is clear (now that you point it out) that we can think of the ill-posedness coming from our insistence that the solution conform to aggregative utilitarianism, and it may be possible to sidestep the paradox if we choose another paradigm of decision theory. Still, I think it’s worth working as an example, because, as you say, AU is a good general standard, and many readers will be familiar with it. At the minimum, this would be an interesting finite AU decision problem.
I disagree that the mathematics of original problem is ill-posed, and I think DanielLC made the same point. The point of contention seems to center on the use of infinities in the original problem, which is indeed an issue if they were manipulated as real numbers, but they were not. It is perfectly acceptable and mathematical to have a countably infinite set of objects, and to define a sequence of subsets corresponding to the time evolution of that set. Infinite sets are not defined as the limit of some sequence of finite sets! There is no ambiguity in the mathematics of original problem.*
Because the use of infinity in the original problem is not in the sense of a limit, there is no good reason to think that we should take limits, or that the limits of the solutions to the finite problems should correspond in any way to the solutions of the original problem.
Where there are ambiguities are in the use of the word “utility” and similar concepts as though they were well-defined in this context. And in this sense, I agree that the original problem is ill-posed.
* There are mathematical ambiguities in an unfavorable reading of the original problem, but the following steelman removes them: Biject the people with the natural numbers, and then transfer the nth person on day n.
I added a section called “Deciding how to decide” that (hopefully) deals with this issue appropriately. I also amended the conclusion, and added you as an acknowledgement.
I suggest using the phrase “additive utilitarianism” rather than “aggregative utilitarianism”. It was entirely my fault for saying aggregative utilitarianism in my comment, which was a misnomer; I got it mixed up with aggregative consequentialism. (All flavors of utilitarianism are by definition aggregative because they take into account the utilities from some collection of beings, but not all flavors are additive.)
Unfortunately, I think that ascribes too much power to VNM utility functions (that term itself is a LessWrongism; elsewhere, they would be called cardinal utility functions or just utility functions). If we had our hands on a VNM utility function, we would be okay (we simply ask it which option it prefers!), but the VNM theorem simply asserts the existence of a utility function given certain basic axioms, and it doesn’t give us the utility function! So, unfortunately, VNM utility also falls flat on its face unless we already know what we prefer. (An important point is that VNM utility functions cannot work with the “utility” described in the problem. It’s an unfortunate historical accident that the word “utility” is overloaded, because VNM utility requires careful handling.)
Great, I think these are good clarifications!
I actually don’t recall seeing the usage “VNM utility functions” on less wrong at all, prior to this thread. It may have occurred previously but certainly not with sufficient frequency as to be a ‘lesswrongism’. As you say, the “VNM” is unnecessary in that context since is all the VNM part does is say “it must have a utility function because it adheres to these axioms”.
It is sometimes necessary to explicitly refer to things other than ‘utility functions’ with a ‘VNM’ qualifier. This is largely to pre-empt pedants who, when reading unqualified usage ‘consequentialist’, are not willing to assume that it refers to the only kind of consequentialist that is ever significantly discussed here (those that have utility functions).
Not quite, but the point stands. The actual requirement is that there is any way to collect any evidence at all about our preferences (or, to be even more general, any way to cause outcomes to be correlated to our preference).
For the moment, I’m going to strike the comment from the post. I don’t want to ascribe a viewpoint to VincentYu that he doesn’t actually hold.
I would like to include this issue in the post, but I want to make sure I understand it first. Tell me if this is right:
If that’s what you’re saying, I have a couple of questions.
Do you view the paradox as therefore unresolvable as stated, or would you claim that a different resolution is correct?
If I carefully restricted my claim about ill-posedness to the question of which choice is better from a utilitarian sense, would you agree with it?
That’s an accurate interpretation of my comment.
I do think that confusion arises in this context from the concept of “utility”, but not because “utility only serves as a basis on which to make decision for finite problems.” The “utility” in the problem is clearly not that of VNM-utility (of which I previously gave a brief explanation) because we not assigning utility to actions, decisions, or choices (a VNM-utility function U would generally have no problem responding to an infinite set of choices, as it simply says: do argmax_{choice}(U(choice))). This severely undermines what we can do with the “utility” in the problem because we are left with the various flavors of aggregative utilitarianism, which suffer from intractable problems even in finite situations! Attempting to extend them to the situation at hand is problematic (and, as Kaj_Sotala remarked, dealing with infinities in aggregative consequentialism is the topic of one of Bostrom’s papers).
I think that the appearance of the paradox is a consequence of unfamiliarity with infinite sets, and that it is not too surprising that our intuition appears to contradict itself in this context (by presenting each option as better than the other). The contradictory intuitions don’t correspond to a logical contradiction, so the apparent paradox needs no resolution. The actual problem (choosing between the two options) is a matter of preference, just as the choice between strawberry and chocolate is a matter of preference.
Absolutely. I think aggregative utilitarianism (as a moral theory) is screwed even in finite scenarios, much less infinite scenarios. (But I also think aggregative utilitarianism is a good but ill-defined general standard for comparing consequences in real life.)
Ok, I think I’ve got it. I’m not familiar with VNM utility, and I’ll make sure to educate myself.
I’m going to edit the post to reflect this issue, but it may take me some time. It is clear (now that you point it out) that we can think of the ill-posedness coming from our insistence that the solution conform to aggregative utilitarianism, and it may be possible to sidestep the paradox if we choose another paradigm of decision theory. Still, I think it’s worth working as an example, because, as you say, AU is a good general standard, and many readers will be familiar with it. At the minimum, this would be an interesting finite AU decision problem.
Thanks for all the time you’ve put into this.