Absolutely it is the case that utility should be bounded. However as best I can tell you’ve left out the most fundamental reason why, so I think I should explain that here. (Perhaps I should make this a separate post?)
The basic question is: Where do utility functions come from? Like, why should one model a rational agent as having a utility function at all? The answer of course is either the VNM theorem or Savage’s theorem, depending on whether or not you’re pre-assuming probability (you shouldn’t, really, but that’s another matter). Right, both these theorems take the form of, here’s a bunch of conditions any rational agent should obey, let’s show that such an agent must in fact be acting according to a utility function (i.e. trying to maximize its expected value).
Now here’s the thing: The utility functions output by Savage’s theorem are always bounded. Why is that? Well, essentially, because otherwise you could set up a St. Petersburg paradox that would contradict the assumed rationality conditions (in short, you can set up two gambles, both of “infinite expected utility”, but where one dominates the other, and show that both A. the agent must prefer the first to the second, but also B. the agent must be indifferent between them, contradiction). Thus we conclude that the utility function must be bounded.
OK, but what if we base things around the VNM theorem, then? It requires pre-assuming the notion of probability, but the utility functions output by the VNM theorem aren’t guaranteed to be bounded.
Here’s the thing: The VNM theorem only guarantees that the utility function it outputs works for finite gambles. Seriously. The VNM theorem gives no guarantee that the agent is acting according to the specified utility function when presented with a gamble with infinitely many possible outcomes, only when presented with a gamble with finitely many outcomes.
Similarly, with Savage’s theorem, the assumption that forces utility functions to be bounded—P7 -- is the same one that guarantees that the utility function works for infinite gambles. You can get rid of P7, and you’ll no longer be guaranteed to get a bounded utility function, but neither will you be guaranteed that the utility function will work for gambles with infinitely many possible outcomes.
This means that, fundamentally, if you want to work with infinite gambles, you need to only be talking about bounded utility functions. If you talk about infinite gambles in the context of unbounded utility functions, well, you’re basically talking nonsense, because there’s just absolutely no guarantee that the utility function you’re using applies in such a situation. The problems of unbounded utility that Eliezer keeps pointing out, that he insists we need to solve, really are just straight contradictions arising from him making bad assumptions that need to be thrown out. Like, they all stem from him assuming that unbounded utility functions work in the case of infinite gambles, and there simply is no such guarantee; not in the VNM theorem, not in Savage’s theorem.
If you’re assuming infinite gambles, you need to assume bounded utility functions, or else you need to accept that in cases of infinite gambles the utility function doesn’t actually apply—making the utility function basically useless, because, well, everything has infinitely many possible outcomes. Between a utility function that remains valid in the face of infinite gambles, and unbounded utility, it’s pretty clear you should choose the former.
And between Savage’s axiom P7 and unbounded utility, it’s pretty clear you should choose the former. Because P7 is an assumption that directly describes a rationality condition on the agent’s preferences, a form of the sure-thing principle, one we can clearly see had better be true of any rational agent; while unbounded utility… means what, exactly, in terms of the agent’s preferences? Something, certainly, but not something we obviously need. And in fact we don’t need it.
As best I can tell, Eliezer keeps insisting we need unbounded utility functions out of some sort of commitment to total utilitarianism or something along the lines of such (that’s my summary of his position, anyway). I would consider that to be on much shakier ground (there are so many nonobvious large assumptions for something like that to even make sense, seriously I’m not even going into it) than obvious things like the sure-thing principle, or that a utility function is nearly useless if it’s not valid for infinite gambles. And like I said, as best I can tell, Eliezer keeps assuming that the utility function is valid in such situations even though there’s nothing guaranteeing this; and this assumption is just in contradiction with his assumption of an unbounded utility function. He should keep the validity assumption (which we need) and throw out the unboundedness one (which we don’t).
That, to my mind, is the most fundamental reason we should only be considering bounded utility functions!
I’m not familiar with Savage’s theorem, but I was aware of the things you said about the VNM theorem, and in fact, I often bring up the same arguments you’ve been making. The standard response that I hear is that some probability distributions cannot be updated to without an infinite amount of information (e.g. if a priori the probability of the nth outcome is proportional to 1/3^n, then there can’t be any evidence that could occur with nonzero probability that would convince you that the probability of the nth outcome is 1/2^n for each n), and there’s no need for a utility function to converge on gambles that it is impossible even in theory for you to be convinced are available options.
When I ask why they assume that their utility function should be valid on those infinite gambles that are possible for them to consider, if they aren’t assuming that their preference relation is closed in the strong topology (which implies that the utility function is bounded), they’ll say something like that their utility function not being valid where their preference relation is defined seems weirdly discontinuous (in some sense that they can’t quite formalize and definitely isn’t the preference relation being closed in the strong topology), or that the examples I gave them of VNM-rational preference relations for which the utility function isn’t valid for infinite gambles all have some other pathology, like that there’s an infinite gamble which is considered either better than all of or worse than all of the constituent outcomes, and there might be a representation theorem saying something like that has to happen, even though they can’t point me to one.
Anyway, I agree that that’s a more fundamental reason to only consider bounded utility functions, but I decided I could probably be more convincing by abandoning that line of argument, and showing that if you sweep convergence issues under the rug, unbounded utility functions still suggest insane behavior in concrete situations.
Huh, I only just saw this for some reason. Anyway, if you’re not familiar with Savage’s theorem, that’s why I wrote the linked article here about it! :)
The problems of unbounded utility that Eliezer keeps pointing out, that he insists we need to solve, really are just straight contradictions arising from him making bad assumptions that need to be thrown out. Like, they all stem from him assuming that unbounded utility functions work in the case of infinite gambles
Just to be clear, you’re not thinking of 3↑↑↑3 when you talk about infinite gambles, right?
I’m not sure I know what argument of Eliezer’s you’re talking about when you reference infinite gambles. Is there an example you can link to?
He means gambles that can have infinitely many different outcomes. This causes problems for unbounded utility functions because of the Saint Petersburg paradox.
But the way you solve the St Petersburg paradox in real life is to note that nobody has infinite money, nor infinite time, and therefore it doesn’t matter if your utility function spits out a weird outcome for it because you can have a prior of 0 that it will actually happen. Am I missing something?
I don’t have an example to hand of Eliezer’s remarks. By which, I remember seeing on old LW, but I can’t find it at the moment. (Note that I’m interpreting what he said… very charitably. What he actually said made considerably less sense, but we can perhaps steelman it as a strong commitment to total utilitarianism.)
Absolutely it is the case that utility should be bounded. However as best I can tell you’ve left out the most fundamental reason why, so I think I should explain that here. (Perhaps I should make this a separate post?)
The basic question is: Where do utility functions come from? Like, why should one model a rational agent as having a utility function at all? The answer of course is either the VNM theorem or Savage’s theorem, depending on whether or not you’re pre-assuming probability (you shouldn’t, really, but that’s another matter). Right, both these theorems take the form of, here’s a bunch of conditions any rational agent should obey, let’s show that such an agent must in fact be acting according to a utility function (i.e. trying to maximize its expected value).
Now here’s the thing: The utility functions output by Savage’s theorem are always bounded. Why is that? Well, essentially, because otherwise you could set up a St. Petersburg paradox that would contradict the assumed rationality conditions (in short, you can set up two gambles, both of “infinite expected utility”, but where one dominates the other, and show that both A. the agent must prefer the first to the second, but also B. the agent must be indifferent between them, contradiction). Thus we conclude that the utility function must be bounded.
OK, but what if we base things around the VNM theorem, then? It requires pre-assuming the notion of probability, but the utility functions output by the VNM theorem aren’t guaranteed to be bounded.
Here’s the thing: The VNM theorem only guarantees that the utility function it outputs works for finite gambles. Seriously. The VNM theorem gives no guarantee that the agent is acting according to the specified utility function when presented with a gamble with infinitely many possible outcomes, only when presented with a gamble with finitely many outcomes.
Similarly, with Savage’s theorem, the assumption that forces utility functions to be bounded—P7 -- is the same one that guarantees that the utility function works for infinite gambles. You can get rid of P7, and you’ll no longer be guaranteed to get a bounded utility function, but neither will you be guaranteed that the utility function will work for gambles with infinitely many possible outcomes.
This means that, fundamentally, if you want to work with infinite gambles, you need to only be talking about bounded utility functions. If you talk about infinite gambles in the context of unbounded utility functions, well, you’re basically talking nonsense, because there’s just absolutely no guarantee that the utility function you’re using applies in such a situation. The problems of unbounded utility that Eliezer keeps pointing out, that he insists we need to solve, really are just straight contradictions arising from him making bad assumptions that need to be thrown out. Like, they all stem from him assuming that unbounded utility functions work in the case of infinite gambles, and there simply is no such guarantee; not in the VNM theorem, not in Savage’s theorem.
If you’re assuming infinite gambles, you need to assume bounded utility functions, or else you need to accept that in cases of infinite gambles the utility function doesn’t actually apply—making the utility function basically useless, because, well, everything has infinitely many possible outcomes. Between a utility function that remains valid in the face of infinite gambles, and unbounded utility, it’s pretty clear you should choose the former.
And between Savage’s axiom P7 and unbounded utility, it’s pretty clear you should choose the former. Because P7 is an assumption that directly describes a rationality condition on the agent’s preferences, a form of the sure-thing principle, one we can clearly see had better be true of any rational agent; while unbounded utility… means what, exactly, in terms of the agent’s preferences? Something, certainly, but not something we obviously need. And in fact we don’t need it.
As best I can tell, Eliezer keeps insisting we need unbounded utility functions out of some sort of commitment to total utilitarianism or something along the lines of such (that’s my summary of his position, anyway). I would consider that to be on much shakier ground (there are so many nonobvious large assumptions for something like that to even make sense, seriously I’m not even going into it) than obvious things like the sure-thing principle, or that a utility function is nearly useless if it’s not valid for infinite gambles. And like I said, as best I can tell, Eliezer keeps assuming that the utility function is valid in such situations even though there’s nothing guaranteeing this; and this assumption is just in contradiction with his assumption of an unbounded utility function. He should keep the validity assumption (which we need) and throw out the unboundedness one (which we don’t).
That, to my mind, is the most fundamental reason we should only be considering bounded utility functions!
I’m not familiar with Savage’s theorem, but I was aware of the things you said about the VNM theorem, and in fact, I often bring up the same arguments you’ve been making. The standard response that I hear is that some probability distributions cannot be updated to without an infinite amount of information (e.g. if a priori the probability of the nth outcome is proportional to 1/3^n, then there can’t be any evidence that could occur with nonzero probability that would convince you that the probability of the nth outcome is 1/2^n for each n), and there’s no need for a utility function to converge on gambles that it is impossible even in theory for you to be convinced are available options.
When I ask why they assume that their utility function should be valid on those infinite gambles that are possible for them to consider, if they aren’t assuming that their preference relation is closed in the strong topology (which implies that the utility function is bounded), they’ll say something like that their utility function not being valid where their preference relation is defined seems weirdly discontinuous (in some sense that they can’t quite formalize and definitely isn’t the preference relation being closed in the strong topology), or that the examples I gave them of VNM-rational preference relations for which the utility function isn’t valid for infinite gambles all have some other pathology, like that there’s an infinite gamble which is considered either better than all of or worse than all of the constituent outcomes, and there might be a representation theorem saying something like that has to happen, even though they can’t point me to one.
Anyway, I agree that that’s a more fundamental reason to only consider bounded utility functions, but I decided I could probably be more convincing by abandoning that line of argument, and showing that if you sweep convergence issues under the rug, unbounded utility functions still suggest insane behavior in concrete situations.
Huh, I only just saw this for some reason. Anyway, if you’re not familiar with Savage’s theorem, that’s why I wrote the linked article here about it! :)
Just to be clear, you’re not thinking of 3↑↑↑3 when you talk about infinite gambles, right?
I’m not sure I know what argument of Eliezer’s you’re talking about when you reference infinite gambles. Is there an example you can link to?
He means gambles that can have infinitely many different outcomes. This causes problems for unbounded utility functions because of the Saint Petersburg paradox.
But the way you solve the St Petersburg paradox in real life is to note that nobody has infinite money, nor infinite time, and therefore it doesn’t matter if your utility function spits out a weird outcome for it because you can have a prior of 0 that it will actually happen. Am I missing something?
No.
Huh, I only just saw this for some reason.
Anyway yes AlexMennen has the right of it.
I don’t have an example to hand of Eliezer’s remarks. By which, I remember seeing on old LW, but I can’t find it at the moment. (Note that I’m interpreting what he said… very charitably. What he actually said made considerably less sense, but we can perhaps steelman it as a strong commitment to total utilitarianism.)