Secondly, and more importantly, I question whether it is possible even in theory to produce infinite expected value. At some point you’ve created every possible flourishing mind in every conceivable permutation of eudaimonia, satisfaction, and bliss, and the added value of another instance of any of them is basically nil. In reality I would expect to reach a point where the universe is so damn good that there is literally nothing the Cosmic Flipper could offer me that would be worth risking it all.
This very much depends on the rate of growth.
For most human beings, this is probably right, because their values have a function that grows slower than logarithmic, which leads to bounds on the utility even assuming infinite consumption.
But it’s definitely possible in theory to generate utility functions that have infinite expected utility from infinite consumption.
You are however pointing to something very real here, and that’s the fact that utility theory loses a lot of it’s niceness in the infinite realm, and while there might be something like a utility theory that can handle infinity, it will have to lose a lot of very nice properties that it had in the finite case.
For most human beings, this is probably right, because their values have a function that grows slower than logarithmic, which leads to bounds on the utility even assuming infinite consumption.
Growing slower than logarithmic does not help. Only being bounded in the limit gives you, well, a bound in the limit.
You are however pointing to something very real here, and that’s the fact that utility theory loses a lot of it’s niceness in the infinite realm, and while there might be something like a utility theory that can handle infinity, it will have to lose a lot of very nice properties that it had in the finite case.
“Bounded utility solves none of the problems of unbounded utility.” Thus the title of something I’m working on, on and off.
The reason that bounded utility does not help is that any problem that arises at infinity will already practically arise at a sufficiently large finite stage. Repeated plays of the finite games discussed in that paper will eventually give you a payoff that has a high probability of being close (in relative terms) to the expected value. But the time it takes for this to happen grows exponentially with the lengths of the individual games. You are unlikely to ever see your theoretically expected value, however long you play. The infinite game is non-ergodic; the game truncated to finitely many steps and finite payoffs is ergodic only on impractical timescales.
Infinitude in problems like these is better understood as an approximation to the finite, rather than the other way round. (There’s a blog post by Terry Tao on this theme, but I’ve lost the reference to it.) The problems at infinity point to problems with the finite.
The reason that bounded utility does not help is that any problem that arises at infinity will already practically arise at a sufficiently large finite stage. Repeated plays of the finite games discussed in that paper will eventually give you a payoff that has a high probability of being close (in relative terms) to the expected value. But the time it takes for this to happen grows exponentially with the lengths of the individual games. You are unlikely to ever see your theoretically expected value, however long you play. The infinite game is non-ergodic; the game truncated to finitely many steps and finite payoffs is ergodic only on impractical timescales.
Infinitude in problems like these is better understood as an approximation to the finite, rather than the other way round. (There’s a blog post by Terry Tao on this theme, but I’ve lost the reference to it.) The problems at infinity point to problems with the finite.
I definitely agree that the problems of infinite utilities are approximately preserved by the finitary version of the problem, and while there are situations where you can get niceness assuming utilities are bounded (conditional on giving players exponentially large lifespans), it’s not the common or typical case.
Infinity makes things worse in that you no longer get any cases where nice properties like ergodicity or dominance are consistent with other properties, but yeah the finitary version is only a little better.
This very much depends on the rate of growth.
For most human beings, this is probably right, because their values have a function that grows slower than logarithmic, which leads to bounds on the utility even assuming infinite consumption.
But it’s definitely possible in theory to generate utility functions that have infinite expected utility from infinite consumption.
You are however pointing to something very real here, and that’s the fact that utility theory loses a lot of it’s niceness in the infinite realm, and while there might be something like a utility theory that can handle infinity, it will have to lose a lot of very nice properties that it had in the finite case.
See these 2 posts by Paul Christiano for why:
https://www.lesswrong.com/posts/hbmsW2k9DxED5Z4eJ/impossibility-results-for-unbounded-utilities
https://www.lesswrong.com/posts/gJxHRxnuFudzBFPuu/better-impossibility-result-for-unbounded-utilities
Growing slower than logarithmic does not help. Only being bounded in the limit gives you, well, a bound in the limit.
“Bounded utility solves none of the problems of unbounded utility.” Thus the title of something I’m working on, on and off.
It’s not ready yet. For a foretaste, some of the points it will make can be found in an earlier unpublished paper “Unbounded Utility and Axiomatic Foundations”, section 3.
The reason that bounded utility does not help is that any problem that arises at infinity will already practically arise at a sufficiently large finite stage. Repeated plays of the finite games discussed in that paper will eventually give you a payoff that has a high probability of being close (in relative terms) to the expected value. But the time it takes for this to happen grows exponentially with the lengths of the individual games. You are unlikely to ever see your theoretically expected value, however long you play. The infinite game is non-ergodic; the game truncated to finitely many steps and finite payoffs is ergodic only on impractical timescales.
Infinitude in problems like these is better understood as an approximation to the finite, rather than the other way round. (There’s a blog post by Terry Tao on this theme, but I’ve lost the reference to it.) The problems at infinity point to problems with the finite.
Thanks for catching that error, I did not realize this.
I think I got it from here:
https://www.lesswrong.com/posts/EhHdZ5yBgEvLLx6Pw/chad-jones-paper-modeling-ai-and-x-risk-vs-growth
I definitely agree that the problems of infinite utilities are approximately preserved by the finitary version of the problem, and while there are situations where you can get niceness assuming utilities are bounded (conditional on giving players exponentially large lifespans), it’s not the common or typical case.
Infinity makes things worse in that you no longer get any cases where nice properties like ergodicity or dominance are consistent with other properties, but yeah the finitary version is only a little better.