Alternately, letting “utility” back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility which is the sum total of the time, matter, and energy in the universe.
Why can’t my utility function be:
0 if I don’t get ice cream
1 if I get vanilla ice cream
infinity if I get chocolate ice cream
?
I.e. why should we forbid a utility function that returns infinity for certain scenarios, except insofar that it may lead to the types of problems that the OP is worrying about?
I was bringing the example into the presumed finite universe in which we live, where Maximum Utility = The Entire Universe. If we are discussing a finite-quantity problem than infinite quantity is ipso facto ruled out.
I think Nebu was making the point that while we normally use utility to talk about a kind of abstract gain, computers can be programmed with an arbitrary utility function. We would generally put certain restraints on it so that the computer/robot would behave consistently, but those are the only limitation. So even if there does not exist such a thing as infinite utility, a rational agent may still be required to solve for these scenarios.
I guess I’m asking “Why would a finite-universe necessarily dictate a finite utility score?”
In other words, why can’t my utility function be:
0 if you give me the entire universe minus all the ice cream.
1 if you give me the entire universe minus all the chocolate ice cream.
infinity if I get chocolate ice cream, regardless of how much chocolate ice cream I receive, and regardless of whether the rest of the universe is included with it.
Why can’t my utility function be:
0 if I don’t get ice cream
1 if I get vanilla ice cream
infinity if I get chocolate ice cream
?
I.e. why should we forbid a utility function that returns infinity for certain scenarios, except insofar that it may lead to the types of problems that the OP is worrying about?
I was bringing the example into the presumed finite universe in which we live, where Maximum Utility = The Entire Universe. If we are discussing a finite-quantity problem than infinite quantity is ipso facto ruled out.
I think Nebu was making the point that while we normally use utility to talk about a kind of abstract gain, computers can be programmed with an arbitrary utility function. We would generally put certain restraints on it so that the computer/robot would behave consistently, but those are the only limitation. So even if there does not exist such a thing as infinite utility, a rational agent may still be required to solve for these scenarios.
I guess I’m asking “Why would a finite-universe necessarily dictate a finite utility score?”
In other words, why can’t my utility function be:
0 if you give me the entire universe minus all the ice cream.
1 if you give me the entire universe minus all the chocolate ice cream.
infinity if I get chocolate ice cream, regardless of how much chocolate ice cream I receive, and regardless of whether the rest of the universe is included with it.