Let’s taboo “perfect”, and “utility” as well. As I see it, you are looking for an agent who is capable of choosing The Highest Number. This number does not exist. Therefore it can not be chosen. Therefore this agent can not exist. Because numbers are infinite. Infinity paradox is all I see.
Alternately, letting “utility” back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility which is the sum total of the time, matter, and energy in the universe. There will be an number which corresponds to this. Your opponent can choose a number higher than this but he will find the utility he seeks does not exist.
Alternately, letting “utility” back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility which is the sum total of the time, matter, and energy in the universe.
Why can’t my utility function be:
0 if I don’t get ice cream
1 if I get vanilla ice cream
infinity if I get chocolate ice cream
?
I.e. why should we forbid a utility function that returns infinity for certain scenarios, except insofar that it may lead to the types of problems that the OP is worrying about?
I was bringing the example into the presumed finite universe in which we live, where Maximum Utility = The Entire Universe. If we are discussing a finite-quantity problem than infinite quantity is ipso facto ruled out.
I think Nebu was making the point that while we normally use utility to talk about a kind of abstract gain, computers can be programmed with an arbitrary utility function. We would generally put certain restraints on it so that the computer/robot would behave consistently, but those are the only limitation. So even if there does not exist such a thing as infinite utility, a rational agent may still be required to solve for these scenarios.
I guess I’m asking “Why would a finite-universe necessarily dictate a finite utility score?”
In other words, why can’t my utility function be:
0 if you give me the entire universe minus all the ice cream.
1 if you give me the entire universe minus all the chocolate ice cream.
infinity if I get chocolate ice cream, regardless of how much chocolate ice cream I receive, and regardless of whether the rest of the universe is included with it.
“You are looking for an agent who is capable of choosing The Highest Number”—the agent wants to maximise utility, not to pick the highest number for its own sake, so that is misrepresenting my position. If you want to taboo utility, let’s use the word “lives saved” instead. Anyway, you say “Therefore this agent (the perfect life maximising agent) can not exist”, which is exactly what I was concluding. Concluding the exact same thing as I concluded, supports my argument, it doesn’t contradict it like you seem to think it does.
“Alternately, letting “utility” back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility”—my argument is that there does not exist perfect rationality within the imagined infinite universe. I said nothing about the actual, existing universe.
Sorry, I missed that you postulated an infinite universe in your game.
I don’t believe I am misrepresenting your position. “Maximizing utility” is achieved by-, and therefore can be defined as- “choosing the highest number”. The wants of the agent need not be considered. “Choosing the highest number” is an example of “doing something impossible”. I think your argument breaks down to “An agent who can do the impossible can not exist.” or “It is impossible to do the impossible”. I agree with this statement, but I don’t think it tells us anything useful. I think, but I haven’t thought it out fully, that it is the concept of infinity that is tripping you up.
What you’ve done is take my argument and transform it into an equivalent obvious statement. That isn’t a counter-argument. In fact, in mathematics, it is a method of proving a theorem.
If you read the other comments, then you’ll see that other people disagree with what I’ve said (and in a different manner than you), so I’m not just stating something obvious that everyone already knows and agrees with.
“What you’ve done is take my argument and transform it into an equivalent obvious statement. That isn’t a counter-argument. In fact, in mathematics, it is a method of proving a theorem.
If you read the other comments, then you’ll see that other people disagree with what I’ve said”
You’re welcome? Feel free to make use of my proof in your conversations with those guys. It looks pretty solid to me.
If a Perfect Rational Agent is one who can choose Maximum Finite Utility.
And Utility is numerically quantifiable and exists in infinite quantities.
And the Agent must choose the quantity of Utility by finite number.
Then no such agent can exist.
Therefore a Perfect Rational Agent does not exist in all possible worlds.
I suppose I’m agreeing but unimpressed. Might could be this is the wrong website for me. Any thought experiment involving infinity does run the risk of sounding dangerously close to Theology to my ears. Angels on pinheads and such. I’m not from around here and only dropped in to ask a specific question elsewhere. Cheers.
Let’s taboo “perfect”, and “utility” as well. As I see it, you are looking for an agent who is capable of choosing The Highest Number. This number does not exist. Therefore it can not be chosen. Therefore this agent can not exist. Because numbers are infinite. Infinity paradox is all I see.
Alternately, letting “utility” back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility which is the sum total of the time, matter, and energy in the universe. There will be an number which corresponds to this. Your opponent can choose a number higher than this but he will find the utility he seeks does not exist.
Why can’t my utility function be:
0 if I don’t get ice cream
1 if I get vanilla ice cream
infinity if I get chocolate ice cream
?
I.e. why should we forbid a utility function that returns infinity for certain scenarios, except insofar that it may lead to the types of problems that the OP is worrying about?
I was bringing the example into the presumed finite universe in which we live, where Maximum Utility = The Entire Universe. If we are discussing a finite-quantity problem than infinite quantity is ipso facto ruled out.
I think Nebu was making the point that while we normally use utility to talk about a kind of abstract gain, computers can be programmed with an arbitrary utility function. We would generally put certain restraints on it so that the computer/robot would behave consistently, but those are the only limitation. So even if there does not exist such a thing as infinite utility, a rational agent may still be required to solve for these scenarios.
I guess I’m asking “Why would a finite-universe necessarily dictate a finite utility score?”
In other words, why can’t my utility function be:
0 if you give me the entire universe minus all the ice cream.
1 if you give me the entire universe minus all the chocolate ice cream.
infinity if I get chocolate ice cream, regardless of how much chocolate ice cream I receive, and regardless of whether the rest of the universe is included with it.
“You are looking for an agent who is capable of choosing The Highest Number”—the agent wants to maximise utility, not to pick the highest number for its own sake, so that is misrepresenting my position. If you want to taboo utility, let’s use the word “lives saved” instead. Anyway, you say “Therefore this agent (the perfect life maximising agent) can not exist”, which is exactly what I was concluding. Concluding the exact same thing as I concluded, supports my argument, it doesn’t contradict it like you seem to think it does.
“Alternately, letting “utility” back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility”—my argument is that there does not exist perfect rationality within the imagined infinite universe. I said nothing about the actual, existing universe.
Sorry, I missed that you postulated an infinite universe in your game.
I don’t believe I am misrepresenting your position. “Maximizing utility” is achieved by-, and therefore can be defined as- “choosing the highest number”. The wants of the agent need not be considered. “Choosing the highest number” is an example of “doing something impossible”. I think your argument breaks down to “An agent who can do the impossible can not exist.” or “It is impossible to do the impossible”. I agree with this statement, but I don’t think it tells us anything useful. I think, but I haven’t thought it out fully, that it is the concept of infinity that is tripping you up.
What you’ve done is take my argument and transform it into an equivalent obvious statement. That isn’t a counter-argument. In fact, in mathematics, it is a method of proving a theorem.
If you read the other comments, then you’ll see that other people disagree with what I’ve said (and in a different manner than you), so I’m not just stating something obvious that everyone already knows and agrees with.
“What you’ve done is take my argument and transform it into an equivalent obvious statement. That isn’t a counter-argument. In fact, in mathematics, it is a method of proving a theorem. If you read the other comments, then you’ll see that other people disagree with what I’ve said” You’re welcome? Feel free to make use of my proof in your conversations with those guys. It looks pretty solid to me.
If a Perfect Rational Agent is one who can choose Maximum Finite Utility. And Utility is numerically quantifiable and exists in infinite quantities. And the Agent must choose the quantity of Utility by finite number. Then no such agent can exist. Therefore a Perfect Rational Agent does not exist in all possible worlds.
I suppose I’m agreeing but unimpressed. Might could be this is the wrong website for me. Any thought experiment involving infinity does run the risk of sounding dangerously close to Theology to my ears. Angels on pinheads and such. I’m not from around here and only dropped in to ask a specific question elsewhere. Cheers.
“Lives saved” is finite within a given light cone.
A very specific property of our universe, but not universes in general.