Continuing from what I said in my last comment about the more general problem with Expected Utility Maximizing, I think I might have a solution. I may be entirely wrong, so any criticism is welcome.
Instead of calculating Expected Utility, calculate the probability that an action will result in a higher utility than another action. Choose the one that is more likely to end up with a higher utility. For example, if giving Pascal’s mugger the money only has a one out of a trillionth chance of ending up with a higher utility than not giving him your money, you wouldn’t give it.
Now there is an apparent inconsistency with this system. If there is a lottery, and you have a 1⁄100 chance of winning, you would never buy a ticket. Even if the reward is $200 and the cost of a ticket only $1. Or even regardless how big the reward is. However if you are offered the chance to buy a lot of tickets all at once, you would do so, since the chance of winning becomes large enough to outgrow the chance of not winning.
However I don’t think that this is a problem. If you expect to play the lottery a bunch of times in a row, then you will choose to buy the ticket, because making that choice in this one instance also means that you will make the same choice in every other instance. Then the probability of ending up with more money at the end of the day is higher.
So if you expect to play the lottery a lot, or do other things that have low chances of ending up with high utilities, you might participate in them. Then when all is done, you are more likely to end up with a higher utility than if you had not done so. However if you get in a situation with an absurdly low chance of winning, it doesn’t matter how large the reward is. You wouldn’t participate, unless you expect to end up in the same situation an absurdly large number of times.
This method is consistent, it seems to “work” in that most agents that follow it will end up with higher utilities than agents that don’t follow it, and Expected Utility is just a special case of it that only happens when you expect to end up in similar situations a lot. It also seems closer to how humans actually make decisions. So can anyone find something wrong with this?
So if I’m getting what you’re saying correctly, it would not sacrifice a single cent for a 49% chance to save a human life?
And on the other hand it could be tempted to a game where it’d have 51% chance of winning a cent, and 49% chance of being destroyed?
If the solution for the problem of infinitesmal probabilities, is to effectively ignore every probability under 50%, that’s a solution that’s worse than the problem...
I stupidly didn’t consider that kind of situation for some reason… Back to the drawing board I guess.
Though to be fair it would still come out ahead 51% of the time, and in a real world application it would probably choose to spend the penny, since it would expect to make choices similarly in the future, and that would help it come out ahead an even higher percent of the time.
But yes, a 51% chance of losing a penny for nothing probably shouldn’t be worth more than a 49% chance at saving a life for a penny. However allowing a large enough reward to outweigh a small enough probability means the system will get stuck in situations where it is pretty much guaranteed to lose, on the slim, slim chance that it could get a huge reward.
Caring only about the percent of the time you “win” seemed like a more rational solution but I guess not.
Though another benefit of this system could be that you could have weird utility functions. Like a rule that says any outcome where one life is saved is worth more than any amount of money lost. Or Asimov’s three laws of robotics, which wouldn’t work under an Expected Utility function since it would only care about the first law. This is allowed because in the end all that matters is which outcomes you prefer to which other outcomes. You don’t have to turn utilities into numbers and do math on them.
Continuing from what I said in my last comment about the more general problem with Expected Utility Maximizing, I think I might have a solution. I may be entirely wrong, so any criticism is welcome.
Instead of calculating Expected Utility, calculate the probability that an action will result in a higher utility than another action. Choose the one that is more likely to end up with a higher utility. For example, if giving Pascal’s mugger the money only has a one out of a trillionth chance of ending up with a higher utility than not giving him your money, you wouldn’t give it.
Now there is an apparent inconsistency with this system. If there is a lottery, and you have a 1⁄100 chance of winning, you would never buy a ticket. Even if the reward is $200 and the cost of a ticket only $1. Or even regardless how big the reward is. However if you are offered the chance to buy a lot of tickets all at once, you would do so, since the chance of winning becomes large enough to outgrow the chance of not winning.
However I don’t think that this is a problem. If you expect to play the lottery a bunch of times in a row, then you will choose to buy the ticket, because making that choice in this one instance also means that you will make the same choice in every other instance. Then the probability of ending up with more money at the end of the day is higher.
So if you expect to play the lottery a lot, or do other things that have low chances of ending up with high utilities, you might participate in them. Then when all is done, you are more likely to end up with a higher utility than if you had not done so. However if you get in a situation with an absurdly low chance of winning, it doesn’t matter how large the reward is. You wouldn’t participate, unless you expect to end up in the same situation an absurdly large number of times.
This method is consistent, it seems to “work” in that most agents that follow it will end up with higher utilities than agents that don’t follow it, and Expected Utility is just a special case of it that only happens when you expect to end up in similar situations a lot. It also seems closer to how humans actually make decisions. So can anyone find something wrong with this?
So if I’m getting what you’re saying correctly, it would not sacrifice a single cent for a 49% chance to save a human life?
And on the other hand it could be tempted to a game where it’d have 51% chance of winning a cent, and 49% chance of being destroyed?
If the solution for the problem of infinitesmal probabilities, is to effectively ignore every probability under 50%, that’s a solution that’s worse than the problem...
I stupidly didn’t consider that kind of situation for some reason… Back to the drawing board I guess.
Though to be fair it would still come out ahead 51% of the time, and in a real world application it would probably choose to spend the penny, since it would expect to make choices similarly in the future, and that would help it come out ahead an even higher percent of the time.
But yes, a 51% chance of losing a penny for nothing probably shouldn’t be worth more than a 49% chance at saving a life for a penny. However allowing a large enough reward to outweigh a small enough probability means the system will get stuck in situations where it is pretty much guaranteed to lose, on the slim, slim chance that it could get a huge reward.
Caring only about the percent of the time you “win” seemed like a more rational solution but I guess not.
Though another benefit of this system could be that you could have weird utility functions. Like a rule that says any outcome where one life is saved is worth more than any amount of money lost. Or Asimov’s three laws of robotics, which wouldn’t work under an Expected Utility function since it would only care about the first law. This is allowed because in the end all that matters is which outcomes you prefer to which other outcomes. You don’t have to turn utilities into numbers and do math on them.