As I said, at this point I am most interested in determining why we disagree,
The meaning of a phrase, primarily. And slightly about the proper use of an abstract concept.
A utility function should be a representation of my values. If my values are such that paying a mugger is the best option then I am glad to pay a mugger.
Suppose that you have somehow calculated that, with probability 10^(-100), the mugger will cause X to happen if you don’t pay him $5. Would you pay him? If you would pay him, then why?
If I were to pay him it would be because I happen to value not having a 10^(-100) chance of X happening more than I value $5.
As an aside, if you wouldn’t pay him then the definition of utility implies that u($5) > 10^(-100) u(X), which implies that u(X), and therefore the entire utility function, is bounded.
My utility function quite likely is bounded. Not because that is a way around pascal’s mugging. Simply because that happens to be what the arbitrary value system represented by this particular bunch of atoms happens to be.
Hm...it sounds like we agree on far more than I thought, then.
What I am saying is that my utility function is bounded because it would be ridiculous to be Pascal’s mugged, even in the hypothetical universe I created that disobeys komponisto’s priors. Put another way, I am simply not willing to seriously consider events at probabilities of, say, 10^(-10^(100)), because such events don’t happen. For this same reason, I have a hard time taking anyone seriously who claims to have an unbounded utility function, because they would then care about events that can’t happen in a sense at least as strong as the sense that 1 is not equal to 2.
Would you object to anything in the above paragraph? Thanks for bearing with me on this, by the way.
P.S. Am I the only one who is always tempted to write “mugged by Pascal” before realizing that this is comically different from being “Pascal’s mugged”?
Put another way, I am simply not willing to seriously consider events at probabilities of, say, 10^(-10^(100)), because such events don’t happen.
As far as I know they do happen. To know that such a number cannot represent an altogether esoteric feature of the universe that can nevertheless be the legitimate subject of infinite value I would need to know the smallest number that can be assigned to a quantum state.
(This objection is purely tangential. See below for significant disagreement.)
I have a hard time taking anyone seriously who claims to have an unbounded utility function, because they would then care about events that can’t happen in a sense at least as strong as the sense that 1 is not equal to 2.
That isn’t true. Someone can assign infinite utility to Australia winning the ashes if that is what they really want. I’d think them rather silly but that is just my subjective evaluation, nothing to do with maths.
To know that such a number cannot represent an altogether esoteric feature of the universe that can nevertheless be the legitimate subject of infinite value I would need to know the smallest number that can be assigned to a quantum state.
I think you are conflating quantum probabilities with Bayesian probabilities here, but I’m not sure. Unless you think this point is worth discussing further I’ll move on to your more significant disagreement.
Someone can assign infinite utility to Australia winning the ashes if that is what they really want. I’d think them rather silly but that is just my subjective evaluation, nothing to do with maths.
Hm...I initially wrote a two-paragraph explanation of why you were wrong, then deleted it because I changed my mind. So, I think we are making progress!
I initially thought I accorded disdain to unbounded utility functions for the same reason that I accorded disdain to ridiculous priors. But the difference is that your priors affect your epistemic state, and in the case of beliefs there is only one right answer. On the other hand, there is nothing inherently wrong with being a paperclip maximizer.
I think the actual issue I’m having is that I suspect that most people who claim to have unbounded utility functions would have been unwilling to make the trades implied by this before reading about VNM utility / “Shut up and multiply”. So my objection is not that unbounded utility functions are inherently wrong, but that they cannot possibly reflect the preferences of a human.
I think the actual issue I’m having is that I suspect that most people who claim to have unbounded utility functions would have been unwilling to make the trades implied by this before reading about VNM utility / “Shut up and multiply”. So my objection is not that unbounded utility functions are inherently wrong, but that they cannot possibly reflect the preferences of a human.
The meaning of a phrase, primarily. And slightly about the proper use of an abstract concept.
A utility function should be a representation of my values. If my values are such that paying a mugger is the best option then I am glad to pay a mugger.
If I were to pay him it would be because I happen to value not having a 10^(-100) chance of X happening more than I value $5.
My utility function quite likely is bounded. Not because that is a way around pascal’s mugging. Simply because that happens to be what the arbitrary value system represented by this particular bunch of atoms happens to be.
Hm...it sounds like we agree on far more than I thought, then.
What I am saying is that my utility function is bounded because it would be ridiculous to be Pascal’s mugged, even in the hypothetical universe I created that disobeys komponisto’s priors. Put another way, I am simply not willing to seriously consider events at probabilities of, say, 10^(-10^(100)), because such events don’t happen. For this same reason, I have a hard time taking anyone seriously who claims to have an unbounded utility function, because they would then care about events that can’t happen in a sense at least as strong as the sense that 1 is not equal to 2.
Would you object to anything in the above paragraph? Thanks for bearing with me on this, by the way.
P.S. Am I the only one who is always tempted to write “mugged by Pascal” before realizing that this is comically different from being “Pascal’s mugged”?
As far as I know they do happen. To know that such a number cannot represent an altogether esoteric feature of the universe that can nevertheless be the legitimate subject of infinite value I would need to know the smallest number that can be assigned to a quantum state.
(This objection is purely tangential. See below for significant disagreement.)
That isn’t true. Someone can assign infinite utility to Australia winning the ashes if that is what they really want. I’d think them rather silly but that is just my subjective evaluation, nothing to do with maths.
I think you are conflating quantum probabilities with Bayesian probabilities here, but I’m not sure. Unless you think this point is worth discussing further I’ll move on to your more significant disagreement.
Hm...I initially wrote a two-paragraph explanation of why you were wrong, then deleted it because I changed my mind. So, I think we are making progress!
I initially thought I accorded disdain to unbounded utility functions for the same reason that I accorded disdain to ridiculous priors. But the difference is that your priors affect your epistemic state, and in the case of beliefs there is only one right answer. On the other hand, there is nothing inherently wrong with being a paperclip maximizer.
I think the actual issue I’m having is that I suspect that most people who claim to have unbounded utility functions would have been unwilling to make the trades implied by this before reading about VNM utility / “Shut up and multiply”. So my objection is not that unbounded utility functions are inherently wrong, but that they cannot possibly reflect the preferences of a human.
On this I believe we approximately agree.