You’re right that “those values are irrational” is a category mistake, if we’re being precise. But Houshalter has an important point...
Any time you violate the axioms of a coherent utility-maximization agent, e.g. falling for the Allais paradox, you can always use meta factors to argue why your revealed preferences actually were coherent.
Like, “Yes the money pump just took some of my money, but you haven’t considered that the pump made a pleasing whirring sound which I enjoyed, which definitely outweighed the value of the money it pumped from me.”
While that may be a coherent response, we know that humans are born being somewhat farther-than-optimal from the ideal utility maximizer, and practicing the art of rationality adds value to their lives by getting them somewhat closer to the ideal than where they started.
A “rationality test” is a test that provides Bayesian evidence to distinguish people earlier vs. later on this path toward a more reflectively coherent utility function.
Having so grounded all the terms, I mostly agree with pwno and Houshalter.
you can always use meta factors to argue why your revealed preferences actually were coherent.
Three observations. First, those aren’t meta factors, those are just normal positive terms in the utility function that one formulation ignores and another one includes. Second, “you can always use” does not necessarily imply that the argument is wrong. Third, we are not arguing about coherency—why would the claim that, say, I value the perception of myself as someone who votes for X more than 10c be incoherent?
we know that humans are born being somewhat farther-than-optimal from the ideal utility maximizer, and practicing the art of rationality adds value to their lives by getting them somewhat closer to the ideal than where they started.
I disagree, both with the claim that getting closer to the ideal of a perfect utility maximizer necessarily adds value to people’s lives, and with the interpretation of the art of rationality as the art of getting people to be more like that utility maximizer.
Besides, there is still the original point: even if you posit some entilty as a perfect utility maximizer, what would its utility function include? Can you use the utility function to figure out which terms should go into the utility function? Colour me doubtful. In crude terms, how do you know what to maximize?
Well I guess I’ll focus on what seems to be our most fundamental disagreement, my claim that getting value from studying rationality usually involves getting yourself to be closer to an ideal utility maximizer (not necessarily all the way there).
Reading the Allais Paradox post can make a reader notice their contradictory preferences, and reflect on it, and subsequently be a little less contradictory, to their benefit. That seems like a good representative example of what studying rationality looks like and how it adds value.
Assume the following situation. You are very rich. You meet a poor old lady in a dark alley who carries a purse with her, with some money which is a lot from her perspective. Maybe it’s all her savings, maybe she just got lucky once and received it as a gift or as alms. If you mug her, nobody will ever find it out and you get to keep that money. Would you do it? As a utility maximization agent, based on what you just wrote, you should.
As a utility maximization agent, based on what you just wrote, you should.
Only if your utility function gives negligible weight to her welfare. Having a utility function is not at all the same thing as being wholly selfish.
(Also, your scenario is unrealistic; you couldn’t really be sure of not getting caught. If you’re very rich, the probability of getting caught doesn’t have to be very large to make this an expected loss even from a purely selfish point of view.)
Surely you ‘should’ only do something like this iff acquiring this amount of money has a higher utility to you than not ruining this lady’s day. Which, for most people, it doesn’t.
Since you’re saying ‘you are very rich’ and ‘some money which is a lot from her perspective’, you seem to be deliberately presenting gaining this money as very low utility, which you seem to assume should logically still outweigh what you seem to consider the zero utility of leaving the lady alone. But since I do actually give a duck about old ladies getting home safely (and, for that matter, about not feeling horribly guilty), mugging one has a pretty huge negative utility.
Have you read the LW sequences? Because like gjm explained, your question reveals a simple and objective misunderstanding of what utility functions look like when they model realistic people’s preferences.
You’re right that “those values are irrational” is a category mistake, if we’re being precise. But Houshalter has an important point...
Any time you violate the axioms of a coherent utility-maximization agent, e.g. falling for the Allais paradox, you can always use meta factors to argue why your revealed preferences actually were coherent.
Like, “Yes the money pump just took some of my money, but you haven’t considered that the pump made a pleasing whirring sound which I enjoyed, which definitely outweighed the value of the money it pumped from me.”
While that may be a coherent response, we know that humans are born being somewhat farther-than-optimal from the ideal utility maximizer, and practicing the art of rationality adds value to their lives by getting them somewhat closer to the ideal than where they started.
A “rationality test” is a test that provides Bayesian evidence to distinguish people earlier vs. later on this path toward a more reflectively coherent utility function.
Having so grounded all the terms, I mostly agree with pwno and Houshalter.
Three observations. First, those aren’t meta factors, those are just normal positive terms in the utility function that one formulation ignores and another one includes. Second, “you can always use” does not necessarily imply that the argument is wrong. Third, we are not arguing about coherency—why would the claim that, say, I value the perception of myself as someone who votes for X more than 10c be incoherent?
I disagree, both with the claim that getting closer to the ideal of a perfect utility maximizer necessarily adds value to people’s lives, and with the interpretation of the art of rationality as the art of getting people to be more like that utility maximizer.
Besides, there is still the original point: even if you posit some entilty as a perfect utility maximizer, what would its utility function include? Can you use the utility function to figure out which terms should go into the utility function? Colour me doubtful. In crude terms, how do you know what to maximize?
Well I guess I’ll focus on what seems to be our most fundamental disagreement, my claim that getting value from studying rationality usually involves getting yourself to be closer to an ideal utility maximizer (not necessarily all the way there).
Reading the Allais Paradox post can make a reader notice their contradictory preferences, and reflect on it, and subsequently be a little less contradictory, to their benefit. That seems like a good representative example of what studying rationality looks like and how it adds value.
You assert this as if it were an axiom. It doesn’t look like one to me. Show me the benefit.
And I still don’t understand why would I want to become an ideal utility maximizer.
For the sake of organization, I suggest discussing such things on the comment threads of Sequence posts.
If you could flip a switch right now that makes you an ideal utility maximizer, you wouldn’t do it?
Who gets to define my utility function? I don’t have one at the moment.
I would never flip a switch like that.
And why should we be utility maximization agents?
Assume the following situation. You are very rich. You meet a poor old lady in a dark alley who carries a purse with her, with some money which is a lot from her perspective. Maybe it’s all her savings, maybe she just got lucky once and received it as a gift or as alms. If you mug her, nobody will ever find it out and you get to keep that money. Would you do it? As a utility maximization agent, based on what you just wrote, you should.
Would you?
Only if your utility function gives negligible weight to her welfare. Having a utility function is not at all the same thing as being wholly selfish.
(Also, your scenario is unrealistic; you couldn’t really be sure of not getting caught. If you’re very rich, the probability of getting caught doesn’t have to be very large to make this an expected loss even from a purely selfish point of view.)
Surely you ‘should’ only do something like this iff acquiring this amount of money has a higher utility to you than not ruining this lady’s day. Which, for most people, it doesn’t.
Since you’re saying ‘you are very rich’ and ‘some money which is a lot from her perspective’, you seem to be deliberately presenting gaining this money as very low utility, which you seem to assume should logically still outweigh what you seem to consider the zero utility of leaving the lady alone. But since I do actually give a duck about old ladies getting home safely (and, for that matter, about not feeling horribly guilty), mugging one has a pretty huge negative utility.
Have you read the LW sequences? Because like gjm explained, your question reveals a simple and objective misunderstanding of what utility functions look like when they model realistic people’s preferences.