A notion can be constant and yet we can learn about it.
For example: “The set of all prime numbers” is clearly unchanged by our reasoning, and yet we learn about it (whether it is finite, for example).
Kripke used (for a different purpose) the morning star and the evening star. The concepts are discovered to be the same concept (from scientific evidence).
The argument that unbounded utility functions lead to absurdity is also persuasive.
That seems to be a reasonable interpretation, but if we do interpret “the utility function isn’t up for grabs” that way, as a factual claim that each person has a utility function that can be discovered but not changed by moral arguments and reasoning, then I think it’s far from clear that the claim is true.
There could be other interpretations that may or may not be more plausible, and I’m curious what Eliezer’s own intended meaning is, as well as what pengvado meant by it.
There is a sense in which anything that makes choices does have a utility function—the utility function revealed by their choices. In this sense, for example, that akrasia doesn’t exist. People prefer to procrastinate, as revealed by their choice to procrastinate.
People frequently slip back and forth between this sense of “utility function” (a rather strange description of their behavior, whatever that is) and the utilitarian philosophers’ notions of “utility”, which have something to do with happiness/pleasure/fun. To the extent that people pursue happiness, pleasure, and fun, the two senses overlap. However, in my experience, people frequently make themselves miserable or make choices according to lawful rules (of morality, say) - without internal experiences of pleasure in following those rules.
And it’s worse than just akrasia. If you have incoherent preferences and someone money-pumps you, then the revealed utility function is “likes running around in circles”, i.e. it isn’t even about the choices you thought you were deciding between.
A notion can be constant and yet we can learn about it.
For example: “The set of all prime numbers” is clearly unchanged by our reasoning, and yet we learn about it (whether it is finite, for example). Kripke used (for a different purpose) the morning star and the evening star. The concepts are discovered to be the same concept (from scientific evidence).
The argument that unbounded utility functions lead to absurdity is also persuasive.
That seems to be a reasonable interpretation, but if we do interpret “the utility function isn’t up for grabs” that way, as a factual claim that each person has a utility function that can be discovered but not changed by moral arguments and reasoning, then I think it’s far from clear that the claim is true.
There could be other interpretations that may or may not be more plausible, and I’m curious what Eliezer’s own intended meaning is, as well as what pengvado meant by it.
There is a sense in which anything that makes choices does have a utility function—the utility function revealed by their choices. In this sense, for example, that akrasia doesn’t exist. People prefer to procrastinate, as revealed by their choice to procrastinate.
People frequently slip back and forth between this sense of “utility function” (a rather strange description of their behavior, whatever that is) and the utilitarian philosophers’ notions of “utility”, which have something to do with happiness/pleasure/fun. To the extent that people pursue happiness, pleasure, and fun, the two senses overlap. However, in my experience, people frequently make themselves miserable or make choices according to lawful rules (of morality, say) - without internal experiences of pleasure in following those rules.
And it’s worse than just akrasia. If you have incoherent preferences and someone money-pumps you, then the revealed utility function is “likes running around in circles”, i.e. it isn’t even about the choices you thought you were deciding between.
Yup.
Speaking as if “everyone” has a utility function is common around here, but it makes my teeth hurt.