(Note that I said preferences and not utility function. There is no assumption that your preferences HAVE to be a utility function, or at least I don’t think so. Eliezer might have a different view).
Your preferences are a utility function if they’re consistent, but if you’re a human, they aren’t.
Consistent in what sense? Utility function over what domain? Under what prior? In this context, some unjustified assumptions, although understandably traditional to a point where objecting is weird.
Your preferences are a utility function if they’re consistent, but if you’re a human, they aren’t.
Consistent in what sense? Utility function over what domain? Under what prior? In this context, some unjustified assumptions, although understandably traditional to a point where objecting is weird.