But the “meta-preferences” are a bit more worrying. Are they genuine meta-preferences? Especially since the second one is one that was more subconscious, and the third one looks more like a standard preference than a meta-preference. If the category of meta-preference is not clear, then that part of the research agenda needs to be improved.
I think one of the challenges is that, to me at least, it’s still unclear if we really have anything like meta-preferences that behave in systematic ways. That is, is there a systematic way in which our highly conditional preferences (which, in a very real sense, exist only momentarily at a particular decision point situated within the causal history of the universe) combine such that we can say more than that there are some statistical regularities to our preferences. Our preferences may manage to have some coherent statistical features about which we can make some stochastically consistent statements, but I think this falls short of what we are usually hoping for in terms of meta-preferences, and certainly seems to fall short in terms of how I understand you to be thinking about them (though maybe I misunderstand you: I think of you of thinking of meta-preferences as something that can ultimately be made to have nice mathematical properties, like some version of rationality, that would allow them to be optimized against without weird things happening).
I think one of the challenges is that, to me at least, it’s still unclear if we really have anything like meta-preferences that behave in systematic ways. That is, is there a systematic way in which our highly conditional preferences (which, in a very real sense, exist only momentarily at a particular decision point situated within the causal history of the universe) combine such that we can say more than that there are some statistical regularities to our preferences. Our preferences may manage to have some coherent statistical features about which we can make some stochastically consistent statements, but I think this falls short of what we are usually hoping for in terms of meta-preferences, and certainly seems to fall short in terms of how I understand you to be thinking about them (though maybe I misunderstand you: I think of you of thinking of meta-preferences as something that can ultimately be made to have nice mathematical properties, like some version of rationality, that would allow them to be optimized against without weird things happening).
“I want to be a more generous person”: what would you classify that as? Or “I want to want to write”?
I’d describe that as a statistical regularity over statistical regularities over preferences.