AIs that are actually FAIs (which include by definition my own survival) are negligible.
Formal preference is a more abstract concept than survival in particular, and even though all else equal, in usual situations, survival is preferable to non-survival, there could be situations even better than “survival”. It’s not out of the question “by definition” (you know better than to invoke this argument pattern).
Formal preference is one particular thing. You can’t specify additional details without changing the concept. If preference says that “survival” is a necessary component, that worlds without “survival” are equally worthless, then so be it. But it could say otherwise. You can’t study something and already know the answer, you can’t just assume to know that this property that intuitively appeals to you is unquestionably present. How do you know? I’d rather build on clear foundation, and remain in doubt about what I can’t yet see.
Formal preference is a more abstract concept than survival in particular, and even though all else equal, in usual situations, survival is preferable to non-survival, there could be situations even better than “survival”. It’s not out of the question “by definition” (you know better than to invoke this argument pattern).
Formal preference is one particular thing. You can’t specify additional details without changing the concept. If preference says that “survival” is a necessary component, that worlds without “survival” are equally worthless, then so be it. But it could say otherwise. You can’t study something and already know the answer, you can’t just assume to know that this property that intuitively appeals to you is unquestionably present. How do you know? I’d rather build on clear foundation, and remain in doubt about what I can’t yet see.