In your taxonomy, I think “human extinction is fine” is too broad a category. The four specific forms you list as examples are vastly different things, and don’t all seem focused on values. Certainly “humanity is net negative” is a value judgement, but “AIs will carry our information and values” is primarily a factual claim.
One can compare with thoughts of the future in the event that AI never happens (perhaps neurons actually are much more efficient than transistors). Surely no one thinks that in 10 million years there will still be creatures closely similar to present-day humans? Maybe we’ll have gone extinct, which would be bad, but more likely there will be one or many successor species that differ substantially from us. I don’t find that particularly distressing (though of course it could end up going badly, from our present viewpoint).
The factual claims involved here are of course open to question, and overlap a lot with factual claims regarding “alignment” (whatever that means). Dismissing it all as differing values seems to me to miss a lot.
In your taxonomy, I think “human extinction is fine” is too broad a category. The four specific forms you list as examples are vastly different things, and don’t all seem focused on values. Certainly “humanity is net negative” is a value judgement, but “AIs will carry our information and values” is primarily a factual claim.
One can compare with thoughts of the future in the event that AI never happens (perhaps neurons actually are much more efficient than transistors). Surely no one thinks that in 10 million years there will still be creatures closely similar to present-day humans? Maybe we’ll have gone extinct, which would be bad, but more likely there will be one or many successor species that differ substantially from us. I don’t find that particularly distressing (though of course it could end up going badly, from our present viewpoint).
The factual claims involved here are of course open to question, and overlap a lot with factual claims regarding “alignment” (whatever that means). Dismissing it all as differing values seems to me to miss a lot.