But I was thrown by your suggestion above that such a paperclipper could actually understand first-person phenomenal states,
Was that claimed? The standard claim is that superintelligences can “model” other entities. That may not be enough to to understand qualia.
Was that claimed? The standard claim is that superintelligences can “model” other entities. That may not be enough to to understand qualia.