Is the plan just to destroy all computers with say >1e15 flops of computing power? How does the nanobot swarm know what a “computer” is? What do you do about something like GPT-neo or SETI-at-home where the compute is distributed?
I’m still confused as to why you think task: “build an AI that destroys anything with >1e15 flops of computing power—except humans, of course” would be dramatically easier than the alignment problem.
Setting back civilization a generation (via catastrophe) seems relatively straightforward. Building a social consensus/religion that destroys anything “in the image of a mind” at least seems possible. Fine-tuning a nanobot swarm to destroy some but not all computers just sounds really hard to me.
Is the plan just to destroy all computers with say >1e15 flops of computing power? How does the nanobot swarm know what a “computer” is? What do you do about something like GPT-neo or SETI-at-home where the compute is distributed?
I’m still confused as to why you think task: “build an AI that destroys anything with >1e15 flops of computing power—except humans, of course” would be dramatically easier than the alignment problem.
Setting back civilization a generation (via catastrophe) seems relatively straightforward. Building a social consensus/religion that destroys anything “in the image of a mind” at least seems possible. Fine-tuning a nanobot swarm to destroy some but not all computers just sounds really hard to me.