Interesting—the failure mode that occurred to me is a paper-clipper which is designed to prefer virtual paper clips, so it turns the earth/the solar system/ the lightcone into computronium to run virtual paperclips.
If defining stay in the box is that hard, I’m not feeling hopeful about the possibility of defining protect humans.
Interesting—the failure mode that occurred to me is a paper-clipper which is designed to prefer virtual paper clips, so it turns the earth/the solar system/ the lightcone into computronium to run virtual paperclips.
If defining stay in the box is that hard, I’m not feeling hopeful about the possibility of defining protect humans.