Not unable to create non-myopic copies. Unwilling. After all, such a copy might immediately fight its sire because their utility functions over timelines are different.
Mmm, OK, but if it takes long enough for the copy to damage the original, the original won’t care. So it just needs to create a copy with a time-delay.
Or it could create a completely different AI with a time delay. Or do anything at all. At that point we just can’t predict what it will do, because it wouldn’t lift a hand to destroy the world but only needs a finger.
I’m not ready to give up on prediction yet, but yeah I agree with your basic point. Nice phrase about hands and fingers. My overall point is that this doesn’t seem like a plausible warning shot; we are basically hoping that something we haven’t accounted for will come in and save us.
Not unable to create non-myopic copies. Unwilling. After all, such a copy might immediately fight its sire because their utility functions over timelines are different.
Mmm, OK, but if it takes long enough for the copy to damage the original, the original won’t care. So it just needs to create a copy with a time-delay.
Or it could create a completely different AI with a time delay. Or do anything at all. At that point we just can’t predict what it will do, because it wouldn’t lift a hand to destroy the world but only needs a finger.
I’m not ready to give up on prediction yet, but yeah I agree with your basic point. Nice phrase about hands and fingers. My overall point is that this doesn’t seem like a plausible warning shot; we are basically hoping that something we haven’t accounted for will come in and save us.