One must compare a plan with alternative plans, not with status quo.
Okay.
The previous plan was “don’t let AGI run free”, which in this case effectively preserves the status quo until someone breaks it.
I suppose you could revise that lower limit downward to the effects of the plan “turn on the UFAI that’s about to be turned on”. Like, steal the UFAI’s source code and instead of paperclips shaped like paperclips, make paperclips that spell “whoops”.
Okay.
The previous plan was “don’t let AGI run free”, which in this case effectively preserves the status quo until someone breaks it.
I suppose you could revise that lower limit downward to the effects of the plan “turn on the UFAI that’s about to be turned on”. Like, steal the UFAI’s source code and instead of paperclips shaped like paperclips, make paperclips that spell “whoops”.