At that point, you face the same dilemma. Once you have allowed such highly capable entities to arise, how are you going to contain what they do or what people do with them? How are you going to keep the AIs or those who rely on and turn power over to the AIs from ending up in control? From doing great harm? The default answer is you can’t, and you won’t, but the only way you could hope to is again via highly intrusive surveillance and restrictions.
[...]
I don’t see any reason we couldn’t have a provably safe airplane, or at least an provably arbitrarily safe airplane
Are you using the same definition of “safe” in both places (i.e. “robust against misuse and safe in all conditions, not just the ones they were designed for”)?
Are you using the same definition of “safe” in both places (i.e. “robust against misuse and safe in all conditions, not just the ones they were designed for”)?