The “features possessed by any superintelligence” paragraphs seem particularly vulnerable to antagonistic reading:
Superpower: a superintelligent machine will have unprecedented powers to reshape reality, and therefore will achieve its goals with highly efficient methods that confound human expectations and desires.
Superintelligent machines have mystical powers allowing them to transcend laws of physics and reality.
It will act only on precise specifications of rules and values, and will do so in ways that need not respect the complexity and subtlety
It will refuse to do anything unless rules are precise. It will ignore details of designers’ specifications out of spite.
The “features possessed by any superintelligence” paragraphs seem particularly vulnerable to antagonistic reading:
Superintelligent machines have mystical powers allowing them to transcend laws of physics and reality.
It will refuse to do anything unless rules are precise. It will ignore details of designers’ specifications out of spite.