DO NOT DEPLOY THE AGI UNTIL YOU HAVE COMPLETED THIS CHECKLIST.
Check the cryptographic signature of the utility function against MIRI’s public key.
Have someone who has memorized the known-benevolent utility function you plan to deploy check that it matches their memory exactly. If no such person is available, do not deploy.
Make sure that the code references that utility function, and not another one.
Make sure the code is set to maximize utility, not minimize it.
Deploy the AGI.
(This was written in jest, and is almost certainly incomplete or wrong. Do not use when deploying a real super-intelligent AGI.)
Super-intelligence deployment checklist:
DO NOT DEPLOY THE AGI UNTIL YOU HAVE COMPLETED THIS CHECKLIST.
Check the cryptographic signature of the utility function against MIRI’s public key.
Have someone who has memorized the known-benevolent utility function you plan to deploy check that it matches their memory exactly. If no such person is available, do not deploy.
Make sure that the code references that utility function, and not another one.
Make sure the code is set to maximize utility, not minimize it.
Deploy the AGI.
(This was written in jest, and is almost certainly incomplete or wrong. Do not use when deploying a real super-intelligent AGI.)
I think a shorter checklist would do (or longer, depending on how you look at it):
DO NOT DEPLOY THE AGI UNTIL YOU HAVE COMPLETED THIS CHECKLIST.
GO TO 1.