The simbox idea seems like a valuable guide for safely testing AIs, even if the rest of the post turns out to be wrong.
Here’s my too-terse summary of the post’s most important (and more controversial) proposal: have the AI grow up in an artificial society, learning self-empowerment and learning to model other agents. Use something like retargeting the search to convert the AI’s goals from self-empowerment to empowering other agents.
I’m reaffirming my relatively extensive review of this post.
The simbox idea seems like a valuable guide for safely testing AIs, even if the rest of the post turns out to be wrong.
Here’s my too-terse summary of the post’s most important (and more controversial) proposal: have the AI grow up in an artificial society, learning self-empowerment and learning to model other agents. Use something like retargeting the search to convert the AI’s goals from self-empowerment to empowering other agents.