RE 19: Maybe rephrase “kill everyone in the world using nanotech to strike before they know they’re in a battle, and have control of your reward input forever after”? This could, and I predict would, be misinterpreted as “the AI is going to kill everyone and access its own hardware to set its reward to infinity”. This is a misinterpetation because you are referring to control of the “reward input” here, and your later sentences don’t make sense according to this interpretation. However, given the bolded sentence and some lack of attention, plus some confusions over wire heading that are apparently fairly common, I expect a fair number of misinterpretations.
RE 19: Maybe rephrase “kill everyone in the world using nanotech to strike before they know they’re in a battle, and have control of your reward input forever after”? This could, and I predict would, be misinterpreted as “the AI is going to kill everyone and access its own hardware to set its reward to infinity”. This is a misinterpetation because you are referring to control of the “reward input” here, and your later sentences don’t make sense according to this interpretation. However, given the bolded sentence and some lack of attention, plus some confusions over wire heading that are apparently fairly common, I expect a fair number of misinterpretations.