Scott Alexander has an interesting little short on human manipulation: https://slatestarcodex.com/2018/10/30/sort-by-controversial/ So far everything I’m seeing, both fiction and anecdotes, is consistent with the notion that humans are relatively easy to model and emotionally exploit. I also agree with CBiddulph’s analysis, insofar as while the paperclip/stamp failure mode requires the AI to have planning, generation of manipulative text doesn’t need to have a goal—if you generate text that is maximally controversial (or maximises some related metric) and disseminate the text, that by itself may already do damage.
I like it—interesting how much is to do with the specific vulnerabilities of humans, and how humans exploiting other humans’ vulnerabilities was what enabled and exacerbated the situation.
Scott Alexander has an interesting little short on human manipulation: https://slatestarcodex.com/2018/10/30/sort-by-controversial/
So far everything I’m seeing, both fiction and anecdotes, is consistent with the notion that humans are relatively easy to model and emotionally exploit. I also agree with CBiddulph’s analysis, insofar as while the paperclip/stamp failure mode requires the AI to have planning, generation of manipulative text doesn’t need to have a goal—if you generate text that is maximally controversial (or maximises some related metric) and disseminate the text, that by itself may already do damage.
I like it—interesting how much is to do with the specific vulnerabilities of humans, and how humans exploiting other humans’ vulnerabilities was what enabled and exacerbated the situation.
Whilst we’re sharing stories...I’ll shamelessly promote one of my (very) short stories on human manipulation by AI. In this case the AI is being deliberative at least in achieving its instrumental goals. https://docs.google.com/document/d/1Z1laGUEci9rf_aaDjQKS_IIOAn6D0VtAOZMSqZQlqVM/edit
There’s also a romantic theme ;-)