Is anyone else terrified by the notion of a super-AI that looked at human history and figured out what morality humans actually subscribed to, rather than claimed to, and then instantiated it?
I am.
I was worried that I’d get into an argument with Eliezer when I started reading this post, but at the end I didn’t have any nits to pick.
I don’t think there’s any evidence that he things an AI converting to Christianity is worse than paper-clipping the universe. He thinks it’s bad, but never compared the two possibilities.
Is anyone else terrified by the notion of a super-AI that looked at human history and figured out what morality humans actually subscribed to, rather than claimed to, and then instantiated it? I am.
I was worried that I’d get into an argument with Eliezer when I started reading this post, but at the end I didn’t have any nits to pick.
I don’t think there’s any evidence that he things an AI converting to Christianity is worse than paper-clipping the universe. He thinks it’s bad, but never compared the two possibilities.