That’s the alignment problem, the primary topic of this site. Opinions vary and arguments are plentiful. The general consensus is that there’s tons of reasons it might wipe us, the informed average something like a 50‰ estimate, and overconfidence is usually from ignorance of those arguments. I won’t try to restate them all, and I don’t know of a place they’re all collected, but they’re all over this site.
That’s the alignment problem, the primary topic of this site. Opinions vary and arguments are plentiful. The general consensus is that there’s tons of reasons it might wipe us, the informed average something like a 50‰ estimate, and overconfidence is usually from ignorance of those arguments. I won’t try to restate them all, and I don’t know of a place they’re all collected, but they’re all over this site.