Author: @HoldenKarnofsky

I’ve been working on a new series of posts about the most important century.

• The original series focused on why and how this could be the most important century for humanity. But it had relatively little to say about what we can do today to improve the odds of things going well.

• The new series will get much more specific about the kinds of events that might lie ahead of us, and what actions today look most likely to be helpful.

• A key focus of the new series will be the threat of misaligned AI: AI systems disempowering humans entirely, leading to a future that has little to do with anything humans value. (Like in the Terminator movies, minus the time travel and the part where humans win.)

Many people have trouble taking this “misaligned AI” possibility seriously. They might see the broad point that AI could be dangerous, but they instinctively imagine that the danger comes from ways humans might misuse it. They find the idea of AI itself going to war with humans to be comical and wild. I’m going to try to make this idea feel more serious and real.