Yes. That is one of the things in possibility space. I don’t think unaligned means safe. We work with unaligned people all the time, and some of them aren’t safe either.
The main thing I was hoping people would understand from this is that an unaligned AI is near a 100% possibility. Alignment isn’t a one and done goal that so many people act like it is. Even if you successfully align an AI, all it takes is one failure to align and the genie is out of the bottle. One single point of failure and it becomes a cascading failure.
So let’s imagine an ASI that works on improving itself. How does it ensure the alignment of an intelligence greater than itself.
With hundreds, maybe thousands of people working to create AI, someone will fail to align.
The future is unaligned.
Are we taking that seriously? Working on alignment is great, but it is not the future we should be prepping for. Do you have a plan? I don’t yet, but I’m thinking about the world where there are intelligences greater than me abound (already true) and we don’t share the same interests (also already true).
Working on alignment is great, but it is not the future we should be prepping for. Do you have a plan?
I do not, because a future where an unaligned superintelligence takes over is precisely as survivable as a future in which the sun spontaneously implodes.
Any apocalypse that you can plan for isn’t really an apocalypse.
Yes. That is one of the things in possibility space. I don’t think unaligned means safe. We work with unaligned people all the time, and some of them aren’t safe either.
The main thing I was hoping people would understand from this is that an unaligned AI is near a 100% possibility. Alignment isn’t a one and done goal that so many people act like it is. Even if you successfully align an AI, all it takes is one failure to align and the genie is out of the bottle. One single point of failure and it becomes a cascading failure.
So let’s imagine an ASI that works on improving itself. How does it ensure the alignment of an intelligence greater than itself.
With hundreds, maybe thousands of people working to create AI, someone will fail to align.
The future is unaligned.
Are we taking that seriously? Working on alignment is great, but it is not the future we should be prepping for. Do you have a plan? I don’t yet, but I’m thinking about the world where there are intelligences greater than me abound (already true) and we don’t share the same interests (also already true).
I do not, because a future where an unaligned superintelligence takes over is precisely as survivable as a future in which the sun spontaneously implodes.
Any apocalypse that you can plan for isn’t really an apocalypse.