I say it is a rebrand of the “AI (x-)safety” community. When AI alignment came along we were calling it AI safety, even though it was really basically AI existential safety all along that everyone in the community meant. “AI safety” was (IMO) a somewhat successful bid for more mainstream acceptance, that then lead to dillution and confusion, necessitating a new term.
I don’t think the history is that important; what’s important is having good terminology going forward. This is also why I stress that I work on AI existential safety.
So I think people should just say what kind of technical work they are doing and “existential safety” should be considered as a social-technical problem that motivates a community of researchers, and used to refer to that problem and that community. In particular, I think we are not able to cleanly delineate what is or isn’t technical AI existential safety research at this point, and we should welcome intellectual debates about the nature of the problem and how different technical research may or may not contribute to increasing x-safety.
I say it is a rebrand of the “AI (x-)safety” community.
When AI alignment came along we were calling it AI safety, even though it was really basically AI existential safety all along that everyone in the community meant. “AI safety” was (IMO) a somewhat successful bid for more mainstream acceptance, that then lead to dillution and confusion, necessitating a new term.
I don’t think the history is that important; what’s important is having good terminology going forward.
This is also why I stress that I work on AI existential safety.
So I think people should just say what kind of technical work they are doing and “existential safety” should be considered as a social-technical problem that motivates a community of researchers, and used to refer to that problem and that community. In particular, I think we are not able to cleanly delineate what is or isn’t technical AI existential safety research at this point, and we should welcome intellectual debates about the nature of the problem and how different technical research may or may not contribute to increasing x-safety.