Shiroe—my worry is that if we focus only on the ‘low-hanging fruit’ (e.g. AI aligned with individuals, or with all of humanity), we’ll overlook the really dangerous misalignments among human individuals, families, groups, companies, nation-states, religions, etc. that could be exacerbated by access to powerful AI systems.
Also, while it’s true that very few individuals or groups want to torture everyone to death, there are plenty of human groups (eg anti-natalists, eco-extremists, etc) that advocate for human extinction, and that would consider ‘aligned AI’ to be any AI aligned with their pro-extinction mission.
Shiroe—my worry is that if we focus only on the ‘low-hanging fruit’ (e.g. AI aligned with individuals, or with all of humanity), we’ll overlook the really dangerous misalignments among human individuals, families, groups, companies, nation-states, religions, etc. that could be exacerbated by access to powerful AI systems.
Also, while it’s true that very few individuals or groups want to torture everyone to death, there are plenty of human groups (eg anti-natalists, eco-extremists, etc) that advocate for human extinction, and that would consider ‘aligned AI’ to be any AI aligned with their pro-extinction mission.