Something that’s been bugging me is that a “solution to the alignment problem” seems to address accident risk, but not structural risk. A “pivotal act AI” addresses both, but many people are skeptical of this strategy. It’s not obvious to me that pivotal act skeptics should be mostly focused on alignment. To put it crudely (and it being so crude is why this is a comment, not a top level post), to reduce structural risk it seems to me very important to
Understand which actions are actually dangerous
Stop people from taking them
and comparatively less important to find new ways to get AI to do impressive things, safely or otherwise. It might be instrumentally valuable to e.g. get an AI that can help you do the two things above, but this would make AI alignment instrumental, not an end in and of itself. Actually, these remarks seem to apply in equal force to the pivotal act strategy, and the disagreement may just be how much AGI is necessary to achieve these aims.
Something that’s been bugging me is that a “solution to the alignment problem” seems to address accident risk, but not structural risk. A “pivotal act AI” addresses both, but many people are skeptical of this strategy. It’s not obvious to me that pivotal act skeptics should be mostly focused on alignment. To put it crudely (and it being so crude is why this is a comment, not a top level post), to reduce structural risk it seems to me very important to
Understand which actions are actually dangerous
Stop people from taking them
and comparatively less important to find new ways to get AI to do impressive things, safely or otherwise. It might be instrumentally valuable to e.g. get an AI that can help you do the two things above, but this would make AI alignment instrumental, not an end in and of itself. Actually, these remarks seem to apply in equal force to the pivotal act strategy, and the disagreement may just be how much AGI is necessary to achieve these aims.