Sharing lists is awesome. Reading lists often causes me to learn things or add framings to my toolbox; indeed, I got both of those benefits here. Strong-upvoted.
In some places, there’s quite low-hanging fruit for saying a little more:
I worry that it’s easy to read the list below as saying that this narrow slice, all clustered in one portion of the neighborhood, is a very big slice of the space of possible ways an AGI group may have to burn down its lead.
Finally, note that this is only intended as a brainstorm of things that might force a leading team to burn a large number of months; it is not intended to be an exhaustive list of reasons that alignment is hard. (That would include various other factors such as “what sorts of easy temptations will be available that the team has to avoid?” and “how hard is it to find a viable deployment strategy?” and so on.)
Spending a paragraph outlining other big slices of the space (without claiming those are exhaustive either), and spending a paragraph expanding on the “various other factors,” would be very useful to me. (Or linking to a relevant source, but I’m not aware of any analysis of these questions.)
Sharing lists is awesome. Reading lists often causes me to learn things or add framings to my toolbox; indeed, I got both of those benefits here. Strong-upvoted.
In some places, there’s quite low-hanging fruit for saying a little more:
Spending a paragraph outlining other big slices of the space (without claiming those are exhaustive either), and spending a paragraph expanding on the “various other factors,” would be very useful to me. (Or linking to a relevant source, but I’m not aware of any analysis of these questions.)