I mostly agree with points 1 and 2. Many interacting AI’s are important to consider, and I think individual AI’s will have an incentive to multiply. And I agree that people will continue to push the capability frontier beyond what is directly useful, potentially giving AI dangerous abilities.
I think we differ on the timeline of those changes, not whether those changes are possible or important. This is the question I’m trying to highlight.
Longer timelines for dangerous AI don’t mean that we shouldn’t prepare for things going wrong; these problems will still need to be solved eventually. Ideally, some researchers would act as if timelines are really short and “roll the dice” on alignment research that could pan out in a few years.
But I’m arguing that the bulk of the field should feel safe investing in infrastructure, movement building, and projects over a 10-20 year timeframe. People considering pursuing AI safety should assume they will have enough time to make an impact. Finding ways to extend this horizon further into the future is valuable because it gives the field more time to grow.
I mostly agree with points 1 and 2. Many interacting AI’s are important to consider, and I think individual AI’s will have an incentive to multiply. And I agree that people will continue to push the capability frontier beyond what is directly useful, potentially giving AI dangerous abilities.
I think we differ on the timeline of those changes, not whether those changes are possible or important. This is the question I’m trying to highlight.
Longer timelines for dangerous AI don’t mean that we shouldn’t prepare for things going wrong; these problems will still need to be solved eventually. Ideally, some researchers would act as if timelines are really short and “roll the dice” on alignment research that could pan out in a few years.
But I’m arguing that the bulk of the field should feel safe investing in infrastructure, movement building, and projects over a 10-20 year timeframe. People considering pursuing AI safety should assume they will have enough time to make an impact. Finding ways to extend this horizon further into the future is valuable because it gives the field more time to grow.