I want to push back a bit on the framing used here. Instead of the framing “slowing down AI,” another framing we could use is, “lay the groundwork for slowing down in the future, when extra time is most needed.” I prefer this latter framing/emphasis because:
An extra year in which the AI safety field has access to pretty advanced AI capabilities seems much more valuable for the field’s progress (say, maybe 10x) than an extra year with current AI capabilities, since the former type of year would give the field much better opportunities to test safety ideas and more clarity about what types of AI systems are relevant.
One counterargument is that AI safety will likely be bottlenecked by serial time, because discarding bad theories and formulating better ones takes serial time, making extra years early on very useful. But my very spotty understanding of the history of science suggests that it doesn’t just take time for bad theories to get replaced by better ones—it takes time along with the accumulation of lots of empirical evidence. This supports the view that late-stage time is much more valuable than early-stage time.
Slowing down in the future seems much more tractable than slowing down now, since many critical actors seem much more likely to support slowing down if and when there are clear, salient demonstrations of its importance (i.e. warning shots).
Given that slowing down later is much more valuable and much more tractable than just slowing down now, it seems much better to focus on slowing down later. But the broader framing of “slow down” doesn’t really suggest that focus, and maybe it even discourages it.
I don’t know who you are beyond having read a few of your LW posts, I know Mauricio a little and don’t have much reason to think of him as an ‘expert’ (I believe he’s worked in policy spaces for ~1 year), so I’m not taking your comment as much evidence about his claims.
Edit: I wish the above comment had just been retracted (i.e. had a line put through it) rather than removed, it makes it unclear to future readers what happened. If I recall correctly the comment said that while he had some disagreements with Mauricio, he can personally vouch for him as an expert in this area and well-positioned to have a good opinion on the subject.
Thanks for writing!
I want to push back a bit on the framing used here. Instead of the framing “slowing down AI,” another framing we could use is, “lay the groundwork for slowing down in the future, when extra time is most needed.” I prefer this latter framing/emphasis because:
An extra year in which the AI safety field has access to pretty advanced AI capabilities seems much more valuable for the field’s progress (say, maybe 10x) than an extra year with current AI capabilities, since the former type of year would give the field much better opportunities to test safety ideas and more clarity about what types of AI systems are relevant.
One counterargument is that AI safety will likely be bottlenecked by serial time, because discarding bad theories and formulating better ones takes serial time, making extra years early on very useful. But my very spotty understanding of the history of science suggests that it doesn’t just take time for bad theories to get replaced by better ones—it takes time along with the accumulation of lots of empirical evidence. This supports the view that late-stage time is much more valuable than early-stage time.
Slowing down in the future seems much more tractable than slowing down now, since many critical actors seem much more likely to support slowing down if and when there are clear, salient demonstrations of its importance (i.e. warning shots).
Given that slowing down later is much more valuable and much more tractable than just slowing down now, it seems much better to focus on slowing down later. But the broader framing of “slow down” doesn’t really suggest that focus, and maybe it even discourages it.
I don’t know who you are beyond having read a few of your LW posts, I know Mauricio a little and don’t have much reason to think of him as an ‘expert’ (I believe he’s worked in policy spaces for ~1 year), so I’m not taking your comment as much evidence about his claims.
Edit: I wish the above comment had just been retracted (i.e.
had a line put through it) rather than removed, it makes it unclear to future readers what happened. If I recall correctly the comment said that while he had some disagreements with Mauricio, he can personally vouch for him as an expert in this area and well-positioned to have a good opinion on the subject.