Also, I don’t feel that this article adequately addressed the downside of SA that it accelerates an arms race. SA is only favored when alignment is easy with high probability and you’re confident that you will win the arms race, and you’re confident that it’s better for you to win than for the other guy[1], and you’re talking about a specific kind of alignment where an “aligned” AI doesn’t necessarily behave ethically, it just does what its creator intends.
[1] How likely is a US-controlled (or, more accurately, Sam Altman/Dario Amodei/Mark Zuckerberg-controlled) AGI to usher in a global utopia? How likely is a China-controlled AGI to do the same? I think people are too quick to take it for granted that the former probability is larger than the latter.
We do discuss this in the article and tried to convey that it is a very significant downside of SA. All 3 plans have enormous downsides though, so a plan posing massive risks is not disqualifying. The key is understanding when these risks might be worth taking given the alternatives.
CD might be too weak if TAI is offense-dominant, regardless of regulations or cooperative partnerships, and result in misuse or misalignment catastrophe
If GM fails it might blow any chance of producing protective TAI and hand over the lead to the most reckless actors.
SA might directly provoke a world war or produce unaligned AGI ahead of schedule.
SA is favored when alignment is easy or moderately difficult (e.g. at the level where interpretability probes, scalable oversight etc. help) with high probability, and you expect to win the arms race. But it doesn’t require you to be the ‘best’. The key isn’t whether US control is better than Chinese control, but whether centralized development under any actor is preferable to widespread proliferation of TAI capabilities to potentially malicious actors
Regarding whether the US (remember on SA there’s assumed to be extensive government oversight) is better than the CCP: I think the answer is yes and I talk a bit more about why here. I don’t consider US AI control being better than Chinese AI control to be the most important argument in favor of SA, however. That fact alone doesn’t remotely justify SA: you also need easy/moderate alignment and you need good evidence than an arms race is likely unavoidable regardless of what we recommend.
Also, I don’t feel that this article adequately addressed the downside of SA that it accelerates an arms race. SA is only favored when alignment is easy with high probability and you’re confident that you will win the arms race, and you’re confident that it’s better for you to win than for the other guy[1], and you’re talking about a specific kind of alignment where an “aligned” AI doesn’t necessarily behave ethically, it just does what its creator intends.
[1] How likely is a US-controlled (or, more accurately, Sam Altman/Dario Amodei/Mark Zuckerberg-controlled) AGI to usher in a global utopia? How likely is a China-controlled AGI to do the same? I think people are too quick to take it for granted that the former probability is larger than the latter.
We do discuss this in the article and tried to convey that it is a very significant downside of SA. All 3 plans have enormous downsides though, so a plan posing massive risks is not disqualifying. The key is understanding when these risks might be worth taking given the alternatives.
CD might be too weak if TAI is offense-dominant, regardless of regulations or cooperative partnerships, and result in misuse or misalignment catastrophe
If GM fails it might blow any chance of producing protective TAI and hand over the lead to the most reckless actors.
SA might directly provoke a world war or produce unaligned AGI ahead of schedule.
SA is favored when alignment is easy or moderately difficult (e.g. at the level where interpretability probes, scalable oversight etc. help) with high probability, and you expect to win the arms race. But it doesn’t require you to be the ‘best’. The key isn’t whether US control is better than Chinese control, but whether centralized development under any actor is preferable to widespread proliferation of TAI capabilities to potentially malicious actors
Regarding whether the US (remember on SA there’s assumed to be extensive government oversight) is better than the CCP: I think the answer is yes and I talk a bit more about why here. I don’t consider US AI control being better than Chinese AI control to be the most important argument in favor of SA, however. That fact alone doesn’t remotely justify SA: you also need easy/moderate alignment and you need good evidence than an arms race is likely unavoidable regardless of what we recommend.