Uhh, with the additional context from this post in mind, I would argue that your initial interpretation was entirely correct. Like, this is precisely what I’d expected from reading that first quote.
Not the worst-case scenario of Yann LeCun, admittedly, but getting there.
Edit: Actually it’s a bit worse than I’d expected. “Advocating for an international treaty is a call for violence” is just an embarrassing take.
Which is while he didn’t explicitly call for violence, he did argue for very drastic actions, and contra others, treaties are not usually enforced with nuclear war, for obvious reasons.
Eliezer did not call for enforcement via nuclear war. He said that:
Make it explicit in international diplomacy that preventing AI extinction scenarios is considered a priority above preventing a full nuclear exchange, and that allied nuclear countries are willing to run some risk of nuclear exchange if that’s what it takes to reduce the risk of large AI training runs.
and later clarified:
My TIME piece did not suggest nuclear strikes against countries that refuse to sign on to a global agreement against large AI training runs. It said that, if a non-signatory country is building a datacenter that might kill everyone on Earth, you should be willing to preemptively destroy that datacenter; the intended reading is that you should do this even if the non-signatory country is a nuclear power and even if they try to threaten nuclear retaliation for the strike. This is what is meant by “Make it explicit… that allied nuclear countries are willing to run some risk of nuclear exchange if that’s what it takes to reduce the risk of large AI training runs.”
Violently enforcing certain particularly important principles on non-signatories is entirely within the norm, the ban on international-trade-endangering piracy being the prime example. The idea that applying a qualitatively similar standard to AI risk is “deranged” is only valid if you don’t believe that catastrophic AI risk is real: if you don’t believe that a rogue superintelligence somewhere in North Korea can hurt you in the US.
Anyway, that’s not even the crux here. The crux is that there’s a day-and-night difference between:
Arguing that the geopolitical entities, whose monopoly on violence we already accept as foundational to the social contract keeping our civilization together, should add another point to the list of things they enforce.
Arguing for violating the social contract to carry out unilateral violent action.
The difference between those is far beyond the fine points of whether it’s okay or not to enforce an international treaty on nukes-having non-signatories. And the worst falsehoods being spread are those misrepresenting (1) as (2), and Joshua Achiam’s quotes above likewise seem to fail to see the difference between the two (though I don’t think he’s doing that maliciously).
Violently enforcing certain particularly important principles on non-signatories is entirely within the norm
True as stated, though I’m not aware of examples of this being enforced on non-signatories which are nuclear powers. This is just quantitatively riskier, not a notable change in norms.
And I agree this seems clearly non-outrageous if we replace AGI training datacenter with something like “wet lab credibly planning on developing a virus that would kill literally every human if it escaped where this wet lab isn’t planning on taking any serious precautions against a lab leak”.
This is just quantitatively riskier, not a notable escalation.
I think this is actually a fairly extreme escalation compared to how states deal with threats, and whether or not you think this is a good policy, it is a very, very important escalation step, and that this is evidence for it being a very extreme escalation:
True as stated, though I’m not aware of examples of this being enforced on non-signatories which are nuclear powers.
Hm, a disagreement I have is that the norms around escalating to nuclear war are way, way stronger than basically any other norm in international relations, and there’s a reason basically all states do their actions through proxies/covert wars, because the taboo on nuclear war is way stronger than a lot of other norms in the international setting.
I agree that it is norms violating for a country to respond to a conventional strike on their datacenter with a nuclear response. This is different from the statement that the conventional strike from the other country is norms violating.
I don’t think conventional strikes on military assets of nuclear power are that norms violating. In fact, recently, a huge number of missiles were launched at a nuclear power. (Iran launched them at Israel which is widely believed to have nukes.)
(I believe the US has never directly launched a strike on a nuclear power within their territory. However, it has indirectly assisted with such strikes in the Russia Ukraine war and participated in proxy wars.)
The point that would justify an airstrike isn’t violation of a treaty, but posing an immediate and grave risk to the international community. The treaty is only the precondition that makes effective and coordinated action possible.
Uhh, with the additional context from this post in mind, I would argue that your initial interpretation was entirely correct. Like, this is precisely what I’d expected from reading that first quote.
Not the worst-case scenario of Yann LeCun, admittedly, but getting there.
Edit: Actually it’s a bit worse than I’d expected. “Advocating for an international treaty is a call for violence” is just an embarrassing take.
My view on this:
The context is fairly critical here, and I broadly share iceman’s perspective regarding the article’s call for an international treaty:
https://www.lesswrong.com/posts/gZkYvA6suQJthvj4E/#SCnooEfdjJaQHS4TB
Which is while he didn’t explicitly call for violence, he did argue for very drastic actions, and contra others, treaties are not usually enforced with nuclear war, for obvious reasons.
Eliezer did not call for enforcement via nuclear war. He said that:
and later clarified:
Violently enforcing certain particularly important principles on non-signatories is entirely within the norm, the ban on international-trade-endangering piracy being the prime example. The idea that applying a qualitatively similar standard to AI risk is “deranged” is only valid if you don’t believe that catastrophic AI risk is real: if you don’t believe that a rogue superintelligence somewhere in North Korea can hurt you in the US.
Anyway, that’s not even the crux here. The crux is that there’s a day-and-night difference between:
Arguing that the geopolitical entities, whose monopoly on violence we already accept as foundational to the social contract keeping our civilization together, should add another point to the list of things they enforce.
Arguing for violating the social contract to carry out unilateral violent action.
The difference between those is far beyond the fine points of whether it’s okay or not to enforce an international treaty on nukes-having non-signatories. And the worst falsehoods being spread are those misrepresenting (1) as (2), and Joshua Achiam’s quotes above likewise seem to fail to see the difference between the two (though I don’t think he’s doing that maliciously).
True as stated, though I’m not aware of examples of this being enforced on non-signatories which are nuclear powers. This is just quantitatively riskier, not a notable change in norms.
And I agree this seems clearly non-outrageous if we replace AGI training datacenter with something like “wet lab credibly planning on developing a virus that would kill literally every human if it escaped where this wet lab isn’t planning on taking any serious precautions against a lab leak”.
I think this is a disagreement I have:
I think this is actually a fairly extreme escalation compared to how states deal with threats, and whether or not you think this is a good policy, it is a very, very important escalation step, and that this is evidence for it being a very extreme escalation:
Sorry, I actually meant “not a notable change in norms”. I agree that it is quantiatively much costlier from the perspective of the US.
Hm, a disagreement I have is that the norms around escalating to nuclear war are way, way stronger than basically any other norm in international relations, and there’s a reason basically all states do their actions through proxies/covert wars, because the taboo on nuclear war is way stronger than a lot of other norms in the international setting.
I agree that it is norms violating for a country to respond to a conventional strike on their datacenter with a nuclear response. This is different from the statement that the conventional strike from the other country is norms violating.
I don’t think conventional strikes on military assets of nuclear power are that norms violating. In fact, recently, a huge number of missiles were launched at a nuclear power. (Iran launched them at Israel which is widely believed to have nukes.)
(I believe the US has never directly launched a strike on a nuclear power within their territory. However, it has indirectly assisted with such strikes in the Russia Ukraine war and participated in proxy wars.)
Yes, I was solely referring to nuclear strikes.
The point that would justify an airstrike isn’t violation of a treaty, but posing an immediate and grave risk to the international community. The treaty is only the precondition that makes effective and coordinated action possible.