Further followup (I think I do disagree here with the implication of how easy it is to come away with the impression if you’re reading the post un-primed – it looks like probably some LessWrongers here came away with this impression and probably read it pretty quickly on their own. But, I think it’s useful to have this spelled out)
Sometimes—even when you say something whose intended reading is immediately obvious to any reader who hasn’t seen it before—it’s possible to tell people to see something in writing that isn’t there, and then they see it.
My TIME piece did not suggest nuclear strikes against countries that refuse to sign on to a global agreement against large AI training runs. It said that, if a non-signatory country is building a datacenter that might kill everyone on Earth, you should be willing to preemptively destroy that datacenter; the intended reading is that you should do this *even if* the non-signatory country is a nuclear power and *even if* they try to threaten nuclear retaliation for the strike. This is what is meant by “Make it explicit… that allied nuclear countries are willing to run some risk of nuclear exchange if that’s what it takes to reduce the risk of large AI training runs.”
I’d hope that would be clear from any plain reading, if you haven’t previously been lied-to about what it says. It does not say, “Be willing to *use* nuclear weapons” to reduce the risk of training runs. It says, “Be willing to run some risk of nuclear exchange” [initiated by the other country] to reduce the risk of training runs.
The taboo against first use of nuclear weapons continues to make sense to me. I don’t see why we’d need to throw that away in the course of adding “first use of GPU farms” to the forbidden list.
I further note: Among the reasons to spell this all out, is that it’s important to be explicit, in advance, about things that will cause your own country / allied countries to use military force. Lack of clarity about this is how World War I *and* World War II both started.
If (say) the UK, USA, and China come to believe that large GPU runs run some risk of utterly annihilating their own populations and all of humanity, they would not deem it in their own interests to allow Russia to proceed with building a large GPU farm *even if* it were a true and certain fact that Russia would retaliate with nuclear weapons to the destruction of that GPU farm. In this case—unless I’m really missing something about how this game is and ought to be played—you really want all the Allied countries to make it very clear, well in advance, that this is what they believe and this is how they will act. This would be true even in a world where it was, in reality, factually false that the large GPU farm ran a risk of destroying humanity. It would still be extremely important that the Allies be very explicit about what they believed and how they’d act as a result. You would not want Russia believing that the Allies would back down from destroying the GPU farm given a credible commitment by Russia to nuke in reply to any conventional attack, and the Allies in fact believing that the danger to humanity meant they had to airstrike the GPU farm anyways.
So if I’d meant “Be willing to employ first use of nuclear weapons against a country for refusing to sign the agreement,” or even “Use nukes to destroy rogue datacenters, instead of conventional weapons, for some unimaginable reason,” I’d have said that, in words, very clearly, because you do not want to be vague about that sort of thing.
It is not what I meant, and there’d be no reason to say it, and the TIME piece plainly does not say it; and if somebody else told you I said that, update how much you trust them about anything else either.
And goes on to say:
So long as I’m clarifying things: I do not dispute those critics who have noted that most international agreements, eg nuclear non-proliferation, bind only their signatories. I agree that an alliance which declares its intent to strike a non-signatory country for dangerous behavior is extraordinary; though precedents would include Israel’s airstrike on Iraq’s unfinished Osirak reactor in 1981 (without which Iraq might well have possessed nuclear weapons at the time it invaded Kuwait—the later US misbehavior around Iraq does not change this earlier historical point).
My TIME piece does not say, “Hey, this problem ought to be solvable by totally conventional normal means, let’s go use conventional treaties and diplomacy to solve it.” It says, “If anyone anywhere builds a sufficiently powerful AI, under anything remotely like present conditions, everyone will die. Here is what we’d have to do to prevent that.” And no, I do not expect that policy proposal to be adopted, in real life, now that we’ve come to this. I spent the last twenty years trying to have there be options that were Not This, not because I dislike this ultimate last resort… though it *is* horrible… but because I don’t expect we actually have that resort. This is not what I expect to happen, now that we’ve been reduced to this last resort. I expect that we all die. That is why I tried so hard to have things not end up here.
But if one day a lot of people woke up and decided that they didn’t want to die, it seems to me that this is something extraordinary that a coalition of nuclear countries could decide to do, and maybe we wouldn’t die.
If *all* the countries on Earth had to voluntarily sign on, it would not be an imaginable or viable plan even then; there’s extraordinary, and then there’s impossible. Which is why I tried to spell out that, if the allied countries were willing to behave in the extraordinary way of “be willing to airstrike a GPU farm built by a non-signatory country” and “be willing to run a risk of nuclear retaliation from a nuclear non-signatory country”, maybe those allied countries could decide to just-not-die *even if* Russia refused to be part of the coalition.
Further followup (I think I do disagree here with the implication of how easy it is to come away with the impression if you’re reading the post un-primed – it looks like probably some LessWrongers here came away with this impression and probably read it pretty quickly on their own. But, I think it’s useful to have this spelled out)
And goes on to say: