I’m not sure about how costly these sorts of proposals are (e.g. because it makes customers think you’re crazy). Possibly, labs could coordinate to release things like this simultaneously to avoid tragedy of the commons (there might be anti-trust issues with this).
Yep, buy-in from the majority of frontier labs seems pretty important here. If OpenAI went out and said ‘We think that there’s a 10% chance that AGI we develop kills over 1 billion people’, but Meta kept their current stance (along the lines of ‘we think that the AI x-risk discussion is fearmongering and the systems we’re building will be broadly beneficial for humanity’) then I’d guess that OpenAI would lose a ton of business. From the point of view of an enterprise using OpenAI’s products, it can’t help your public image to be using the products of a lab that thinks it has a 10% chance of ending the world—especially if there are other labs offering similar products that don’t carry this burden. In a worst-case scenario, I can imagine that this puts OpenAI directly in the firing line of regulators, whilst Meta gets off far more lightly.
I’m not sure this effect is as strong as one might think. For one, Dario Amodei (CEO of Anthropic) claimed his P(doom) was around 25% (specifically, “the chance that his technology could end human civilisation”). I remember Sam Altman saying something similar, but can’t find an exact figure right now. Meanwhile, Yann LeCun (Chief AI Scientist at Meta) maintains approximately the stance you describe. None of this, as far as I’m aware, has led to significant losses for OpenAI or Anthropic.
Is it really the case that making these claims at an institutional level, on a little corner of one’s website, is so much stronger than the CEO of one’s company espousing these views very publicly in interviews? Intuitively, this seems like it wouldn’t make a massive difference.
In a worst-case scenario, I can imagine that this puts OpenAI directly in the firing line of regulators, whilst Meta gets off far more lightly.
I’m interested to know if there’s any precedent for this, ie. a company being regulated further because they claimed their industry needed it, while those restrictions weren’t applied universally.
Yep, buy-in from the majority of frontier labs seems pretty important here. If OpenAI went out and said ‘We think that there’s a 10% chance that AGI we develop kills over 1 billion people’, but Meta kept their current stance (along the lines of ‘we think that the AI x-risk discussion is fearmongering and the systems we’re building will be broadly beneficial for humanity’) then I’d guess that OpenAI would lose a ton of business. From the point of view of an enterprise using OpenAI’s products, it can’t help your public image to be using the products of a lab that thinks it has a 10% chance of ending the world—especially if there are other labs offering similar products that don’t carry this burden. In a worst-case scenario, I can imagine that this puts OpenAI directly in the firing line of regulators, whilst Meta gets off far more lightly.
I’m not sure this effect is as strong as one might think. For one, Dario Amodei (CEO of Anthropic) claimed his P(doom) was around 25% (specifically, “the chance that his technology could end human civilisation”). I remember Sam Altman saying something similar, but can’t find an exact figure right now. Meanwhile, Yann LeCun (Chief AI Scientist at Meta) maintains approximately the stance you describe. None of this, as far as I’m aware, has led to significant losses for OpenAI or Anthropic.
Is it really the case that making these claims at an institutional level, on a little corner of one’s website, is so much stronger than the CEO of one’s company espousing these views very publicly in interviews? Intuitively, this seems like it wouldn’t make a massive difference.
I’m interested to know if there’s any precedent for this, ie. a company being regulated further because they claimed their industry needed it, while those restrictions weren’t applied universally.