I both agree that the race dynamic is concerning (and would like to see Anthropic address them explicitly), and also think that Anthropic should get a fair bit of credit for not releasing Claude before ChatGPT, a thing they could have done and probably gained a lot of investment / hype over. I think Anthropic’s “let’s not contribute to AI hype” is good in the same way that OpenAI’s “let’s generate massive” hype strategy is bad.
Like definitely I’m worried about the incentive to stay competitive, especially in the product space. But I think it’s worth highlighting that Anthropic (and Deepmind and Google AI fwiw) have not rushed to product when they could have. There’s still the relevant question “is building SOTA systems net positive given this strategy”, and it’s not clear to me what the answer is, but I want to acknowledge that “building SOTA systems and generating hype / rushing to market” is the default for startups and “build SOTA systems and resist the juicy incentive” is what Anthropic has done so far & that’s significant.
To be clear, I think Anthropic has done a pretty admirable job of showing some restraint here. It is objectively quite impressive. My wariness is “Man, I think the task here is really hard and even a very admirably executed company may not be sufficient.”
I both agree that the race dynamic is concerning (and would like to see Anthropic address them explicitly), and also think that Anthropic should get a fair bit of credit for not releasing Claude before ChatGPT, a thing they could have done and probably gained a lot of investment / hype over.
I mean, didn’t the capabilities of Claude leak specifically to OpenAI employees, so that it’s pretty unclear that not releasing actually had much of an effect on preventing racing? My current best guess, though I am only like 30% of this hypothesis since there are many possible hypotheses here, is that Chat-GPT was developed in substantial parts because someone saw or heard about a demo of Claude and thought it was super impressive.
Yeah I think it can both be true that OpenAI felt more pressure to release products faster due to perceived competition risk from Anthropic, and also that Anthropic showed restraint in not trying to race them to get public demos or a product out. In terms of speeding up AI development, not building anything > building something and keeping it completely secret > building something that your competitors learn about > building something and generating public hype about it via demos > building something with hype and publicly releasing it to users & customers. I just want to make sure people are tracking the differences.
so that it’s pretty unclear that not releasing actually had much of an effect on preventing racing
It seems like if OpenAI didn’t publicly release ChatGPT then that huge hype wave wouldn’t have happened, at least for a while, since Anthropic sitting on Claude rather than release. I think it’s legit to question whether any group scaling SOTA models is net positive but I want to be clear about credit assignment, and the ChatGPT release was an action taken by OpenAI.
In terms of speeding up AI development, not building anything > building something and keeping it completely secret > building something that your competitors learn about > building something and generating public hype about it via demos > building something with hype and publicly releasing it to users & customers.
I think it is very helpful, and healthy for the discourse, to make this distinction. I agree that many of these things might get lumped together.
But also, I want to flag the possibility that something can be very very bad to do, even if there are there other things that would have been progressively worse to do.
I want to make sure that groups get the credit that is due to them when they do good things against their incentives.
I also want to avoided falling into a pattern of thinking “well they didn’t do the worst thing, or the second worst thing, so that’s pretty good!” if in isolation I would have thought that action was pretty bad / blameworthy.
As of this moment, I don’t have a particular opinion one way or the other about how good or bad Anthropic’s release policy is. I’m merely making the abstract point at this time.
My guess is that training cutting edge models, and not releasing them is a pretty good play, or would have been, if there wasn’t huge AGI hype.
As it is, information about your models is going to leak, and in most cases the fact that something is possible is most of the secret to reverse engineering it (note: this might be true in the regime of transformer models, but it might not be true for other tasks or sub-problems).
But on the other hand, given the hype, people are going to try to do the things that you’re doing anyway, so maybe leaks about your capabilities don’t make that much difference?
This does point out an important consideration, which is “how much information needs to leak from your lab to enable someone else to replicate your results?”
It seems like, in many cases, there’s an obvious way to do some task, and the mere fact that you succeeded is enough info to recreate your result. But presumably there are cases, where you figure out a clever trick, and even if the evidence of your model’s performance leaks, that doesn’t tell the world how to do it (though it does cause maybe hundreds of smart people to start looking for how you did it, trying to discover how to do it themselves).
I think I should regard the situation differently depending on the status of that axis.
I both agree that the race dynamic is concerning (and would like to see Anthropic address them explicitly), and also think that Anthropic should get a fair bit of credit for not releasing Claude before ChatGPT, a thing they could have done and probably gained a lot of investment / hype over. I think Anthropic’s “let’s not contribute to AI hype” is good in the same way that OpenAI’s “let’s generate massive” hype strategy is bad.
Like definitely I’m worried about the incentive to stay competitive, especially in the product space. But I think it’s worth highlighting that Anthropic (and Deepmind and Google AI fwiw) have not rushed to product when they could have. There’s still the relevant question “is building SOTA systems net positive given this strategy”, and it’s not clear to me what the answer is, but I want to acknowledge that “building SOTA systems and generating hype / rushing to market” is the default for startups and “build SOTA systems and resist the juicy incentive” is what Anthropic has done so far & that’s significant.
Yeah I agree with this.
To be clear, I think Anthropic has done a pretty admirable job of showing some restraint here. It is objectively quite impressive. My wariness is “Man, I think the task here is really hard and even a very admirably executed company may not be sufficient.”
Yeah I think we should all be scared of the incentives here.
I mean, didn’t the capabilities of Claude leak specifically to OpenAI employees, so that it’s pretty unclear that not releasing actually had much of an effect on preventing racing? My current best guess, though I am only like 30% of this hypothesis since there are many possible hypotheses here, is that Chat-GPT was developed in substantial parts because someone saw or heard about a demo of Claude and thought it was super impressive.
Yeah I think it can both be true that OpenAI felt more pressure to release products faster due to perceived competition risk from Anthropic, and also that Anthropic showed restraint in not trying to race them to get public demos or a product out. In terms of speeding up AI development, not building anything > building something and keeping it completely secret > building something that your competitors learn about > building something and generating public hype about it via demos > building something with hype and publicly releasing it to users & customers. I just want to make sure people are tracking the differences.
It seems like if OpenAI didn’t publicly release ChatGPT then that huge hype wave wouldn’t have happened, at least for a while, since Anthropic sitting on Claude rather than release. I think it’s legit to question whether any group scaling SOTA models is net positive but I want to be clear about credit assignment, and the ChatGPT release was an action taken by OpenAI.
I think it is very helpful, and healthy for the discourse, to make this distinction. I agree that many of these things might get lumped together.
But also, I want to flag the possibility that something can be very very bad to do, even if there are there other things that would have been progressively worse to do.
I want to make sure that groups get the credit that is due to them when they do good things against their incentives.
I also want to avoided falling into a pattern of thinking “well they didn’t do the worst thing, or the second worst thing, so that’s pretty good!” if in isolation I would have thought that action was pretty bad / blameworthy.
As of this moment, I don’t have a particular opinion one way or the other about how good or bad Anthropic’s release policy is. I’m merely making the abstract point at this time.
Yeah, I agree with all of this, seems worth saying. Now to figure out the object level… 🤔
That’s the hard part.
My guess is that training cutting edge models, and not releasing them is a pretty good play, or would have been, if there wasn’t huge AGI hype.
As it is, information about your models is going to leak, and in most cases the fact that something is possible is most of the secret to reverse engineering it (note: this might be true in the regime of transformer models, but it might not be true for other tasks or sub-problems).
But on the other hand, given the hype, people are going to try to do the things that you’re doing anyway, so maybe leaks about your capabilities don’t make that much difference?
This does point out an important consideration, which is “how much information needs to leak from your lab to enable someone else to replicate your results?”
It seems like, in many cases, there’s an obvious way to do some task, and the mere fact that you succeeded is enough info to recreate your result. But presumably there are cases, where you figure out a clever trick, and even if the evidence of your model’s performance leaks, that doesn’t tell the world how to do it (though it does cause maybe hundreds of smart people to start looking for how you did it, trying to discover how to do it themselves).
I think I should regard the situation differently depending on the status of that axis.