The first Dario quote sounds squarely in line with releasing a Claude 3 on par with GPT-4 but well afterwards. The second Dario quote has a more ambiguous connotation, but if read explicitly it strikes me as compatible with the Claude 3 release.
If you spent a while looking for the most damning quotes, then these quotes strike me as evidence the community was just wishfully thinking while in reality Anthropic comms were fairly clear throughout. Privately pitching aggressive things to divert money from more dangerous orgs while minimizing head-on competition with OpenAI seems best to me (though obviously it’s also evidence that they’ll actually do the aggressive scaling things, so hard to know).
To make concrete the disagreement, I’d be interested in people predicting on “If Anthropic releases a GPT-5 equivalent X months behind, then their dollars/compute raised will be Y times lower than OpenAI” for various values of X.
“Diverting money” strikes me as the wrong frame here. Partly because I doubt this actually was the consequence—i.e., I doubt OpenAI etc. had a meaningfully harder time raising capital because of Anthropic’s raise—but also because it leaves out the part where this purported desirable consequence was achieved via (what seems to me like) straightforward deception!
If indeed Dario told investors he hoped to obtain an insurmountable lead soon, while telling Dustin and others that he was committed to avoid gaining any meaningful lead, then it sure seems like one of those claims was a lie. And by my ethical lights, this seems like a horribly unethical thing to lie about, regardless of whether it somehow caused OpenAI to have less money.
I don’t see any direct contradiction/lie there, at least between my version and the investor paraphrase. You don’t necessarily have to release to public general access the best model, in order to be so far ahead that competitors can’t catch up.
For example, LLMs at the research frontier could be a natural (Bertrand?) oligopoly where there’s a stable two-player oligopoly for the best models (#1 by XYZ, and #2 by Anthropic), and everyone else gives up: there is no point in spending $10b to stand up your own model to try to become #1 when XYZ/Anthropic will just cut prices or release the next iteration that they’d been holding back and you get relegated to #3 and there’s no reason for anyone to buy yours instead, and you go bankrupt. (This would be similar to other historical examples like Intel/AMD or Illumina: they enjoyed large margins and competing with them was possible, but was very dangerous because they had a lot of pent-up improvements they could unleash if you spent enough to become a threat. Or in the case of the highly stable iOS/Android mobile duopoly, just being too incredibly capital-intensive to replicate and already low-margin because the creators make their money elsewhere like devices/ads, having commoditized their complement.)
And then presumably at some point you either solve safety or the models are so capable that further improvement is unnecessary or you can’t increase capability; then the need for the AI-arms-race policy is over, and you just do whatever makes pragmatic sense in that brave new world.
It seems plausible that this scenario could happen, i.e., that Anthropic and OpenAI end up in a stable two-player oligopoly. But I would still be pretty surprised if Anthropic’s pitch to investors, when asking for billions of dollars in funding, is that they pre-commit to never release a substantially better product than their main competitor.
How surprising would you say you find the idea of a startup trying to, and successfully raising, not billions but tens of billions of dollars by pitching investors they’re asking that their investment could be canceled at any time at the wave of a hand, the startup pre-commits that the investments will be canceled in the best-case scenario of the product succeeding, & that the investors ought to consider their investment “in the spirit of a donation”?
LLMs at the research frontier could be a natural oligopoly where there’s a stable two-player oligopoly for the best models (#1 by XYZ, and #2 by Anthropic), and everyone else gives up: there is no point in spending $10b to stand up your own model to try to become #1 when XYZ/Anthropic
Absolutely. The increasing cost of training compute and architecture searches, and relatively low cost of inference compute guarantees this. A model that has had more training compute and a better architecture will perform better on more affordable levels of compute across the board. This is also why an Intel or AMD CPU, or Nvidia GPU, is more worth the same amount of silicon than an inferior product.
Wonder why it’s a stable two-player oligopoly and not a straight monopoly? From large corporate buyers preventing a monopoly by buying enough from the 2nd place player to keep them afloat?
I agree that most investment wouldn’t have otherwise gone to OAI. I’d speculate that investments from VCs would likely have gone to some other AI startup which doesn’t care about safety; investments from Google (and other big tech) would otherwise have gone into their internal efforts. I agree that my framing was reductive/over-confident and that plausibly the modal ‘other’ AI startup accelerates capabilities less than Anthropic even if they don’t care about safety. On the other hand, I expect diverting some of Google and Meta’s funds and compute to Anthropic is net good, but I’m very open to updating here given further info on how Google allocates resources.
I don’t agree with your ‘horribly unethical’ take. I’m not particularly informed here, but my impression was that it’s par-for-the-course to advertise and oversell when pitching to VCs as a startup? Such an industry-wide norm could be seen as entirely unethical, but I don’t personally have such a strong reaction.
I agree it’s common for startups to somewhat oversell their products to investors, but I think it goes far beyond “somewhat”—maybe even beyond the bar for criminal fraud, though I’m not sure—to tell investors you’re aiming to soon get “too far ahead for anyone to catch up in subsequent cycles,” if your actual plan is to avoid getting meaningfully ahead at all.
Not making any claims about actual Anthropic strategy here, but as gwern notes, I don’t think that these are necessarily contradictory. For example, you could have a strategy of getting far enough ahead that new entrants like e.g. Mistral would have a hard time keeping up, but staying on pace with or behind current competitors like e.g. OpenAI.
I assumed “anyone” was meant to include OpenAI—do you interpret it as just describing novel entrants? If so I agree that wouldn’t be contradictory, but it seems like a strange interpretation to me in the context of a pitch deck asking investors for a billion dollars.
I agree that this is a plausible read of their pitch to investors, but I do think it’s a bit of a stretch to consider it the most likely explanation. It’s hard for me to believe that Anthropic would receive billions of dollars in funding if they’re explicitly telling investors that they’re committingto only release equivalent or inferior products relative to their main competitor.
The first Dario quote sounds squarely in line with releasing a Claude 3 on par with GPT-4 but well afterwards. The second Dario quote has a more ambiguous connotation, but if read explicitly it strikes me as compatible with the Claude 3 release.
If you spent a while looking for the most damning quotes, then these quotes strike me as evidence the community was just wishfully thinking while in reality Anthropic comms were fairly clear throughout. Privately pitching aggressive things to divert money from more dangerous orgs while minimizing head-on competition with OpenAI seems best to me (though obviously it’s also evidence that they’ll actually do the aggressive scaling things, so hard to know).
To make concrete the disagreement, I’d be interested in people predicting on “If Anthropic releases a GPT-5 equivalent X months behind, then their dollars/compute raised will be Y times lower than OpenAI” for various values of X.
“Diverting money” strikes me as the wrong frame here. Partly because I doubt this actually was the consequence—i.e., I doubt OpenAI etc. had a meaningfully harder time raising capital because of Anthropic’s raise—but also because it leaves out the part where this purported desirable consequence was achieved via (what seems to me like) straightforward deception!
If indeed Dario told investors he hoped to obtain an insurmountable lead soon, while telling Dustin and others that he was committed to avoid gaining any meaningful lead, then it sure seems like one of those claims was a lie. And by my ethical lights, this seems like a horribly unethical thing to lie about, regardless of whether it somehow caused OpenAI to have less money.
I don’t see any direct contradiction/lie there, at least between my version and the investor paraphrase. You don’t necessarily have to release to public general access the best model, in order to be so far ahead that competitors can’t catch up.
For example, LLMs at the research frontier could be a natural (Bertrand?) oligopoly where there’s a stable two-player oligopoly for the best models (#1 by XYZ, and #2 by Anthropic), and everyone else gives up: there is no point in spending $10b to stand up your own model to try to become #1 when XYZ/Anthropic will just cut prices or release the next iteration that they’d been holding back and you get relegated to #3 and there’s no reason for anyone to buy yours instead, and you go bankrupt. (This would be similar to other historical examples like Intel/AMD or Illumina: they enjoyed large margins and competing with them was possible, but was very dangerous because they had a lot of pent-up improvements they could unleash if you spent enough to become a threat. Or in the case of the highly stable iOS/Android mobile duopoly, just being too incredibly capital-intensive to replicate and already low-margin because the creators make their money elsewhere like devices/ads, having commoditized their complement.)
And then presumably at some point you either solve safety or the models are so capable that further improvement is unnecessary or you can’t increase capability; then the need for the AI-arms-race policy is over, and you just do whatever makes pragmatic sense in that brave new world.
It seems plausible that this scenario could happen, i.e., that Anthropic and OpenAI end up in a stable two-player oligopoly. But I would still be pretty surprised if Anthropic’s pitch to investors, when asking for billions of dollars in funding, is that they pre-commit to never release a substantially better product than their main competitor.
How surprising would you say you find the idea of a startup trying to, and successfully raising, not billions but tens of billions of dollars by pitching investors they’re asking that their investment could be canceled at any time at the wave of a hand, the startup pre-commits that the investments will be canceled in the best-case scenario of the product succeeding, & that the investors ought to consider their investment “in the spirit of a donation”?
Absolutely. The increasing cost of training compute and architecture searches, and relatively low cost of inference compute guarantees this. A model that has had more training compute and a better architecture will perform better on more affordable levels of compute across the board. This is also why an Intel or AMD CPU, or Nvidia GPU, is more worth the same amount of silicon than an inferior product.
Wonder why it’s a stable two-player oligopoly and not a straight monopoly? From large corporate buyers preventing a monopoly by buying enough from the 2nd place player to keep them afloat?
Note that this situation is not ideal for Nvidia. This only sells 2 sets of training compute clusters sufficient to move the SOTA forward. Why sell 2 when you can sell at least 66? https://blogs.nvidia.com/blog/world-governments-summit/
The reasoning driving it being a government cannot really trust someone else’s model, everyone needs their own.
I agree that most investment wouldn’t have otherwise gone to OAI. I’d speculate that investments from VCs would likely have gone to some other AI startup which doesn’t care about safety; investments from Google (and other big tech) would otherwise have gone into their internal efforts. I agree that my framing was reductive/over-confident and that plausibly the modal ‘other’ AI startup accelerates capabilities less than Anthropic even if they don’t care about safety. On the other hand, I expect diverting some of Google and Meta’s funds and compute to Anthropic is net good, but I’m very open to updating here given further info on how Google allocates resources.
I don’t agree with your ‘horribly unethical’ take. I’m not particularly informed here, but my impression was that it’s par-for-the-course to advertise and oversell when pitching to VCs as a startup? Such an industry-wide norm could be seen as entirely unethical, but I don’t personally have such a strong reaction.
I agree it’s common for startups to somewhat oversell their products to investors, but I think it goes far beyond “somewhat”—maybe even beyond the bar for criminal fraud, though I’m not sure—to tell investors you’re aiming to soon get “too far ahead for anyone to catch up in subsequent cycles,” if your actual plan is to avoid getting meaningfully ahead at all.
Not making any claims about actual Anthropic strategy here, but as gwern notes, I don’t think that these are necessarily contradictory. For example, you could have a strategy of getting far enough ahead that new entrants like e.g. Mistral would have a hard time keeping up, but staying on pace with or behind current competitors like e.g. OpenAI.
I assumed “anyone” was meant to include OpenAI—do you interpret it as just describing novel entrants? If so I agree that wouldn’t be contradictory, but it seems like a strange interpretation to me in the context of a pitch deck asking investors for a billion dollars.
I agree that this is a plausible read of their pitch to investors, but I do think it’s a bit of a stretch to consider it the most likely explanation. It’s hard for me to believe that Anthropic would receive billions of dollars in funding if they’re explicitly telling investors that they’re committing to only release equivalent or inferior products relative to their main competitor.