I don’t see any direct contradiction/lie there, at least between my version and the investor paraphrase. You don’t necessarily have to release to public general access the best model, in order to be so far ahead that competitors can’t catch up.
For example, LLMs at the research frontier could be a natural (Bertrand?) oligopoly where there’s a stable two-player oligopoly for the best models (#1 by XYZ, and #2 by Anthropic), and everyone else gives up: there is no point in spending $10b to stand up your own model to try to become #1 when XYZ/Anthropic will just cut prices or release the next iteration that they’d been holding back and you get relegated to #3 and there’s no reason for anyone to buy yours instead, and you go bankrupt. (This would be similar to other historical examples like Intel/AMD or Illumina: they enjoyed large margins and competing with them was possible, but was very dangerous because they had a lot of pent-up improvements they could unleash if you spent enough to become a threat. Or in the case of the highly stable iOS/Android mobile duopoly, just being too incredibly capital-intensive to replicate and already low-margin because the creators make their money elsewhere like devices/ads, having commoditized their complement.)
And then presumably at some point you either solve safety or the models are so capable that further improvement is unnecessary or you can’t increase capability; then the need for the AI-arms-race policy is over, and you just do whatever makes pragmatic sense in that brave new world.
It seems plausible that this scenario could happen, i.e., that Anthropic and OpenAI end up in a stable two-player oligopoly. But I would still be pretty surprised if Anthropic’s pitch to investors, when asking for billions of dollars in funding, is that they pre-commit to never release a substantially better product than their main competitor.
How surprising would you say you find the idea of a startup trying to, and successfully raising, not billions but tens of billions of dollars by pitching investors they’re asking that their investment could be canceled at any time at the wave of a hand, the startup pre-commits that the investments will be canceled in the best-case scenario of the product succeeding, & that the investors ought to consider their investment “in the spirit of a donation”?
LLMs at the research frontier could be a natural oligopoly where there’s a stable two-player oligopoly for the best models (#1 by XYZ, and #2 by Anthropic), and everyone else gives up: there is no point in spending $10b to stand up your own model to try to become #1 when XYZ/Anthropic
Absolutely. The increasing cost of training compute and architecture searches, and relatively low cost of inference compute guarantees this. A model that has had more training compute and a better architecture will perform better on more affordable levels of compute across the board. This is also why an Intel or AMD CPU, or Nvidia GPU, is more worth the same amount of silicon than an inferior product.
Wonder why it’s a stable two-player oligopoly and not a straight monopoly? From large corporate buyers preventing a monopoly by buying enough from the 2nd place player to keep them afloat?
I don’t see any direct contradiction/lie there, at least between my version and the investor paraphrase. You don’t necessarily have to release to public general access the best model, in order to be so far ahead that competitors can’t catch up.
For example, LLMs at the research frontier could be a natural (Bertrand?) oligopoly where there’s a stable two-player oligopoly for the best models (#1 by XYZ, and #2 by Anthropic), and everyone else gives up: there is no point in spending $10b to stand up your own model to try to become #1 when XYZ/Anthropic will just cut prices or release the next iteration that they’d been holding back and you get relegated to #3 and there’s no reason for anyone to buy yours instead, and you go bankrupt. (This would be similar to other historical examples like Intel/AMD or Illumina: they enjoyed large margins and competing with them was possible, but was very dangerous because they had a lot of pent-up improvements they could unleash if you spent enough to become a threat. Or in the case of the highly stable iOS/Android mobile duopoly, just being too incredibly capital-intensive to replicate and already low-margin because the creators make their money elsewhere like devices/ads, having commoditized their complement.)
And then presumably at some point you either solve safety or the models are so capable that further improvement is unnecessary or you can’t increase capability; then the need for the AI-arms-race policy is over, and you just do whatever makes pragmatic sense in that brave new world.
It seems plausible that this scenario could happen, i.e., that Anthropic and OpenAI end up in a stable two-player oligopoly. But I would still be pretty surprised if Anthropic’s pitch to investors, when asking for billions of dollars in funding, is that they pre-commit to never release a substantially better product than their main competitor.
How surprising would you say you find the idea of a startup trying to, and successfully raising, not billions but tens of billions of dollars by pitching investors they’re asking that their investment could be canceled at any time at the wave of a hand, the startup pre-commits that the investments will be canceled in the best-case scenario of the product succeeding, & that the investors ought to consider their investment “in the spirit of a donation”?
Absolutely. The increasing cost of training compute and architecture searches, and relatively low cost of inference compute guarantees this. A model that has had more training compute and a better architecture will perform better on more affordable levels of compute across the board. This is also why an Intel or AMD CPU, or Nvidia GPU, is more worth the same amount of silicon than an inferior product.
Wonder why it’s a stable two-player oligopoly and not a straight monopoly? From large corporate buyers preventing a monopoly by buying enough from the 2nd place player to keep them afloat?
Note that this situation is not ideal for Nvidia. This only sells 2 sets of training compute clusters sufficient to move the SOTA forward. Why sell 2 when you can sell at least 66? https://blogs.nvidia.com/blog/world-governments-summit/
The reasoning driving it being a government cannot really trust someone else’s model, everyone needs their own.