LLMs at the research frontier could be a natural oligopoly where there’s a stable two-player oligopoly for the best models (#1 by XYZ, and #2 by Anthropic), and everyone else gives up: there is no point in spending $10b to stand up your own model to try to become #1 when XYZ/Anthropic
Absolutely. The increasing cost of training compute and architecture searches, and relatively low cost of inference compute guarantees this. A model that has had more training compute and a better architecture will perform better on more affordable levels of compute across the board. This is also why an Intel or AMD CPU, or Nvidia GPU, is more worth the same amount of silicon than an inferior product.
Wonder why it’s a stable two-player oligopoly and not a straight monopoly? From large corporate buyers preventing a monopoly by buying enough from the 2nd place player to keep them afloat?
Absolutely. The increasing cost of training compute and architecture searches, and relatively low cost of inference compute guarantees this. A model that has had more training compute and a better architecture will perform better on more affordable levels of compute across the board. This is also why an Intel or AMD CPU, or Nvidia GPU, is more worth the same amount of silicon than an inferior product.
Wonder why it’s a stable two-player oligopoly and not a straight monopoly? From large corporate buyers preventing a monopoly by buying enough from the 2nd place player to keep them afloat?
Note that this situation is not ideal for Nvidia. This only sells 2 sets of training compute clusters sufficient to move the SOTA forward. Why sell 2 when you can sell at least 66? https://blogs.nvidia.com/blog/world-governments-summit/
The reasoning driving it being a government cannot really trust someone else’s model, everyone needs their own.