Thanks for explaining. I now agree that the current cost of inference isn’t a very good anchor for future costs in slowdown timelines.
I’m uncertain, but I still think OpenAI is likely to go bankrupt in slowdown timelines. Here are some related thoughts:
OpenAI probably won’t pivot to the slowdown in time.
They’d have < 3 years to do before running out of money.
Budgets are set in advance. So they’d have even less time.
All of the improvements you list cost time and money. So they’d need to continue spending on R&D, before that R&D has improved their cost of inference. In practice, they’d need to stop pushing the frontier even earlier, to have more time and money available.
There’s not many generations of frontier models left before Altman would need to halt scaling R&D.
Altman is currently be racing to AGI; and I don’t think it’s possible, on the slowdown hypothetical, for him to get enough evidence to convince him to stop in time.
Revenue (prices) may scale down alongside the cost of inference
Under perfect competition, widely-shared improvements in the production of a commodity will result in price decreases rather than profit increases.
There are various ways competition here is imperfect; but I think that imperfect competition benefits Google more than OpenAI. That’s really bad, since OpenAI’s finances are also much worse.
This still makes the cost of inference/cost of revenue estimates wrong; but OpenAI might not be able to make enough money to cover their debt and what you called “essential R&D”. Dunno.
Everybody but Anthropic are already locked in to much of their inference (and R&D) spending via capex.
AI 2027 assumes that OpenAI uses different chips specialized in training vs. inference.
The cost of the datacenter and the GPU’s it contains is fixed, and I believe it makes up most of the cost of inference today. OpenAI, via Project Stargate, is switching from renting GPU’s to building its own datacenters. (This may or may not bring down the cost of inference on its own, depending on what kind of discount OpenAI got from Azure.)
So for inference improvements to be a pure win, OpenAI’s usage needs to grow to the same extent. But in this scenario capabilities have stopped improving, so it probably won’t. Usage might even shrink if some people adopt AI in preparation for future capability improvements.
How to account for capex seems complicated:
We need to amortize the cost of the GPU’s themselves. GPU’s in heavy usage break down over time; OpenAI would need to make enough of a profit to repay the loans and to buy replacement GPU’s. (If they have lower utilization, their existing stock of GPU’s will last longer.) Also, if semiconductor progress continues it will make their existing GPU’s obsolete. I’m skeptical that semiconductor progress will continue at the same rate; but if it does, OpenAI will be locked into using obsolete and inefficient GPU’s.
Project Stargate is planning on spending 100 billion at first, 50 billion of which would be debt. CoreWeave is paying 11-14% annual interest on its debt; if OpenAI pays 10%, it would be locked into paying ~5 billion/year in interest. This, plus essential R&D, inference electricity, revenue share, and other costs, might or might not be doable; however, they’d almost certainly not have much money left over to buy new GPU’s, save up for a balloon payment, or make a profit.
You seem to be assuming that there’s not significant overhead or delays from negotiating leases, entering bankruptcy, or dealing with specialized hardware, which is very plausibly false.
If nobody is buying new datacenter GPU’s, that will cut GPU progress to ~zero or negative (because production is halted and implicit knowledge is lost). (It will also probably damage broader semiconductor progress.)
This reduces the cost to rent a GPU-hour, but it doesn’t reduce the cost to the owner. (OpenAI, and every frontier lab but Anthropic, will own much or all[1] of their own compute. So this doesn’t do much to help OpenAI in particular.)
I think you have a misconception about accounting. GPU depreciation is considered on an income statement, it is part of the operating expenses, subtracted from gross profit to get net profit. Depreciation due to obsolescence vs. breakdowns isn’t treated differently. If OpenAI drops its prices below the level needed to pay for that depreciation, they won’t be running a (net) profit. Since they won’t be buying new GPU’s, they will die in a few years, once their existing stock of GPU’s breaks down or becomes obsolete. To phrase it another way, if you reduce GPU-time prices 3-5x, the global AI compute buildout has not in fact paid for itself.
OpenAI has deals with CoreWeave and Azure; they may specify fixed prices; even if not, CoreWeave’s independence doesn’t matter here, as they also need to make enough money to buy new GPU’s/repay debt. (Azure is less predictable.)