As Zvi mentioned in one of the roundups, the conventional wisdom for entering a new monopolistic tech niche is to grow as fast as possible.
So it’s likely that OpenAI loses money per user. GitHub copilot allegedly costs $40 in compute per $20 a month subscriber.
So yes, you are right, but no, it doesn’t matter. This is because there’s other variables. The cost of compute is driven up by outside investment. If somehow dynamiting openAI causes all the outside investors to go invest somewhere else—sort of like the hype cycles for nft or crypto—the cost of compute would drop.
For example Nvidia is estimated to pay $3000 to build each H100. If Nvidia charges $5000 a card, and stops charging a 20 percent software license fee, that essentially cuts the compute cost by more than half*, making current AI models at current prices more than profitable.
Nvidia would do this in the hypothetical world of “investors get bored and another ai winter begins”. This neglects Nvidia reducing their costs and developing a cheaper to build card per unit of LLM performance, which they obviously are doing.
*Quick and dirty sanity check: assuming 50 percent utilization (GPU is bounded by memory I/o then it would use $33,000 in electricity over 5 years and currently costs $50,000 at current prices, 25k is list price, 25k is license fee. Were Nvidia to simply charge a more modest margin the all in cost would drop from 83k to 38k. Data center electricity is probably cheaper than 13 cents but there are costs for backup power and other systems)
Conclusion: what’s different now is general ai is bringing in enough revenue to be a self sustaining business. It’s not an industry that can fold and go dormant like failed tech startups that folded and the product or service they developed ceased to be available anywhere.
The time to blow up openAI was prior to the release of chatGPT.
As Zvi mentioned in one of the roundups, the conventional wisdom for entering a new monopolistic tech niche is to grow as fast as possible.
So it’s likely that OpenAI loses money per user. GitHub copilot allegedly costs $40 in compute per $20 a month subscriber.
So yes, you are right, but no, it doesn’t matter. This is because there’s other variables. The cost of compute is driven up by outside investment. If somehow dynamiting openAI causes all the outside investors to go invest somewhere else—sort of like the hype cycles for nft or crypto—the cost of compute would drop.
For example Nvidia is estimated to pay $3000 to build each H100. If Nvidia charges $5000 a card, and stops charging a 20 percent software license fee, that essentially cuts the compute cost by more than half*, making current AI models at current prices more than profitable.
Nvidia would do this in the hypothetical world of “investors get bored and another ai winter begins”. This neglects Nvidia reducing their costs and developing a cheaper to build card per unit of LLM performance, which they obviously are doing.
*Quick and dirty sanity check: assuming 50 percent utilization (GPU is bounded by memory I/o then it would use $33,000 in electricity over 5 years and currently costs $50,000 at current prices, 25k is list price, 25k is license fee. Were Nvidia to simply charge a more modest margin the all in cost would drop from 83k to 38k. Data center electricity is probably cheaper than 13 cents but there are costs for backup power and other systems)
Conclusion: what’s different now is general ai is bringing in enough revenue to be a self sustaining business. It’s not an industry that can fold and go dormant like failed tech startups that folded and the product or service they developed ceased to be available anywhere.
The time to blow up openAI was prior to the release of chatGPT.