Short version: The claim that AI automation of software engineering will erase NVIDIA’s software advantage misunderstands that as markets expand, the rewards for further software improvements grow substantially. While AI may lower the cost of matching existing software capabilities, overall software project costs are likely to keep increasing as returns on optimization rise. Matching the frontier of performance in the future will still be expensive and technically challenging, and access to AI does not necessarily equalize production costs or eliminate NVIDIA’s moat.
I often see the argument that, since NVIDIA is largely software, when AI automates software, NVIDIA will have no moat, and therefore NVIDIA a bad AI bet. The argument goes something like: AI drives down the cost of software, so the barriers to entry will be much lower. Competitors can “hire” AI to generate the required software by, for example, tasking LLMs with porting application-level code into appropriate low-level instructions, which would eliminate NVIDIA’s competitive advantage stemming from CUDA.
However, while the cost of matching existing software capabilities will decline, the overall costs of software projects are likely to continue increasing, as is the usual pattern. This is because, with software, the returns to optimization increase with the size of the addressable market. As the market expands, companies have greater incentives to invest intensely because even small improvements in performance or efficiency can yield substantial overall benefits. These improvements impact a large number of users, and the costs are amortized across this extensive user base.
Consider web browsers and operating systems: while matching 2000s-era capabilities now takes >1000x fewer developer hours using modern frameworks, the investments that Google makes in Chrome and Microsoft in Windows vastly exceed what tech companies spent in the 2000s. Similarly, as AI becomes a larger part of the overall economy, I expect the investments needed for state-of-the-art GPU firmware and libraries to be greater than those today.
When software development is mostly AI-driven, there will be opportunities to optimize software with more spending, such as by spending on AI inference, building better scaffolding, or producing better ways of testing and verifying potential improvements. This just seems to match our understanding of inference scaling for other complex reasoning tasks, such as programming or mathematics.
It’s also unlikely that the relative cost of producing the same software will be much more equalized; that anyone can hire the same “AI” to do the engineering. Just having access to the raw models is often not sufficient for getting state-of-the-art results (good luck matching AlphaProof’s IMO performance with the Gemini API).
To be clear, I am personally not too optimistic about NVIDIA’s long term future. There are good reasons to expect their moat won’t persist:
Dethroning NVIDIA is now a trillion dollar proposition, and their key customers are all trying to produce GPU substitutes
Rapid technological progress tends to erode competitive advantages by enabling substitute technologies
NVIDIA has had issues adopting new technologies, such as CoWoS-L packaging, and therrefore appears less competent in staying ahead of its competition.
My claim is narrower: the argument that “when AI can automate software engineering, companies whose moat involves software will be outcompeted” seems incorrect.
Short version: The claim that AI automation of software engineering will erase NVIDIA’s software advantage misunderstands that as markets expand, the rewards for further software improvements grow substantially. While AI may lower the cost of matching existing software capabilities, overall software project costs are likely to keep increasing as returns on optimization rise. Matching the frontier of performance in the future will still be expensive and technically challenging, and access to AI does not necessarily equalize production costs or eliminate NVIDIA’s moat.
I often see the argument that, since NVIDIA is largely software, when AI automates software, NVIDIA will have no moat, and therefore NVIDIA a bad AI bet. The argument goes something like: AI drives down the cost of software, so the barriers to entry will be much lower. Competitors can “hire” AI to generate the required software by, for example, tasking LLMs with porting application-level code into appropriate low-level instructions, which would eliminate NVIDIA’s competitive advantage stemming from CUDA.
However, while the cost of matching existing software capabilities will decline, the overall costs of software projects are likely to continue increasing, as is the usual pattern. This is because, with software, the returns to optimization increase with the size of the addressable market. As the market expands, companies have greater incentives to invest intensely because even small improvements in performance or efficiency can yield substantial overall benefits. These improvements impact a large number of users, and the costs are amortized across this extensive user base.
Consider web browsers and operating systems: while matching 2000s-era capabilities now takes >1000x fewer developer hours using modern frameworks, the investments that Google makes in Chrome and Microsoft in Windows vastly exceed what tech companies spent in the 2000s. Similarly, as AI becomes a larger part of the overall economy, I expect the investments needed for state-of-the-art GPU firmware and libraries to be greater than those today.
When software development is mostly AI-driven, there will be opportunities to optimize software with more spending, such as by spending on AI inference, building better scaffolding, or producing better ways of testing and verifying potential improvements. This just seems to match our understanding of inference scaling for other complex reasoning tasks, such as programming or mathematics.
It’s also unlikely that the relative cost of producing the same software will be much more equalized; that anyone can hire the same “AI” to do the engineering. Just having access to the raw models is often not sufficient for getting state-of-the-art results (good luck matching AlphaProof’s IMO performance with the Gemini API).
To be clear, I am personally not too optimistic about NVIDIA’s long term future. There are good reasons to expect their moat won’t persist:
Dethroning NVIDIA is now a trillion dollar proposition, and their key customers are all trying to produce GPU substitutes
Rapid technological progress tends to erode competitive advantages by enabling substitute technologies
NVIDIA has had issues adopting new technologies, such as CoWoS-L packaging, and therrefore appears less competent in staying ahead of its competition.
My claim is narrower: the argument that “when AI can automate software engineering, companies whose moat involves software will be outcompeted” seems incorrect.