Sure, it’s easy to imagine scenarios where a specific given company could be larger than it is today. But are you envisioning that if we eliminated antitrust laws and made a few other specific changes, then it would become plausible for a single company to take over the entire economy?
My thesis boils down to the simple assertion that feedback loops need not diverge indefinitely, exponential growth can resolve into an S-curve. In the case of a corporation, the technological advantages, company culture, and other factors that allow a company to thrive in one domain (e.g. Google, web search) might not serve it well in another domain (Google, social networks). In the case of AI self-improvement, it might turn out that we eventually enter a domain – for instance, the point where we’ve exhausted human-generated training data – where the cognitive effort required to push capabilities forwards increases faster than the cognitive effort supplied by those same capabilities. In other words, we might reach a point where each successive generation of recursively-designed AI delivers a decreasing improvement over its predecessor. Note that I don’t claim this is guaranteed to happen, I merely argue that it is possible, but that seems to be enough of a claim to be controversial.
We can look at a skill that’s about applying human intelligence like playing Go. It would be possible that the maximum skill level is near what professional go players are able to accomplish. AlphaGo managed to go very much past what humans can accomplish in a very short timeframe and AlphaGo doesn’t even do any self-recursive editing of it’s own code.
Certainly. I think we see that the ease with which computers can definitively surpass humans depends on the domain. For multiplying large numbers, it’s no contest at all. For Go, computers win definitively, but by a smaller margin than for multiplication. Perhaps, as we move toward more and more complex and open-ended problems, it will get harder and harder to leave humans in the dust? (Not impossible, just harder?) I discuss this briefly in a recent blog post, I’d love to hear thoughts / evidence in either direction.
AI can help with producing GPU’s as well. It’s possible to direct a lot more of the worlds economic output into producing GPU’s than is currently done.
Sure. I’m just suggesting that the self-improvement feedback loop would be slower here, because designing and deploying a new generation of fab equipment has a much longer cycle time than training a new model, no?
Perhaps, as we move toward more and more complex and open-ended problems, it will get harder and harder to leave humans in the dust?
A key issue with training AIs for open-ended problems is that’s a lot harder to create good training data for open-ended problems then it is to create high-quality training data for a game with clear rules.
It’s worth noting that one of the problems where humans outperform computers right now are not really the open-ended tasks but things like how to fold laundry.
A key difference between playing go well and being able to fold laundry well is that training data is easier to come by for go.
If you look at the quality that a lot of professionals make when it comes to a lot of decisions involving probability (meaning there’s a lot of uncertainty) they are pretty bad.
Sure. I’m just suggesting that the self-improvement feedback loop would be slower here, because designing and deploying a new generation of fab equipment has a much longer cycle time than training a new model, no?
You don’t need a new generation of fab equipment to make advances in GPU design. A lot of improvements of the last few years were not about having constantly a new generation of fab equipment.
You don’t need a new generation of fab equipment to make advances in GPU design. A lot of improvements of the last few years were not about having constantly a new generation of fab equipment.
Ah, by “producing GPUs” I thought you meant physical manufacturing. Yes, there has been rapid progress of late in getting more FLOPs per transistor for training and inference workloads, and yes, RSI will presumably have an impact here. The cycle time would still be slower than for software: an improved model can be immediately deployed to all existing GPUs, while an improved GPU design only impacts chips produced in the future.
Ah, by “producing GPUs” I thought you meant physical manufacturing.
Yes, that’s not just about new generations of fab equipment.
GPU performance for training models did increase faster than Moore’s law over the last decade. It’s not something where the curve of improvement is slow even without AI.
Sure, it’s easy to imagine scenarios where a specific given company could be larger than it is today. But are you envisioning that if we eliminated antitrust laws and made a few other specific changes, then it would become plausible for a single company to take over the entire economy?
My thesis boils down to the simple assertion that feedback loops need not diverge indefinitely, exponential growth can resolve into an S-curve. In the case of a corporation, the technological advantages, company culture, and other factors that allow a company to thrive in one domain (e.g. Google, web search) might not serve it well in another domain (Google, social networks). In the case of AI self-improvement, it might turn out that we eventually enter a domain – for instance, the point where we’ve exhausted human-generated training data – where the cognitive effort required to push capabilities forwards increases faster than the cognitive effort supplied by those same capabilities. In other words, we might reach a point where each successive generation of recursively-designed AI delivers a decreasing improvement over its predecessor. Note that I don’t claim this is guaranteed to happen, I merely argue that it is possible, but that seems to be enough of a claim to be controversial.
Certainly. I think we see that the ease with which computers can definitively surpass humans depends on the domain. For multiplying large numbers, it’s no contest at all. For Go, computers win definitively, but by a smaller margin than for multiplication. Perhaps, as we move toward more and more complex and open-ended problems, it will get harder and harder to leave humans in the dust? (Not impossible, just harder?) I discuss this briefly in a recent blog post, I’d love to hear thoughts / evidence in either direction.
Sure. I’m just suggesting that the self-improvement feedback loop would be slower here, because designing and deploying a new generation of fab equipment has a much longer cycle time than training a new model, no?
A key issue with training AIs for open-ended problems is that’s a lot harder to create good training data for open-ended problems then it is to create high-quality training data for a game with clear rules.
It’s worth noting that one of the problems where humans outperform computers right now are not really the open-ended tasks but things like how to fold laundry.
A key difference between playing go well and being able to fold laundry well is that training data is easier to come by for go.
If you look at the quality that a lot of professionals make when it comes to a lot of decisions involving probability (meaning there’s a lot of uncertainty) they are pretty bad.
You don’t need a new generation of fab equipment to make advances in GPU design. A lot of improvements of the last few years were not about having constantly a new generation of fab equipment.
Ah, by “producing GPUs” I thought you meant physical manufacturing. Yes, there has been rapid progress of late in getting more FLOPs per transistor for training and inference workloads, and yes, RSI will presumably have an impact here. The cycle time would still be slower than for software: an improved model can be immediately deployed to all existing GPUs, while an improved GPU design only impacts chips produced in the future.
Yes, that’s not just about new generations of fab equipment.
GPU performance for training models did increase faster than Moore’s law over the last decade. It’s not something where the curve of improvement is slow even without AI.