It seems like to me that if AGI is eventually created via this paradigm, hardware overhang is almost guaranteed. That’s because it requires meta-learning and the initialization of rich environments to train populations of agents, which is extremely computationally expensive compared to running the actual agents themselves. Therefore, if AGI is obtained this way, we could rather easily re-purpose the hardware used for training in order to run a large number of these agents, which could correspondingly perform vast amounts of economic labor relative to the human population.
Whatever is created by this paradigm may turn out just weak enough that repurposing the training hardware will only scale it up to competetive profitability.
I agree that this argument doesn’t automatically imply that there will be a discontinuous jump in profitability of a single system, so it makes sense that the agents created might be just barely more competitive than other agents before it. However, it does imply that by the time we get the hardware necessary to do this, we will have a lot of economic power sitting in our machines by virtue of having a ton of computing power to run the agents.
It seems like to me that if AGI is eventually created via this paradigm, hardware overhang is almost guaranteed. That’s because it requires meta-learning and the initialization of rich environments to train populations of agents, which is extremely computationally expensive compared to running the actual agents themselves. Therefore, if AGI is obtained this way, we could rather easily re-purpose the hardware used for training in order to run a large number of these agents, which could correspondingly perform vast amounts of economic labor relative to the human population.
Whatever is created by this paradigm may turn out just weak enough that repurposing the training hardware will only scale it up to competetive profitability.
I agree that this argument doesn’t automatically imply that there will be a discontinuous jump in profitability of a single system, so it makes sense that the agents created might be just barely more competitive than other agents before it. However, it does imply that by the time we get the hardware necessary to do this, we will have a lot of economic power sitting in our machines by virtue of having a ton of computing power to run the agents.