I think you’re right that there’s an implied tension here.
If you think they’re already planning on to do this, why do you think that keeping the ideas to yourself would meaningfully advance capabilities by more than a few months?
Because my biggest fear is proliferation, as we’ve been discussing in If we solve alignment, do we die anyway?. The more AGIs we get coming online around the same time, the worse off we are. My hope after the discussion in that post is that whoever is ahead in the AGI race realizes that wide proliferation of fully RSI capable AGI is certain doom, and uses their AGI for a soft pivotal act- with many apologies and assurances to the rest of the world that it will be used for the betterment of all. They’ll then have to do that, to some degree at least, so the results could be really good.
As for only a few months of time bought, I agree. But it would tighten the race and so create more proliferation. As for interventions, I don’t actually think any interventions that have been discussed or will become viable would actually help. I could be wrong on this easily; it’s not my focus.
I see a path to human survival and flourishing that needs no interventions and no work, except that it would be helpful to make sure the existing schemes for aligning language model cognitive architectures (and similar proto-AGIs) to personal intent are actually going to work. They look to me like they will, but I haven’t gotten enough people to actually consider them in detail.
I think a crux here is that even conditional on your ideas working IRL, I think that in practice, there will only be 3-4 AGIs by the big labs, and plausibly limited to Deepmind, OpenAI and Anthropic, because of the power and energy cost of training future AIs will become a huge bottleneck by 2030.
Another way to say it is I think open-source will just totally lose the race to AGI, so I only consider the actors of Meta, Deepmind, Anthropic and OpenAI as relevant actors for several years, so these are the only actors that could actually execute on your ideas even if they proliferated.
I think you’re right that there’s an implied tension here.
Because my biggest fear is proliferation, as we’ve been discussing in If we solve alignment, do we die anyway?. The more AGIs we get coming online around the same time, the worse off we are. My hope after the discussion in that post is that whoever is ahead in the AGI race realizes that wide proliferation of fully RSI capable AGI is certain doom, and uses their AGI for a soft pivotal act- with many apologies and assurances to the rest of the world that it will be used for the betterment of all. They’ll then have to do that, to some degree at least, so the results could be really good.
As for only a few months of time bought, I agree. But it would tighten the race and so create more proliferation. As for interventions, I don’t actually think any interventions that have been discussed or will become viable would actually help. I could be wrong on this easily; it’s not my focus.
I see a path to human survival and flourishing that needs no interventions and no work, except that it would be helpful to make sure the existing schemes for aligning language model cognitive architectures (and similar proto-AGIs) to personal intent are actually going to work. They look to me like they will, but I haven’t gotten enough people to actually consider them in detail.
I think a crux here is that even conditional on your ideas working IRL, I think that in practice, there will only be 3-4 AGIs by the big labs, and plausibly limited to Deepmind, OpenAI and Anthropic, because of the power and energy cost of training future AIs will become a huge bottleneck by 2030.
Another way to say it is I think open-source will just totally lose the race to AGI, so I only consider the actors of Meta, Deepmind, Anthropic and OpenAI as relevant actors for several years, so these are the only actors that could actually execute on your ideas even if they proliferated.