One idea that seems potentially promising is to have a single centralised project and minimize the chance it becomes too powerful by minimizing its ability to take actions in the broader world.
Concretely, a ‘Pre-Training Project’ does pre-training and GCR safety assessment, post-training needed for the above activities (including post-training to make AI R&D agents and evaluating the safety of post-training techniques), and nothing else. And then have many (>5) companies that do fine-tuning, scaffolding, productising, selling API access, and use-case-specific safety assessments.
Why is this potentially the best of both worlds?
Much less concentration of power. The Pre-Training Project is strictly banned from these further activities (and indeed from any other activities) and it is closely monitored. This significantly reduces the (massive and very problematic) concentration of power you’d get from just one project selling AGI services to the world. It can’t shape the uses of the technology to its own private benefit, can’t charge monopoly prices, can’t use its superhuman AI and massive profits for political lobbying and shaping public opinion. Instead, multiple private companies will compete to ensure that the rest of the world gets maximum benefit from the tech.
More work is needed to see whether the power of the Pre-Training Project could really be robustly limited in this way.
No ‘race to the bottom’ within the west. Only one project is allowed to increase the effective compute used in pre-training. It’s not racing with other Western projects, so there is no ‘race to the bottom’. (Though obviously international racing here could still be a problem.)
One idea that seems potentially promising is to have a single centralised project and minimize the chance it becomes too powerful by minimizing its ability to take actions in the broader world.
Concretely, a ‘Pre-Training Project’ does pre-training and GCR safety assessment, post-training needed for the above activities (including post-training to make AI R&D agents and evaluating the safety of post-training techniques), and nothing else. And then have many (>5) companies that do fine-tuning, scaffolding, productising, selling API access, and use-case-specific safety assessments.
Why is this potentially the best of both worlds?
Much less concentration of power. The Pre-Training Project is strictly banned from these further activities (and indeed from any other activities) and it is closely monitored. This significantly reduces the (massive and very problematic) concentration of power you’d get from just one project selling AGI services to the world. It can’t shape the uses of the technology to its own private benefit, can’t charge monopoly prices, can’t use its superhuman AI and massive profits for political lobbying and shaping public opinion. Instead, multiple private companies will compete to ensure that the rest of the world gets maximum benefit from the tech.
More work is needed to see whether the power of the Pre-Training Project could really be robustly limited in this way.
No ‘race to the bottom’ within the west. Only one project is allowed to increase the effective compute used in pre-training. It’s not racing with other Western projects, so there is no ‘race to the bottom’. (Though obviously international racing here could still be a problem.)