Resources both closed and open must be overwhelmingly devoted to defense (vs offense) with respect to possible CBRN and other catastrophic risks from both open and closed models[1]. Otherwise the risk of easy offense, hard defense weapons (like bioweapons) puts civilization at dire risk. Competition and the race to AGI could be seen as a significant detractor from the impetus to devote these necessarily overwhelming resources[2].
So how can we reduce possible recklessness from competition without centralized and therefore most likely corrupt control? To me transparency and open source provide an alternative: Transparency into what the closed hyper-scalers are doing with their billions of dollars worth of inference+training compute[3]; And open source + open science to promote healthy competition and innovation along with public insight into safety and security implications.
With such openness, we must assume there will be a degree of malicious misuse. Again, knowing this upfront, we need to devote both inference and training compute now to heading off such threats[2]. Yes it’s easier to destroy than to create & protect; this is why we must devote overwhelmingly more resources to the latter.
This as controlling and closing CBRN capable models, like you mention, is not likely to happen and bad actors should be assumed to have access already.
Since CBRN defense is an advanced capability and requires complex reasoning, it could actually provide an alignment bonus (vs being an alignment tax) to frontier models. So we should not necessarily equate defense and capability as mutually exclusive.
Resources both closed and open must be overwhelmingly devoted to defense (vs offense) with respect to possible CBRN and other catastrophic risks from both open and closed models[1]. Otherwise the risk of easy offense, hard defense weapons (like bioweapons) puts civilization at dire risk. Competition and the race to AGI could be seen as a significant detractor from the impetus to devote these necessarily overwhelming resources[2].
So how can we reduce possible recklessness from competition without centralized and therefore most likely corrupt control? To me transparency and open source provide an alternative: Transparency into what the closed hyper-scalers are doing with their billions of dollars worth of inference+training compute[3]; And open source + open science to promote healthy competition and innovation along with public insight into safety and security implications.
With such openness, we must assume there will be a degree of malicious misuse. Again, knowing this upfront, we need to devote both inference and training compute now to heading off such threats[2]. Yes it’s easier to destroy than to create & protect; this is why we must devote overwhelmingly more resources to the latter.
This as controlling and closing CBRN capable models, like you mention, is not likely to happen and bad actors should be assumed to have access already.
Since CBRN defense is an advanced capability and requires complex reasoning, it could actually provide an alignment bonus (vs being an alignment tax) to frontier models. So we should not necessarily equate defense and capability as mutually exclusive.
E.g. there should be sufficient compute dedicated to advancing CBRN defensive capability