If proliferation continues unchecked, we’ll have RSI-capable AGI in the hands of teenagers and other malcontents eventually. And they often have irrational urges to strike first :)
Contra teenagers and the like, I’m hopeful that very capable open-weights models get banned early enough or at least dangerous capabilities get neutered really well using research in the shape of Tamper-Resistant Safeguards for Open-Weight LLMs.
Might be tougher to deal with ‘other malcontents’ like perhaps some states (North Korea, Russia), especially if weights remain relatively easy to steal by state actors.
Contra teenagers and the like, I’m hopeful that very capable open-weights models get banned early enough or at least dangerous capabilities get neutered really well using research in the shape of Tamper-Resistant Safeguards for Open-Weight LLMs.
Might be tougher to deal with ‘other malcontents’ like perhaps some states (North Korea, Russia), especially if weights remain relatively easy to steal by state actors.