This type of scheme to split a rapidly-growing pie semi fairly will definitely help reduce the urge to strike first.
If proliferation continues unchecked, we’ll have RSI-capable AGI in the hands of teenagers and other malcontents eventually. And they often have irrational urges to strike first :)
But this type of scheme might stabilize the situation amongst a few AGIs in different hands, allowing them to collectively enforce not creating more and proliferating further.
If proliferation continues unchecked, we’ll have RSI-capable AGI in the hands of teenagers and other malcontents eventually. And they often have irrational urges to strike first :)
Contra teenagers and the like, I’m hopeful that very capable open-weights models get banned early enough or at least dangerous capabilities get neutered really well using research in the shape of Tamper-Resistant Safeguards for Open-Weight LLMs.
Might be tougher to deal with ‘other malcontents’ like perhaps some states (North Korea, Russia), especially if weights remain relatively easy to steal by state actors.
(I’ve only skimmed for now but) here’s a reason / framework which might help with things going well: https://aiprospects.substack.com/p/paretotopian-goal-alignment.
There we go!
This type of scheme to split a rapidly-growing pie semi fairly will definitely help reduce the urge to strike first.
If proliferation continues unchecked, we’ll have RSI-capable AGI in the hands of teenagers and other malcontents eventually. And they often have irrational urges to strike first :)
But this type of scheme might stabilize the situation amongst a few AGIs in different hands, allowing them to collectively enforce not creating more and proliferating further.
Contra teenagers and the like, I’m hopeful that very capable open-weights models get banned early enough or at least dangerous capabilities get neutered really well using research in the shape of Tamper-Resistant Safeguards for Open-Weight LLMs.
Might be tougher to deal with ‘other malcontents’ like perhaps some states (North Korea, Russia), especially if weights remain relatively easy to steal by state actors.