Yeah. I was trying to show that humans don’t fare well by default even in a peaceful “rising tide” scenario, but in truth there will probably be more conflict, where AIs protecting humans don’t necessarily win.
I do think there is a difference in strategy though still. In the foom scenario you want to keep small the number of key players or people that might become key players.
In the non-foom you have the unhappy compromise between trying to avoid too many accidents and building up defense early vs practically everyone in time being a key player and needing to know how to handle AGI.
Yeah. I was trying to show that humans don’t fare well by default even in a peaceful “rising tide” scenario, but in truth there will probably be more conflict, where AIs protecting humans don’t necessarily win.
I didn’t know that!
I do think there is a difference in strategy though still. In the foom scenario you want to keep small the number of key players or people that might become key players.
In the non-foom you have the unhappy compromise between trying to avoid too many accidents and building up defense early vs practically everyone in time being a key player and needing to know how to handle AGI.