There’s a fourth possibility for how the first AGI won’t be government-written: goverments might overlook the potential until too late. It seems counter-intuitive to us, since we live in this idea-space, but most goverments are still in the process of noticing the internet. There probably isn’t someone whose job it is to notice that uFAI is a risk, so it’s entirely possible no one will.
Fortunately for startups, big companies are extremely good at denial. If you take the trouble to attack them from an oblique angle, they’ll meet you half-way and maneuver to keep you in their blind spot.
Goverments are even bigger, and even better at denial..
If this seems to be happening, we should probably encourage it.
Hmm, it seems like government AI development might be preferable to a Wild West of private groups. At least in a US-China arms race you have just two parties and so have a shot at treaties and iterated-prisoner’s-dilemma (IPD) game dynamics. With unregulated private developers, you have a multiplayer prisoner’s dilemma, making IPD-type cooperation or other forms of coordination much harder.
most goverments are still in the process of noticing the internet
I don’t know about most governments, but at least some governments are well into the process of achieving full control of the Internet and are also using it to do things they couldn’t do before it existed.
There’s a fourth possibility for how the first AGI won’t be government-written: goverments might overlook the potential until too late. It seems counter-intuitive to us, since we live in this idea-space, but most goverments are still in the process of noticing the internet. There probably isn’t someone whose job it is to notice that uFAI is a risk, so it’s entirely possible no one will.
As Paul Graham wrote in a different context
Goverments are even bigger, and even better at denial..
If this seems to be happening, we should probably encourage it.
Hmm, it seems like government AI development might be preferable to a Wild West of private groups. At least in a US-China arms race you have just two parties and so have a shot at treaties and iterated-prisoner’s-dilemma (IPD) game dynamics. With unregulated private developers, you have a multiplayer prisoner’s dilemma, making IPD-type cooperation or other forms of coordination much harder.
True, but governments have some really scary terminal values.
What are they, how do you know them, and how certain are you?
I don’t know about most governments, but at least some governments are well into the process of achieving full control of the Internet and are also using it to do things they couldn’t do before it existed.