Seems increasingly likely to me that there will be some kind of national AI project before AGI is achieved as the government is waking up to its potential pretty fast. Unsure what the odds are, but last year, I would have said <10%. Now I think it’s between 30% and 60%.
Has anyone done a write up on what the government-led AI project(s) scenario would look like?
I agree that expecting nobody in power to notice the potential before AGI is takeover-capable seems implausible on the slow-takeoff path that now looks likely.
It seems like the incoming administration is pretty free-market oriented. So I’d expect government involvement to mostly be giving the existing orgs money, and just taking over control of their projects as much as seems necessary—or fun.
My guess is that this is reasonably plausible assuming the short timelines are in fact going to happen, but it’s going to be up against a backdrop of a shock to government competence such that the people who could do a national project completely fail to even get started, let alone complete a herculean effort, since all the possible choices for the role are selected based on loyalty, not competence.
I expect the new administration to break the presidential government and agencies competence by extreme amounts, such that I wouldn’t be totally surprised if by the end of the administration, there would be a complete inability to have a national AI project/nationalize the business at all.
Keep in mind the current administration is replacing incompetent bureaucracies with self assembling corporations. The organization is still there, just more competent and under a different name. A government project could just look like telling the labs to create 1 data center, throwing money at them, and cutting red tape for building gas plants.
You first might want to distinguish between national AI projects that are just about boosting the AI economy or managing the use of AI within government, and government-backed research which is specifically aimed at the AGI frontier. Presumably it’s the latter that you’re talking about.
There is also the question of what a government would think it was doing, in embarking on such a project. The commercial enterprise of creating AI is already haunted by the idea that it would be bad for business if your creation wiped out the human race. That hasn’t stopped anyone, but the fear is there, overcome only by greed.
Now, what about politicians and public servants, generals and spymasters? How would they feel about leading a race to create AI? What would they think they were doing? Creating artificial super-scientists, super-soldiers, super-strategists? Compared to Silcon Valley, these people are more about the power motive than the profit motive. What, apart from the arms race, do they have to lure them along the AI path, comparable to the dream of uber-wealth that drives the tech oligarchs? (In dictatorships, I suppose there is also the dream of absolute personal power to motivate them.)
Apart from the arms race, the vision that seems to animate pro-AI western elites, is economic and strategic competition among nations. If China takes the lead in AI, it will have the best products and the best technologies and it will conquer the world that way. So I guess the thinking of Trump 2.0′s AI czar David Sacks (a friend of Thiel and Musk), and the people around him, is going to be some mixture of these themes—the US must lead because AI is the key to economic, technological, and military superiority in the 21st century.
Now I think that even the most self-confident, gung-ho, born-to-rule man-of-destiny who gets involved in the AI race, is surely going to have a moment when they think, am I just creating my own replacement here? Can even my intellect, and my charisma, and my billions, and my social capital, really compete with something smarter than me, and a thousand times faster than me, and capable of putting any kind of human face on its activities?
I’m not saying they’re going to have a come-to-Yudkowsky moment and realize, holy crap, we’d better shut this down after all. Their Darwinist instincts will tell them that if they don’t create AI first, someone else will. But perhaps they will want to be reassured. And this may be one area where techies similar to Ilya Sutskever, and Yann Lecun, and Alec Radford—i.e. the technical leads in these frontier AI research programs—may have a role in addition to their official role as chief of R&D.
The technical people have their own dreams about what a world of AGI and ASI could look like too. They may have a story about prosperity and human flourishing with AI friends and partners. Or maybe they have a story just for their CEO masters, that even the most powerful AI, if properly trained, will just be 100% an extension of their own existing will. And who knows what kind of transhuman dreams they entertain privately, as well?
These days, there’s even the possibility that the AI itself is whispering to the corporate, political, and military leadership, telling them what they want to hear…
I am very much speculating here, I have no personal experience of, or access to, these highest levels of power. But the psychology and ideology of the “decision-makers”—who really just seem to be riding the tiger of technical progress at this point—is surely an important feature of any such AGI Manhattan Project, too.
Seems increasingly likely to me that there will be some kind of national AI project before AGI is achieved as the government is waking up to its potential pretty fast. Unsure what the odds are, but last year, I would have said <10%. Now I think it’s between 30% and 60%.
Has anyone done a write up on what the government-led AI project(s) scenario would look like?
I agree that expecting nobody in power to notice the potential before AGI is takeover-capable seems implausible on the slow-takeoff path that now looks likely.
It seems like the incoming administration is pretty free-market oriented. So I’d expect government involvement to mostly be giving the existing orgs money, and just taking over control of their projects as much as seems necessary—or fun.
My guess is that this is reasonably plausible assuming the short timelines are in fact going to happen, but it’s going to be up against a backdrop of a shock to government competence such that the people who could do a national project completely fail to even get started, let alone complete a herculean effort, since all the possible choices for the role are selected based on loyalty, not competence.
I expect the new administration to break the presidential government and agencies competence by extreme amounts, such that I wouldn’t be totally surprised if by the end of the administration, there would be a complete inability to have a national AI project/nationalize the business at all.
Keep in mind the current administration is replacing incompetent bureaucracies with self assembling corporations. The organization is still there, just more competent and under a different name. A government project could just look like telling the labs to create 1 data center, throwing money at them, and cutting red tape for building gas plants.
You first might want to distinguish between national AI projects that are just about boosting the AI economy or managing the use of AI within government, and government-backed research which is specifically aimed at the AGI frontier. Presumably it’s the latter that you’re talking about.
There is also the question of what a government would think it was doing, in embarking on such a project. The commercial enterprise of creating AI is already haunted by the idea that it would be bad for business if your creation wiped out the human race. That hasn’t stopped anyone, but the fear is there, overcome only by greed.
Now, what about politicians and public servants, generals and spymasters? How would they feel about leading a race to create AI? What would they think they were doing? Creating artificial super-scientists, super-soldiers, super-strategists? Compared to Silcon Valley, these people are more about the power motive than the profit motive. What, apart from the arms race, do they have to lure them along the AI path, comparable to the dream of uber-wealth that drives the tech oligarchs? (In dictatorships, I suppose there is also the dream of absolute personal power to motivate them.)
Apart from the arms race, the vision that seems to animate pro-AI western elites, is economic and strategic competition among nations. If China takes the lead in AI, it will have the best products and the best technologies and it will conquer the world that way. So I guess the thinking of Trump 2.0′s AI czar David Sacks (a friend of Thiel and Musk), and the people around him, is going to be some mixture of these themes—the US must lead because AI is the key to economic, technological, and military superiority in the 21st century.
Now I think that even the most self-confident, gung-ho, born-to-rule man-of-destiny who gets involved in the AI race, is surely going to have a moment when they think, am I just creating my own replacement here? Can even my intellect, and my charisma, and my billions, and my social capital, really compete with something smarter than me, and a thousand times faster than me, and capable of putting any kind of human face on its activities?
I’m not saying they’re going to have a come-to-Yudkowsky moment and realize, holy crap, we’d better shut this down after all. Their Darwinist instincts will tell them that if they don’t create AI first, someone else will. But perhaps they will want to be reassured. And this may be one area where techies similar to Ilya Sutskever, and Yann Lecun, and Alec Radford—i.e. the technical leads in these frontier AI research programs—may have a role in addition to their official role as chief of R&D.
The technical people have their own dreams about what a world of AGI and ASI could look like too. They may have a story about prosperity and human flourishing with AI friends and partners. Or maybe they have a story just for their CEO masters, that even the most powerful AI, if properly trained, will just be 100% an extension of their own existing will. And who knows what kind of transhuman dreams they entertain privately, as well?
These days, there’s even the possibility that the AI itself is whispering to the corporate, political, and military leadership, telling them what they want to hear…
I am very much speculating here, I have no personal experience of, or access to, these highest levels of power. But the psychology and ideology of the “decision-makers”—who really just seem to be riding the tiger of technical progress at this point—is surely an important feature of any such AGI Manhattan Project, too.