Hsu is a long-time China hawk and has been talking up the scientific & technological capabilities of the CCP for a long time, saying they were going to surpass the West any moment now, so I found this interesting when Hsu explains that:
the scientific culture of China is ‘mafia’ like (Hsu’s term, not mine) and focused on legible easily-cited incremental research, and is against making any daring research leaps or controversial breakthroughs...
but is capable of extremely high quality world-class followup and large scientific investments given a clear objective target and government marching orders
there is no interest or investment in an AI arms race, in part because of a “quiet confidence” (ie. apathy/laying-flat) that if anything important happens, China can just catch up a few years later and win the real race. They just aren’t doing it. There is no Chinese Manhattan Project. There is no race. They aren’t dumping the money into it, and other things, like chips and Taiwan and demographics, are the big concerns which have the focus from the top of the government, and no one is interested in sticking their necks out for wacky things like ‘spending a billion dollars on a single training run’ without explicit enthusiastic endorsement from the very top.
Let the crazy Americans with their fantasies of AGI in a few years race ahead and knock themselves out, and China will stroll along, and scoop up the results, and scale it all out cost-effectively and outcompete any Western AGI-related stuff (ie. be the BYD to the Tesla). The Westerners may make the history books, but the Chinese will make the huge bucks.
So, this raises an important question for the arms race people: if you believe it’s OK to race, because even if your race winds up creating the very race you claimed you were trying to avoid, you are still going to beat China to AGI (which is highly plausible, inasmuch as it is easy to win a race when only one side is racing), and you have AGI a year (or two at the most) before China and you supposedly “win”… Then what?
race to AGI and win
trigger a bunch of other countries racing to their own AGI (now that they know it’s doable, increasingly much about how to do it, can borrow/steal/imitate the first AGI, and have to do so “before it’s too late”)
???
profit!
What does winning look like? What do you do next? How do you “bury the body”? You get AGI and you show it off publicly, Xi blows his stack as he realizes how badly he screwed up strategically and declares a national emergency and the CCP starts racing towards its own AGI in a year, and… then what? What do you do in this 1 year period, while you still enjoy AGI supremacy? You have millions of AGIs which can do… stuff. What is this stuff? Are you going to start massive weaponized hacking to subvert CCP AI programs as much as possible short of nuclear war? Lobby the UN to ban rival AGIs and approve US carrier group air strikes on the Chinese mainland? License it to the CCP to buy them off? Just… do nothing and enjoy 10%+ GDP growth for one year before the rival CCP AGIs all start getting deployed? Do you have any idea at all? If you don’t, what is the point of ‘winning the race’?
(This is a question the leaders of the Manhattan Project should have been asking themselves when it became obvious that there were no genuine rival projects in Japan or Germany, and the original “we have to beat Hitler to the bomb” rationale had become totally irrelevant and indeed, an outright propaganda lie. The US got The Bomb, immediately ensuring that everyone else would be interested in getting the bomb, particularly the USSR, in the foreseeable future… and then what? Then what? “I’ll ask the AGIs for an idea how to get us out of this mess” is an unserious response, and it is not a plan if all of the remaining viable plans the AGIs could implement are one of those previous plans which you are unwilling to execute—similar to how ‘nuke Moscow before noon today’ was a viable plan to maintain nuclear supremacy, but wasn’t going to happen, and it would have been better to not put yourself in that position in the first place.)
Perhaps this—originally private email—is saying the quiet part. And now that it is released, the quiet part is out loud. To use terms from the turn based game of Civilisation, perhaps they would use AI to achieve a cultural, espionage, technological, influence, diplomatic, and military victory simultaneously? But why would they declare that beforehand? Declaring it would only invite opposition and competition.
At the very least, you can hack and spy and sabotage other AGI attempts.
To be specific, there are a few areas where, it seems to me, increased intelligence could lead to quick and leveraged benefits. Hacking, espionage, negotiation, finance, and marketing/propaganda. For example, what if you can capture a significant fraction of the world’s trading income, attract a large portion of China’s talent to turn coat and move to your country, and hack into a large part of an opposition’s infrastructure.
If one or more of these tactics can work significantly, you buy time for other tactics to progress.
Also worth noting is Steve Hsu’s recent discussion of his meetings with China VC, government, researchers etc. reporting from on the ground in Shanghai and Beijing etc: https://www.manifold1.com/episodes/letter-from-shanghai-reflections-on-china-in-2024-73/transcript
Hsu is a long-time China hawk and has been talking up the scientific & technological capabilities of the CCP for a long time, saying they were going to surpass the West any moment now, so I found this interesting when Hsu explains that:
the scientific culture of China is ‘mafia’ like (Hsu’s term, not mine) and focused on legible easily-cited incremental research, and is against making any daring research leaps or controversial breakthroughs...
but is capable of extremely high quality world-class followup and large scientific investments given a clear objective target and government marching orders
there is no interest or investment in an AI arms race, in part because of a “quiet confidence” (ie. apathy/laying-flat) that if anything important happens, China can just catch up a few years later and win the real race. They just aren’t doing it. There is no Chinese Manhattan Project. There is no race. They aren’t dumping the money into it, and other things, like chips and Taiwan and demographics, are the big concerns which have the focus from the top of the government, and no one is interested in sticking their necks out for wacky things like ‘spending a billion dollars on a single training run’ without explicit enthusiastic endorsement from the very top.
Let the crazy Americans with their fantasies of AGI in a few years race ahead and knock themselves out, and China will stroll along, and scoop up the results, and scale it all out cost-effectively and outcompete any Western AGI-related stuff (ie. be the BYD to the Tesla). The Westerners may make the history books, but the Chinese will make the huge bucks.
So, this raises an important question for the arms race people: if you believe it’s OK to race, because even if your race winds up creating the very race you claimed you were trying to avoid, you are still going to beat China to AGI (which is highly plausible, inasmuch as it is easy to win a race when only one side is racing), and you have AGI a year (or two at the most) before China and you supposedly “win”… Then what?
race to AGI and win
trigger a bunch of other countries racing to their own AGI (now that they know it’s doable, increasingly much about how to do it, can borrow/steal/imitate the first AGI, and have to do so “before it’s too late”)
???
profit!
What does winning look like? What do you do next? How do you “bury the body”? You get AGI and you show it off publicly, Xi blows his stack as he realizes how badly he screwed up strategically and declares a national emergency and the CCP starts racing towards its own AGI in a year, and… then what? What do you do in this 1 year period, while you still enjoy AGI supremacy? You have millions of AGIs which can do… stuff. What is this stuff? Are you going to start massive weaponized hacking to subvert CCP AI programs as much as possible short of nuclear war? Lobby the UN to ban rival AGIs and approve US carrier group air strikes on the Chinese mainland? License it to the CCP to buy them off? Just… do nothing and enjoy 10%+ GDP growth for one year before the rival CCP AGIs all start getting deployed? Do you have any idea at all? If you don’t, what is the point of ‘winning the race’?
(This is a question the leaders of the Manhattan Project should have been asking themselves when it became obvious that there were no genuine rival projects in Japan or Germany, and the original “we have to beat Hitler to the bomb” rationale had become totally irrelevant and indeed, an outright propaganda lie. The US got The Bomb, immediately ensuring that everyone else would be interested in getting the bomb, particularly the USSR, in the foreseeable future… and then what? Then what? “I’ll ask the AGIs for an idea how to get us out of this mess” is an unserious response, and it is not a plan if all of the remaining viable plans the AGIs could implement are one of those previous plans which you are unwilling to execute—similar to how ‘nuke Moscow before noon today’ was a viable plan to maintain nuclear supremacy, but wasn’t going to happen, and it would have been better to not put yourself in that position in the first place.)
Maybe they have some idea but don’t want to say it. In recently disclosed internal OpenAI emails, Greg Brockman and Ilya Sutskever said to Elon Musk:
Perhaps this—originally private email—is saying the quiet part. And now that it is released, the quiet part is out loud. To use terms from the turn based game of Civilisation, perhaps they would use AI to achieve a cultural, espionage, technological, influence, diplomatic, and military victory simultaneously? But why would they declare that beforehand? Declaring it would only invite opposition and competition.
At the very least, you can hack and spy and sabotage other AGI attempts.
To be specific, there are a few areas where, it seems to me, increased intelligence could lead to quick and leveraged benefits. Hacking, espionage, negotiation, finance, and marketing/propaganda. For example, what if you can capture a significant fraction of the world’s trading income, attract a large portion of China’s talent to turn coat and move to your country, and hack into a large part of an opposition’s infrastructure.
If one or more of these tactics can work significantly, you buy time for other tactics to progress.