It looks like this was only the start. The news from the past few days is alarming at face value: it sounds like a near total ban on everything, TSMC fabbing only the start (!), now all ASML customer support† is gone (!!) not just EUV machines, on top of which all the American-citizen employees (a nontrivial fraction due to education/birth abroad) have halted work literally overnight (!!!), and overall what sounds like the collapse of Chinese semiconductors amidst a Chinese economy already showing serious signs of distress and little capacity to keep an industry on life-support indefinitely as they scramble to survive—and once they go down in a cascade of bankruptcies/liquidations*, each node killing the nodes dependent on it, and everything is sold off and liquidated and employees scatter to the four winds, rebuilding that semiconductor ecosystem (which has already cost $100b+ and decades to build in repeated efforts) years later, when even further behind, will be more like starting from scratch than turning on an idle car. (And how long might that be, if China is entering its ‘lost decades’...?) What’s the exit plan here for the US, what does success look like? If there is one thing to learn from past attempts at strategic embargoes like China’s failed rare earth embargo or Russia’s failed gas extortion, it’s that while they can be extremely effective in the short term, they are one-shot weapons which neutralize themselves in the less short run.
Or to put it another way, if the CCP tries to invade Taiwan in the next 10 years, historians are likely going to point to 2-3 days ago as the pivot: it is now a razor blade cutting China’s throat if AI and high tech in general is the future, with nothing more to lose and everything to gain from destroying TSMC so no one else can have it either. (Meanwhile, the NYT front page story: “Companies Are Hoarding Workers. That Could Be Good News for the Economy.” Glad our media is still doing a bangup job of keeping everything in perspective.)
The implications for AI scaling and timelines are also immense: aside from the obvious disaster for Chinese AI research, which will stagnate at present Chinchilla-level runs, any invasion attempt is tantamount to the destruction of TSMC (you can now be sure some cruise missiles will find their way to TSMC chip fabs conveniently set out as hostages on the western side of Taiwan & most definitely not hardened underground in Taiwan’s mountains) and will set back scaling by years, possibly decades given the realities of chip fab, capital investment, globalized supply chains, tacit knowledge, experience curves, and risk premia leading to compounding falling behind exponential projections.
So uh… yikes? Is this really what’s happening? Is there some ameliorating factor not covered in any of the reporting so far? We should all probably be paying way more attention to this than Elon Musk’s latest bipolar antics, or even Putin’s nuclear blackmail, because this appears to actually be happening right now, not merely possibly some day. We appear to live in interesting times.
Links (in no particular order):
https://www.scmp.com/tech/tech-war/article/3195785/tech-war-chinas-top-chip-equipment-maker-removes-us-employees-product (note: semi-independent; I have not seen any party media report on this, which given that we are in the runup to the party congress & Xi’s coronation‡ where extreme measures are being imposed & all bad news is verboten, suggests that the CCP regards this as rather bad news not to be mentioned at all, otherwise it would either be denying such hostile rumors or excoriating the USA & Dark Biden; Chinese censorship strategy is to silence & distract, so a total blackout from mainland media, rather than semi-independent HK media, is consistent with this being as apocalyptic as it sounds. EDIT: haven’t read any full translations of speeches yesterday, but coverage didn’t highlight any specific chip comments, other than generic Xi comments about the many threats to Chinese national security and ‘stormy waters’ ahead. Still no official CCP or CCP company comment on any of this, apparently, so the total blackout continues.)
Apple has put on hold plans to use memory chips from China’s Yangtze Memory Technologies Co. (YMTC) in its products, multiple sources told Nikkei Asia...Apple originally planned to start using the government-funded YMTC’s chips as early as this year, as they are at least 20% cheaper than those of its leading rivals, supply chain executives said...Apple had already completed the months-long process to certify YMTC’s 128-layer 3D NAND flash memory for use in iPhones...Apple originally planned to start using the government-funded YMTC’s chips as early as this year, as they are at least 20% cheaper than those of its leading rivals, supply chain executives said...YMTC chips were initially planned to be used only for iPhones sold in the Chinese market. One source, however, said Apple was considering eventually purchasing up to 40% of the NAND flash memory needed for all iPhones from YMTC.
...YMTC’s 128-layer memory chips fall under the scope of these rules. This means it might no longer have the technological capacity to produce enough quality and quantity of chips for Apple, even if the iPhone maker wanted to source from the company, analysts said. “Apple may continue wanting to use YMTC in the local market for China. But the way the regulations are set up currently, it’s very unlikely that YMTC will even be able to supply the kind of NAND chips in a couple of years that Apple would want,” said Brent Fredberg, director of investments at Brandes Investment Partners in San Diego.
...Founded in 2016, YMTC is currently ramping up production at its second chip plant, which is slated to begin mass production this year. The potential deal with Apple was viewed by the industry as a massive victory for China’s semiconductor segment, as it would prove its ability to provide quality products for top global brands. YMTC has been stepping up efforts to reduce dependence on American chipmaking equipment and components since 2020, following Washington’s crackdown on Chinese tech champion Huawei Technologies. However, replacing U.S. suppliers is not easy given their stranglehold on key areas like chipmaking tools, and YMTC’s plan to ramp up production still relies heavily on their support.
Apple and YMTC did not respond to requests for comment.
Up to half of all iPhone memory worldwide! That was a huge order they lost, literally billions upon billions. And that’s just one order. And further, it was the one they were counting on to bankroll their chip fab—like a shark, massive capital investments drawing on a huge international web of dependencies don’t do well when suddenly frozen in place...
China’s top technology overseer convened a series of emergency meetings over the past week with leading semiconductor companies, seeking to assess the damage from the Biden administration’s sweeping chip restrictions and pledging support for the critical sector. The Ministry of Industry and Information Technology has summoned executives from firms including Yangtze Memory Technologies Co. and supercomputer specialist Dawning Information Industry Co. into closed-door meetings since Washington unveiled measures to contain China’s technological ambitions.
MIIT officials appeared uncertain about the way forward and at times appeared to have as many questions as answers for the chipmakers, people familiar with the discussions said. While they refrained from hinting about counter-measures, officials stressed the domestic IT market would provide sufficient demand for affected companies to keep operating, the people said, asking to remain anonymous on a sensitive issue.
Many of the participants argued US curbs collectively spell doom for their industry, as well as China’s ambitions to un-tether its economy from American technology. Yangtze Memory, among China’s best hopes of getting into cutting-edge chipmaking, warned the MIIT its future may be in jeopardy, according to one of the people.
“China’s chip executives brace for winter as US sanctions push country’s semiconductor industry to the brink of desperation”, SMCP (notable points: unusually strong language in the title; VC investment has halted; orders have fallen substantially, such as −20%; chip business failures already exceed 2021′s total; SMIC has halved investments; one VC thinks startups will need at least 18 months of cashflow in the bank to survive; there’s some hype/happy-talk about how maybe they can just pivot to photonics out of semiconductor entirely; and there is zero discussion of CCP government bailout/support.)
See also: for historical background on China’s previous failed efforts to create a autarkic domestic chip industry, see ‘The Sour Past of “China Chips”’
...Estevez said the Chinese commerce ministry — which since the Trump administration has refused to allow US officials to conduct end-use checks to ensure that American technology was not being diverted for unauthorised activity such as the manufacture of weapons — had become more receptive since Washington imposed the controls in October.
...Beijing approved visits of US officials to companies in Wuhan, Shanghai and several cities in Guangdong province in November, according to four government officials with direct knowledge of the matter. The decision came after the semiconductor industry and local authorities filed a series of petitions on the sweeping impact of the latest export controls. “It is the industry’s unanimous response to the escalating US ban that has Beijing beginning to waver on whether it should continue to escalate its confrontation with the US over semiconductors,” said a government official in the tech hub of Shenzhen who was familiar with the matter. “Against such a sluggish macroeconomic backdrop, if geopolitical influences continue to penetrate, it does not benefit the Chinese semiconductor chain.”
In China’s southern tech hub of Shenzhen, employees at chipmaking start-up PXW Semiconductor Manufactory began to panic after the US put their company on a trade blacklist last week. “Most team leaders and executives are in emergency meetings, but the rest of us are not allowed to discuss such a ‘sensitive’ matter,” an employee said, adding that their boss’s office door remained closed on Friday, one day after the US added PXW to the “entity list” along with 35 other Chinese companies.
...Some of the companies targeted last week, including PXW, are only just starting to develop their semiconductor business and thus more vulnerable than established players such as Huawei. “The US government has mastered the Chinese semiconductor supply chain and knows who the priorities are and who are with future potential,” said Brady Wang, a Taiwan-based analyst at research firm Counterpoint...“The US is developing an increasingly detailed understanding of the industry in China, including players you would have considered as obscure,” the official said.
PXW has strong support, including funding from the Shenzhen government and the leadership of a former Huawei executive. The company has ordered equipment from various US companies scheduled to arrive next year, but it might now never receive it, according to two company employees.
Another unexpected addition to the list is Hefei Core Storage Electronic, a company founded by former staff of Taiwanese chip design company VIA Technologies to develop a homegrown alternative to Intel-based PC processors. “It is a bad surprise,” said a Hefei Core Storage engineer. “Nobody expected that we would be on their radar.”
...Yangtze Memory Technologies, China’s largest memory chip maker, was already hit hard by the October controls. The company had halted its expansion and asked US equipment manufacturers to return down payments for previously ordered tools, said a senior engineer at YMTC. “At that time, we could still consider retreating to [making less advanced] chips, but now our fate is all but sealed,” he said, referring to the near impossibility of getting licences approved for equipment to expand production after being put on the entity list. YMTC had already suspended talks with Apple on supplying memory chips for iPhones in China. Research company TrendForce predicts it could be forced to exit the market for advanced 3D Nand flash products by 2024 as it has lost critical support from toolmakers to compete with rivals on this particular memory technology.
...Washington also included a prominent developer of chipmaking equipment: SMEE (Shanghai Micro Electronics Equipment), which represents China’s only hope of developing homegrown lithography machines, the critical advanced chipmaking tool currently dominated by Dutch company ASML. The company’s lithography machines rely on imported components and have never run in mass production. “There is still a long way to go,” said a Shanghai official who handled SMEE’s development project. But the official pointed out that the company had formed teams of experienced staff to replace ASML field workers who were providing services but later withdrawn due to US export controls. “SMEE doesn’t have personnel who are US persons like some other Chinese chip equipment makers,” Fuller said. “Therefore the controls on US persons included in the October measures are less effective.”
(BGI Genomics also got blacklisted, not that anyone particularly cares about Chinese genetics at this point.)
* One might think it’d be crazy to try to trigger this sort of systemic crisis in the middle of a global, and Chinese, economic crisis. But of course, like chemotherapy, the question isn’t whether it’s bad for you, but worse for the other guy, and potentially pushing the chip ecosystem into a systemic collapse will never be easier than it is today. The USA is much wealthier than China, it can handle high-priced chips better.
† Presumably this extends to updates, upgrades, replacement of consumables, repairs when things break, replacement of broken machines… (How’s Russian military manufacturing & aviation going these days?)
‡ What an utter insult to Xi Jingping, incidentally. He must be furious to have this drop literally days before. Brother Pooh is not noted for his thick skin. I wouldn’t be surprised if whoever is masterminding this in the Biden administration timed it deliberately—after all, given that it’s been several weeks since the TSMC GPU embargo was announced, they could easily have delayed it to this week & after.
So, it’s been about a month and a week. Seems like enough time to evaluate a little the link dump above (which is in semi-chronological order). Where is this chip embargo now? I’d summarize the news & expert opinion thus far as:
The embargo is still solid.
The Biden administration has not walked it back, and confirmed the more restrictive parts. I have not seen any coverage indicating that Chinese corps are trivially circumventing it, both ASML & TSMC seem to be enforcing it, and Chinese corps are biting the bullet in deliberately gimping their chip designs to comply with it. Further, major players like Apple have been canceling equally major orders. None of this would be happening if it were only on paper or could be easily circumvented with a shell corp or something.
The consequences for the Chinese chip industry are still big, and bad.
We have plenty of reports about major layoffs, large hits to revenue, cutbacks to investments/R&D, exodus of US/Taiwan-linked personnel, and a halt to VC investment. Quotes from insiders like VCs or major chip manufacturer representatives range from ‘dire’ to ‘apocalyptic’, with time-ranges in the years for when—hopefully!---things might be better again. (Much less informatively: My previous comment got some circulation on social media, and mockery aside, there weren’t any comments I saw that looked like good arguments for why the impact would be minimal, rather than vague assertions that they would just somehow be fine even if they couldn’t get any ASML gear etc. I also asked anyone who might know something on my recent SF trip why this might not be a big deal, and got nothing, and overall an impression that everyone has been too distracted by the numerous other things happening of late to really process it.)
How big? It’s unclear because these aren’t great times for the global semiconductor industry either, as they are running into the general economic malaise and the bullwhip effect where the excess COVID-induced demand & past supply shortages are fading out—but it’s worth noting that it seems like non-CCP firms like Nvidia & TSMC were well-aware that they were going to overshoot to some degree and prepared for it, and don’t seem to be in nearly as bad shape overall.
But there does not (!) seem to be any massive CCP bailout of the Chinese chip industry planned.
While there are many fiscal stimuli ongoing, including ones announced since the embargo, chip-specific ones have not been announced—as would be necessary both to coordinate and restore confidence in the ecosystem—and the reported layoffs/cutbacks are highly costly mistakes if you expect a big faucet of billions of dollars of free government money to be turned on any month now, so seem to imply that the post-embargo meetings with the government did not spur a bailout effort. If Bloomberg’s reporting is correct (and I have no problem believing that a meeting attended by that many figures had at least one person willing to recount it all near-verbatim to a Bloomberg journalist), then they already collectively told the CCP that they were ‘doomed’ without massive additional investment, and the CCP appears to’ve shrugged and told them that ‘domestic IT market demand was adequate’ for them to survive.
The prospect of a collapse, beyond merely a hard recession, remains unclear, and will be hard to evaluate.
It may be tempting to say that “well, it’s been a month and while they’ve had some painful blows they are clearly still fine”. But that’s never how systemic collapses happen—remember bubbles like the Japan bubble, the US housing bubble, fracking, the repeatedly-averted Chinese bubble popping etc, or consider cryptocurrencies right now: the industry seemed to have weathered its most recent bubble popping with surprisingly few casualties compared to the historical fallout of each cryptocurrency bubble, in part due to bailout purchases/investments by sterling household brands like FTX… and then literally overnight that fell apart, and many individuals at many entities received unwelcome surprises about what connections there were in the cryptocurrency ecosystem. “There is a great deal of ruin in a nation.”
Almost all entities involved still have runway: I mean, if you were so fragile that you had less than 1 month of expenses (in the worst case of abruptly going to zero cashflow) and could have gone bankrupt already, then you were already doomed, embargo or no embargo. It is just very little time, on an industry-wide scale. Zombie companies can stagger on for a long time before finally going bankrupt. (As the quote goes: “slowly, then suddenly.”) Cash has not run out, reality has not set in, optimism remains high, supply stockpiles are only partially depleted, complex machines have not yet broken down or reached the end of maintenance cycles or expected lifetimes, slashed orders mean capacity losses are less important… Similarly, when Putin invaded Ukraine 275 days ago, they were extensively embargoed, particularly on chips, and there have not been any dramatic consequences with screaming headlines about passenger planes falling out of the sky—but there have been consequences, such as the gradual disappearance of all their good high precision missiles/artillery like cruise missiles, the choking off supplies to their front lines, resorting to cannibalizing many units to get one working unit or using very expensive equipment in incredibly wasteful ways (hypersonic missiles on apartment blocks, anti-ship missiles on land targets, S-300 AAs as ghetto cruise missiles), obsolete equipment & reliance on imports like Iranian drones, and factories you’ve never heard of simply dying for lack of chips & other supplies because they were unable to work around so many dependencies lopped off all at once. It has taken many months for subtle signs to show up of real damage, and indeed, many people early on were quite skeptical any Russian embargoes could do any good (surely they will just import it from China---!).
Points #1/2/4 are no surprise but point #3 is a surprise. I think pretty much everyone took for granted that the basic CCP response would be to double down on all the chip subsidies, and if they’d already dumped in $100b, oh well, now they’d dump in $200b (in for a penny, in for a pound). This seems… not to have happened? That’s surprising. At least, I’m surprised. If you aren’t surprised, why not? Is there a bailout somewhere I missed in the news?
So, let’s take as a hypothesis that there is no bailout for chips, especially on a large enough scale to really matter. Why this neglect?
Let’s take further as a hypothesis that the reason for the neglect is not simply Xi deciding to invade Taiwan & therefore writing off the domestic chip industry, as this is an extreme course of action and one that most people claim to find even more improbable than, say, the idea that a Xi-like dictator like Putin would do something as absurdly self-destructive as actually invade Ukraine this year instead of just saber-rattling his massed troops along the border to intimidate them into concessions.
The remaining conclusion would seem to be that Xi has chosen to take the L: he is neither going to massively bail out the domestic chip industry nor take out their competitor, and is just going to let it take its lumps and whatever happens happens, China will just have to get along with whatever chips it can make and the gimped chips TSMC will deign to manufacture for it. (At least, I can’t think of any additional meaningful choice outside the trilemma of ‘bailout, accept defeat, or invasion’.) OK, but why?
I struggled to see how that could make sense.
Let’s go even further into the hinterlands of geopolitical & psychological amateur speculation and ask what Xi is thinking...
From the scaling-pilled perspective, or even just centrist AI perspective, this is an insane position: it is taking a L on one of, if not the most, important future technological capabilities, which in the long run may win or lose wars. If China wants to dominate Asia, much less surpass the obsolete American empire, or create AGI, or lead in aerospace, or create ‘5G’ or whatever, it’s hard to see how it’s going to do that while paying more for chips which are half a decade or worse out of date.
But Xi is not scaling-pilled (after all, few people are, even in the most cutting-edge AI R&D labs). So maybe we should ask: is he centrist on AI? Er… Oh—does he care about AI at all? What evidence is there that he does? There doesn’t seem to be much. Going further: what evidence is there that he even regards chips in toto as being all that important? From Xi’s perspective, what has ‘chipolitics’ looked like thus far?
Huawei: the US embargo of chips to the national champion Huawei, and Huawei’s near-death experience, is where chiplomacy started getting ugly. This involved ordinary bulk consumer chips for smartphones and equipment like 5G base stations. It did not involve high-end GPUs or future chips at all. Huawei simply needed millions of chips to sell for export of hundreds of millions of smartphones to the entire world like Africa, and couldn’t get them. But, they survived, and they may now struggle to get the chips that they would like and rely on alternate suppliers of lower-end chips (any dreams of challenging Apple on its high-end home turf are long gone), but Huawei as a whole does still sell a ton.
Russia: another embargo post-Ukraine, cutting off supply of all sorts of chips, almost all antiquated chips designed decades ago for specialized equipment: again, nothing like a H100 GPU or in any way connected to stuff like ‘4nm nodes’. Even for drones, you can usually get by with pretty old parts or improvising; it’s more important to have lots of cheap drones than geewhiz. Here too the problem is Russia needs (1) millions upon millions of specific chips to feed into existing manufacturing lines to feed the meatgrinder of its ‘eastern front’ (if you will), (2) needs them very soon, preferably a few months ago, as the Ukrainian assault will not stop and they have burned through much of their reserves, and (3) cannot get millions of those specific chips whether a few months ago or a few months from now.
The current US/TSMC chip ban: little of the damage reported thus far has much to do with failing to develop new nodes or not getting access to A100/H100s. Nvidia canceled a lot of orders of them, but I haven’t seen anything report about big corporations going out of business etc, and Chinese AI research seems more or less to be carrying on as before with its current stock. The damage is coming from losing big bulk orders like dumping memory to Apple and from catchup designs not being fabbed and suppliers of existing stuff being knocked out. As time passes, these ‘seen’ damages will be replaced by much larger ‘unseen’ damages cascading out.
Even from an AI research perspective, the damage will be subtle. You can keep on using existing clusters indefinitely; they will just put a ceiling on what you can do with reasonably trained or sized models before the communications delays kill performance, and they will hamstring your budget by costing several times more over the next few years, increasingly so. You can assemble clusters 1 GPU at a time, buying them on the gray market, eventually, years later, reaching a respectable size. You can keep doing AI research which looks like almost all AI research in 2021*, and which will look respectable, and use hand-me-down public models from the West, and do lots of small-scale optimization work (which will be, or has already been, Bitter-Lessoned but that is cold comfort to those who can’t afford the bigger irons). You can keep pumping out CS conference proceedings with low-grade small-budget research that would barely make a decent blog post. (I read a lot of IEEE papers relating to anime research, typically GANs, typically terrible, typically East Asian, and typically a few GPUs max.)
A chip ban doesn’t doom Chinese DL (or Chinese AI in general) research to becoming a cargo cult field, but it does render it increasingly prone to irrelevancy, brain drain of everyone who wants to work on the future of DL and not the past, involution / l’art pour l’art, and organizational/intellectual sunk costs and Galapagos syndrome. Considering that Chinese R&D and science are not particularly healthy or fraud-free in the best of times… But how would an outsider, such as a political leader, notice that the cargo is no longer landing?
* I realize people like to portray AI scaling as some overwhelmingly dominant paradigm. This isn’t true. You can go look at a page of NIPS abstracts and see that that is not the case, or look at the NLP survey the other month and note how few people will endorse scaling propositions on a mere anonymous survey, or note how few Arxiv submissions a day merit a /r/MLscaling submission despite spreading a broad net. AI scaling is far from the majority of AI research; scaling research is merely the majority of research that will matter.
We could add a few other points:
the senior CCP leadership is semi-famous for being ‘technical’ (typically engineering degrees like hydrology or mechanics or aerospace) but little to do with anything computer. Xi Jinping has a degree in chemical engineering from a low-rigor period 43 years ago, and then a degree in BS, both of which might just be mostly fake (pretty common). Propaganda aside, his major intellectual interest is literature, particularly Goethe. He has not overseen any major technical projects, or made any major intellectual contributions I’m aware of; indeed, reading about him, he’s always struck me as being mediocre in every aspect besides Comunist Party infighting, bureaucracy, and consolidating power.
Techlash: Xi’s reign has been marked by constant hostility towards and suppression of ‘big tech’ ie. the very people who would benefit and use cutting-edge chips the most and who would be explaining to Party officials the long-term prospects. There are innumerable angles here (for example, yesterday, it sounds like the CCP may deign to graciously allow a few video game developers to, after a year or two of it simply not being allowed, ‘release video games’), and suppression of individuals like Jack Ma come off as very personal. The rhetoric of the regime emphasizes redistribution, only thinly veiled as ‘voluntary donations’, and the Party defending the public and the ‘China Dream’ from rapacious corporations.
Conversely, his reign has been marked by an emphasis on legible atom-heavy scientific projects, and a general downplaying of everything related to bits or information, unless it has a national security angle (leading to ‘Dutch disease’ where an ultra-niche like facial recognition gets lavishly funded, crowding out more generalizable research). For all the talk of ‘data is the new oil’ or ‘China as a data superpower’ or the advantages from ‘Chinese lack of privacy’, China still drastically underperforms in making good use of it. There are large GPU clusters; but all the really important DL research still comes from the West—I’ve noted that it seems like you could trade all of Chinese DL research impact for 1 or 2 Western labs like DeepMind’s impact and still have enough change left over for coffee. There are large numbers of genetic datasets fragmented over many Chinese groups; but global genetics research remains driven by UK Biobank from 2014 etc.
Attempts at scientific reform and quality control are on uneven ground. Fraud in Chinese science seems to still be endemic, and impact low; past efforts like paying per publication merely made the problem worse. In ML, when fraudulent rings of peer review are discovered, they often involve Chinese conferences or reviewers. Publication in prestigious foreign journals like Science or Nature (rather than Chinese journals) remains a major goal, as a matter of quality control, because domestic Chinese journals cannot be trusted.
There seems to be considerable contempt for the USA and American capabilities in China among ‘wolf warriors’, taking cues from the top, and with considerable historical precendent for authoritarian countries to mistakenly gauge the USA as ‘decadent’ and ‘weak’. This may have been trimmed a bit after Ukraine and seeing what things like HIMARSs can do, but it runs deep and inside the Chinese bubble, there is little correction. (When was the last time Xi Jinping was in the USA and saw more than political flunkies? Or any Chinese, for that matter, given their multi-year near-shutdown of international travel?) The thinness of the air at the heights Emperor Xi inhabits is prone to induce altitude sickness and hallucinations. (But at least, thanks to “Zero COVID”, among his many problems, personally getting COVID is not one of them.) If the Americans are decadent because of their emphasis on software and compute, and China & Xi are superior because they aren’t decadent...
The many stimuli the CCP has used repeatedly over the past decade, and since the chip embargo was put in place, shows that they can and will and have had enough time to do so, yes, but the flip side is that the more you stimulate, the less ammunition you have left for the next stimulus, and the less credibility you have to the populace or markets. At some point your macroeconomics start looking bad. There may not be as much money left as one would assume.
So, in past chip incidents, the primary problem has been a complete absence of any chips, and not so much the advancement of the chips themselves.
He has never seen anyone lose a war due to lack of AI or GPUs; he’s only seen disasters caused by lacking perfectly ordinary chips that his domestic manufacturers probably could’ve made 10 years ago.
And in learning lessons from past chip incidents, what Xi brings to the table is: zero technical competence or expertise in the relevant area, a hatred of software and everything to do with it, and long-standing prioritization of heavy-industry-like stuff (which is clearly visible to the naked eye and ‘conventional’ and ‘prestigious’ and applauded by old credentialed foreigners).
Further, mistakes in this regard may be hard to see. ‘The seen and the unseen’ is a dangerous trap because it is so much easier to see the seen than it is to see the unseen. If Xi makes a mistake on chips, a military mistake, then by the nature of things military, he may never realize it. If the engineers of, say, hypersonic missiles can’t get enough high-end GPUs, their complaints will be ignored by the next layer of management and never punted all the way up to Beijing, and they will simply run their simulations at a lower resolution or take other shortcuts; and if the hypersonic missile in question turns out to be a lemon, inadequate to hit NATO units or US aircraft carriers, how will anyone ever find out short of a war over Taiwan—at which point it is far too late?
Naturally, of course, given a supply of at least basic chips to work with, the establishment will assure him everything is fine, just like the Russian military assured Putin it was not a paper tiger or hopelessly undermined by corruption, and almost all the time they will be right.
That is, from Xi’s perspective, all in all, maybe it looks fairly reasonable to neglect chips right now. They aren’t that important, and don’t seem in that much worse trouble than anything else, while bailing them out to the degree where they can potentially gain, or at least near, the cutting-edge would use up a ton of an increasingly skint government’s money. Plus, as master of the currents of history piloting China to a glorious Chinese Century avenging the Century of Humiliations, he has much bigger fish to fry, like the house of cards which is real estate, and Zero COVID. There will be side effects, yes, but if gaming GPUs becoming expensive helps turn little Aiguo away from a career as a useless game programmer into a respectable hardworking fusion physicist, perhaps that’s even a feature rather than a bug?
Well, I could be wrong about all this. But now I can see at least one perspective from which the chip embargo is a big deal but also Xi’s rational response is to indeed just take it on the chin, and perhaps tone down the rhetoric and engage in a bit more biding one’s time & hiding one’s strength. (I doubt that the long-term aims have changed meaningfully just because Beijing is calibrating its rhetoric a little down from recent peaks of aggression, but in the short term, things will be superficially more peaceful.)
From the scaling-pilled perspective, or even just centrist AI perspective, this is an insane position: it is taking a L on one of, if not the most, important future technological capabilities, which in the long run may win or lose wars. If China wants to dominate Asia, much less surpass the obsolete American empire, or create AGI, or lead in aerospace, or create ‘5G’ or whatever, it’s hard to see how it’s going to do that while paying more for chips which are half a decade or worse out of date.
The scaling-pilled AI view ought to be that scaling AI kills you. Why pretend that there’s a strategic advantage here, as opposed to a loaded gun you can point at your own head if you’re stupid enough?
It’s one thing to say “given China’s actual beliefs, they ought to do X” or “if China were rationally acting on a correct understanding of the world, they would do Y”. But why criticize China for avoiding a self-destructive action that would make sense to do if they had a specific combination of definitely-true, maybe-true, and definitely-false beliefs—a specific combination they don’t in fact have?
Isn’t the “scaling AI kills you” view the conjunction of “scaling-pilled” and “alignment is extremely difficult” views, rather than being identical with the scaling-pilled view?
One could reason as something like:
If alignment is as hard as people make it out to be, we’re in all likelihood dead anyway since Westerners are going to develop AI even if we don’t.
If alignment isn’t as hard as people make it out to be, then the country that controls the most powerful AI will be the one that becomes dominant in the world.
Thus if alignment is hard it doesn’t matter what we do, and if alignment is less hard we should invest in AI. Thus, we should invest in AI.
(There’s some obvious nuance that this argument is missing, e.g. the chance of arms races increasing the difficulty of alignment, but some form of it still seems reasonable to me.)
From the scaling-pilled perspective, or even just centrist AI perspective, this is an insane position: it is taking a L on one of, if not the most, important future technological capabilities, which in the long run may win or lose wars.
Are you suggesting that the sane policy is for Xi to dump in as much subsidies as needed until China catches up in semiconductors with the US and its allies? I haven’t seen anyone else argue this, and it seems implausible to me, given that the latter collectively has much greater financial and scientific/engineering resources. China’s GDP is only $18T vs $58T for OECD, and as you say there’s a lot more corruption/fraud in Chinese R&D, so how can they hope to win an outright tech race (and starting from about a decade behind)?
Are you suggesting that the sane policy is for Xi to dump in as much subsidies as needed until China catches up in semiconductors with the US and its allies? I haven’t seen anyone else argue this
Yes. And perhaps no one else does because they aren’t scaling proponents. But from a scaling perspective, accepting a permanent straitjacket around GPUs & a tightened noose is tantamount to admitting defeat & abandoning the future to other countries; it’d be like expelling all your Jewish scientists in 1935 & banning the mining of uranium. It’s not the beginning of any story that ends in victory & being #1, only of stories that end with you being #10, or #100. Like the Manhattan Project, you pay whatever it costs, and it costs what it costs.
so how can they hope to win an outright tech race (and starting from about a decade behind)?
Well, it’s certainly not easy. It’s not a great situation for China to be in, and yet, it is in fact the one they are in, and they have to deal with it. Reality has no obligation to make anything easy for you, nor should you expect something like ‘dethrone the global hyperpower and create a new world order around the Middle Kingdom’ to be easy. Similarly, it’s not easy to defeat a Russian or Chinese invasion, nor was it easy to develop atomic bombs, etc, but if you don’t, there will be consequences you may find unacceptable—even, existential, one might say—so you don’t get much of a choice. You miss 100% of the shots you don’t take.
China has a lot of resources yet untapped, both financial and otherwise, and is doing many other things like stimulating other sectors of the economy, so clearly it can try more than it is right now—but it appears not to be. If it’s not a capability issue, then it must be a choice. Xi seems to disagree that the consequences of choosing to accept defeat in chips & AI will be all that unacceptable, and my comment here is about thinking through what the strategic logic could be from his perspective which makes that choice an acceptable one because the the chip race game is not worth the candle.
I agree with Rob Bensinger’s response here, plus it’s just a really weird use of “insane”, like saying that Japan would have been insane not to attack Pearl Harbor after the US imposed an oil embargo on them, because “You miss 100% of the shots you don’t take.” Thinking that way only makes sense if becoming a world or regional hegemon was your one and only goal, but how did that become the standard for sanity of world leaders around here?
It’s not a weird use. It is a completely normal one: becoming the hegemon is the avowed goal and so making choices which drop the odds of success so drastically raise questions about the thought process, which in this case I find much better explained by simply mistaken beliefs and desires on Xi’s part combined with ordinary sanity. I have already explained at length why I think those beliefs are mistaken, and I also think the desires are bad: do I think China (in the sense of its population as a whole) is better off for Xi’s powerhunger, or that China would be better off for launching a scaling Manhattan Project? Certainly not: in my opinion, Xi’s reign has made the Chinese people substantially worse off than they would have been under a more status quo leader, and Xi has probably single-handedly curtailed their long-term growth prospects & condemned them to permanent middle income status as they begin to enter a Japan-style malaise, in addition to authoritarian disasters like the Uighurs. But many Chinese believe otherwise now, and endorse further questing for hegemony, and they & Xi are in charge, not you, and desire what they desire, not what you desire—you (and Rob) are projecting your own preferences and moralizing here, instead of trying to understand what is going on.
so making choices which drop the odds of success so drastically
I wouldn’t say “drastically” here so maybe this is the crux. I think the chances of success if China does make an all out push for semiconductors is very low given its own resources and likely US and allies’ responses (e.g. they could collectively way outspend China on their own subsidies). I could express this as <1% chance of having a world leading semi fab 10 years from now and <5% chance 20 years from now, no matter what China chooses to do at this point. If hegemony was the only goal then even a 1% chance would be worth it, but “drastically” makes me think maybe that’s not what you’re saying. These are off the cuff numbers so I’m pretty open to changing my mind about them, but seem reasonable given what I know about China’s research capabilities and what it took for the world to reach its current level of semiconductor technology.
Separately from gwern’s argument, I say that maintaining the gap is still of vital national interest. As an example, one of the arguments in favor of nuclear testing bans is that it unilaterally favors American nuclear supremacy, because only the US has the computational resources to conduct simulations good enough to be used in engineering new weapons.
That logic was applied to Russia, but the same logic applies to China: advanced simulations are useful for almost every dimension of military competition. If they let advanced compute go, that means that the US will be multiple qualitative generations ahead in terms of our ability to simulate, predict, and test-without-risk.
This is a terrible position to be in, geopolitically.
Thanks for really interesting discussion and summary of the state of play. I’m curious what you think of Ben Thompson’s point that China could have an advantage at the “trailing edge”(28-45nm, and 45+) if TSMC were taken off the board?
Maybe the play is to focus on the trailing edge, which is easier to take control of, but still v important in the short-term at least, and temporarily let go of the cutting edge?
Whenever I try and think about Xi’s actions as rational I get hung up on the neverending Zero COVID. Many genuinely think it’s mostly about saving face but even if I try hard I can’t see how it could look anything but childish. They must have convinced themselves it’s actually a good policy. I could at least understand that!
I wonder if given the COVID and real estate crises, Xi’s government just doesn’t have the financial resources to bail out the chips industry, plus maybe they (correctly?) understand that the likelihood of building an internationally competitive chips industry is poor (given the sanctions) even if they do dump in another $200b?
Also, it seems like China is being less antagonistic towards Taiwan and other countries in the last few days. Together with the lack of chips bailout, maybe it means they’ve realized that it was too early to “go loud” and are pivoting back to a softer, more cooperative strategy.
“China is working on a more than 1 trillion yuan ($143 billion) support package for its semiconductor industry.”
“The majority of the financial assistance would be used to subsidise the purchases of domestic semiconductor equipment by Chinese firms, mainly semiconductor fabrication plants, or fabs, they said.”
“Such companies would be entitled to a 20% subsidy on the cost of purchases, the three sources said.”
Uh… it’s either a very good thing or bad thing. As the Chinese quote goes, it is too soon to tell. If I had to come down on one side, right now, I think I would come down on ‘good’; slowing down Chinese AI, which would by default hand AGI to Xi & generally pays even less attention to safety than everyone else does, is good, and while ‘CCP destroys TSMC’ is far from the ideal approach to restricting compute growth, it is at least going to make a difference—unlike almost all other proposals. The upfront cost in economics, human welfare, peace, rules-based global order etc is going to be exorbitant, however. In my lifetime, wars have not had a good track record of producing solutions at low costs. (The situation reminds me of nukes, Seoul, or EU dependence on Russian gas: it could probably have been prevented cheaply early on with the stroke of a pen, but several decades later...)
Oh, crap, I work in Chinese genetics. If Illumina and other suppliers embargo China, my job will go tits up. Any idea how much BGI Genomics will be affected? I was also considering jumping ship to UAV development, so this has been a really bad month. No one I work with knew any of this was happening when I brought it up earlier this week.
BGI Genomics, like the mothership in mainland China proper? I’m less familiar with the corporate structure and revenue and dependency on overseas supply chains than I would hope you are, but my understanding was that BGI didn’t have much business overseas (having failed to compete with Illumina), and was reliant on domestic demand mostly from agriculture & medicine, and didn’t depend on Illumina having spent the past decade+ trying and mostly failing to surpass Illumina (but at least it does have its own decent sequencers). So, having failed in those ways and forced into autarky, it doesn’t seem like sanctions/embargos can hurt BGI much more than it already has been? And macroeconomics-wise, sequencing seems like it would be reasonably robust a business because farmers won’t stop needing genetics-related services nor will patients stop getting sick.
And politics-wise, I see no particular Xi-techlash angle for him doing things like overnight outlawing the industry or censors just refusing to approve any video game release for a year.
So overall, doesn’t seem too bad.
Honestly, UAV seems like it’d be a worse place to go simply because that sounds to me like it’ll be more disrupted by random chip problems and export issues due to being extremely military-linked dual-use tech. Drones are stuffed full of all sorts of random weird little chips (see: Russian problems getting UAVs and resorts like getting them from Iran), and how much domestic Chinese demand for UAVs could there possibly be to make up for exports?
Thanks, I was just really worried because our entire sequencing pipeline uses Illumina products. But I asked around our sequencing division, and they think the difference between using Illumina and BGI products isn’t too big—what BGI lacks in quality it makes up for in lower costs. Apparently, the difference in Q30 (% of reads with errors <0.1% of bases) is ~90-95 for Illumina and 80 for BGI, which is marginal. Switching wouldn’t be a major problem according to them. BGI also uses Chinese chipsets, which means sanctions aren’t going to impact it much. I don’t think this is going to be as bad as I first thought.
How likely do you think China getting cut off from Illumina is? Do you think consumer GPUs are going to be restricted?
Yes, that was my impression. BGI sequencing is not as good as Illumina, but it’s not like it’d destroy them to switch over. And sequencers don’t use any really high-end GPUs (even if you’d like to have them for bioinformatics), so it’s not like a chip embargo is an immediate halt-production problem the way it is for building a new supercomputer or building cars stuffed full of miscellaneous chips
Do you think consumer GPUs are going to be restricted?
Based on a plain reading, consumer GPUs already are restricted: it’s not a ban on A100s/H100s, it’s a ban on any system as powerful as an A100, to quote Nvidia:
The license requirement also includes any future NVIDIA integrated circuit achieving both peak performance and chip-to-chip I/O performance equal to or greater than thresholds that are roughly equivalent to the A100, as well as any system that includes those circuits.
This is a static threshold, which makes no allowance for Moore’s law. It is an upheld hand: “Thus far, and no further!” And it is a threshold which may be biting already. EDIT: it looks like they are going to grandfather in consumer GPUs, ignoring their TFLOPs and focusing only on the interconnect bandwidth+latency, where I think all Nvidia consumer GPUs < A100. This may be a bad bet: people have had not much incentive to figure out how to yoke together large numbers of consumer GPUs (as most people using >1 consumer GPU are either hobbyists who generally can stick all the GPUs they can afford into a single box, or cryptominers for whom interconnect is irrelevant because the PoW mining is by design embarassingly-parallel), so if Chinese hyperscalers or major AI labs can purchase all the 4090s they want, now there will be vastly more incentive to figure out hardware hacks (will, say, TSMC ban Chinese designs for interconnect chips which do no computation...?) or low-communication strategies (even large constant factor penalties, which would render them useless to Western groups, will eventually be worthwhile to Chinese groups barred from >=A100 GPUs.)
What is the ‘peak performance’ roughly equivalent to an A100? Well, Nvidia’s product page tells me an A100 delivers 19 FP32 TFLOPs, where FP32 is probably a conservative number. (I don’t think there are very many cases in DL where you would still need FP64, and while everyone is trying to move to FP16 and lower, where an A100 does 312 TFLOPS instead, you still can’t always easily or reliably train every arch with FP/BF16 and you can encounter some severe problems especially with the important case of large Transformers.) 19 TFLOPS may sound like a lot… but the top-end gaming GPU Nvidia RTX 4090 goes as high as 86 TFLOPS (specced at 82 TFLOPS, with GeForce RTX 3090 Ti at 40 TFLOPS and GeForce RTX 2080 Ti at 14 TFLOPS).
So, if USG is banning ‘peak performance roughly equivalent to A100’, and gaming GPUs are turning in TFLOPS several times that of the A100, then the obvious interpretation would seem to be that yeah, lots of consumer GPUs are already restricted. All the way back to like 2019 GPUs, possibly.
(I was really surprised by this when I went to check the numbers; I did not think that gaming GPUs were that much faster than the A100. I knew that the A100 was already kinda obsolete with the H100 in the pipeline, that there was a ‘datacenter tax’ and ‘enterprise tax’, and also that everyone has kinda understated GPU progress because they were so hard to get for so long and most people have been trucking along with GPUs years out of date—even the V100 still shows up in papers occasionally—etc, but I didn’t think it would be that much slower, like 4x. Either I am badly misunderstanding units here, or I am greatly overrating A100s & underrating recent gaming GPUs like the absurdly large 4090, or I am making a mistake in looking at FP32 because Nvidia put all of the A100 advantages into lower precisions.)
Under Secretary of Commerce Alan Estevez confirmed at a public @CNASdc
event this morning [2022-10-27] that Commerce intends to keep the same technical threshold for chips in place over time.
>Why are gaming GPUs faster than ML GPUs? Are the two somehow adapted to their special purposes, or should ML people just be using gaming GPUs?
They aren’t really that much faster, they are basically the same chips. It’s just that the pro version is 4X as expensive. It’s mostly a datacenter tax. The gaming GPUs do generally run way hotter and power hungry, especially boosting higher, and this puts them ahead against the equivalent ML GPUs in some scenarios.
Price difference is not only a tax though—the ML GPUs do have differences but it usually swings things by 10 to 30 percent, occasionally more. Additionally the pro versions typically have 2X-4X the GPU memory which is a huge qualitative difference in capability, and they are physically smaller and run cooler so you can put a bunch of them inside one server and link them together with high speed NVLink cables into configurations that aren’t practical with 3090s. 3090s have a single NVLink port. Pro cards have three. 4090s curiously have zero—NVIDIA likely trying to stop the trend of using cheap gaming GPUs for research. Also, the ML GPUs for last gen were also on a a 7nm TSMC process, while the gaming GPUs were on Samsung 8nm process. This means the A100 using 250 watts outperforms the 3090 using 400 watts. But they are overall the same chip.
None of that accounts for a 4X or more cost multiplier, and the TLDR is the chips are not that different. If gaming GPUs came in higher memory configurations, and all supported NVLink, and were legally allowed to be sold in datacenters, nobody would pay the cost multiplier.
I just realized that H100s are still available from online vendors for ~189000 yuan (~$27000), which is international market price.
Well then. Time to cash out my savings and make some money. Do you guys think it’s feasible? How much are Chinese H100 prices likely to rise? Should I be trying to scoop up high-end GPUs instead?
Update: local computer shop says international GPU suppliers are still accepting Chinese orders and Chinese GPU prices are stable for now.
China’s top chipmaker, Semiconductor Manufacturing International Corporation, built a 7nm chip in July of this year. This was a big improvement on their previous best of 14nm and apparently surpassed expectations, though is still well behind IBM’s 2nm chip and other similarly small ones. This was particularly surprising given that the Trump administration blacklisted SMIC two years ago, meaning for years they’ve been subject to similar restrictions as Biden recently imposed on all of China. Tough to put this in the right context, but we should follow progress of China’s chipmakers going forwards.
It looks like this was only the start. The news from the past few days is alarming at face value: it sounds like a near total ban on everything, TSMC fabbing only the start (!), now all ASML customer support† is gone (!!) not just EUV machines, on top of which all the American-citizen employees (a nontrivial fraction due to education/birth abroad) have halted work literally overnight (!!!), and overall what sounds like the collapse of Chinese semiconductors amidst a Chinese economy already showing serious signs of distress and little capacity to keep an industry on life-support indefinitely as they scramble to survive—and once they go down in a cascade of bankruptcies/liquidations*, each node killing the nodes dependent on it, and everything is sold off and liquidated and employees scatter to the four winds, rebuilding that semiconductor ecosystem (which has already cost $100b+ and decades to build in repeated efforts) years later, when even further behind, will be more like starting from scratch than turning on an idle car. (And how long might that be, if China is entering its ‘lost decades’...?) What’s the exit plan here for the US, what does success look like? If there is one thing to learn from past attempts at strategic embargoes like China’s failed rare earth embargo or Russia’s failed gas extortion, it’s that while they can be extremely effective in the short term, they are one-shot weapons which neutralize themselves in the less short run.
Or to put it another way, if the CCP tries to invade Taiwan in the next 10 years, historians are likely going to point to 2-3 days ago as the pivot: it is now a razor blade cutting China’s throat if AI and high tech in general is the future, with nothing more to lose and everything to gain from destroying TSMC so no one else can have it either. (Meanwhile, the NYT front page story: “Companies Are Hoarding Workers. That Could Be Good News for the Economy.” Glad our media is still doing a bangup job of keeping everything in perspective.)
The implications for AI scaling and timelines are also immense: aside from the obvious disaster for Chinese AI research, which will stagnate at present Chinchilla-level runs, any invasion attempt is tantamount to the destruction of TSMC (you can now be sure some cruise missiles will find their way to TSMC chip fabs conveniently set out as hostages on the western side of Taiwan & most definitely not hardened underground in Taiwan’s mountains) and will set back scaling by years, possibly decades given the realities of chip fab, capital investment, globalized supply chains, tacit knowledge, experience curves, and risk premia leading to compounding falling behind exponential projections.
So uh… yikes? Is this really what’s happening? Is there some ameliorating factor not covered in any of the reporting so far? We should all probably be paying way more attention to this than Elon Musk’s latest bipolar antics, or even Putin’s nuclear blackmail, because this appears to actually be happening right now, not merely possibly some day. We appear to live in interesting times.
Links (in no particular order):
https://www.scmp.com/tech/tech-war/article/3195785/tech-war-chinas-top-chip-equipment-maker-removes-us-employees-product (note: semi-independent; I have not seen any party media report on this, which given that we are in the runup to the party congress & Xi’s coronation‡ where extreme measures are being imposed & all bad news is verboten, suggests that the CCP regards this as rather bad news not to be mentioned at all, otherwise it would either be denying such hostile rumors or excoriating the USA & Dark Biden; Chinese censorship strategy is to silence & distract, so a total blackout from mainland media, rather than semi-independent HK media, is consistent with this being as apocalyptic as it sounds. EDIT: haven’t read any full translations of speeches yesterday, but coverage didn’t highlight any specific chip comments, other than generic Xi comments about the many threats to Chinese national security and ‘stormy waters’ ahead. Still no official CCP or CCP company comment on any of this, apparently, so the total blackout continues.)
“At least 43 senior executives working with 16 listed Chinese semiconductor companies hold roles from CEO to vice president”, WSJ (Twitter)
https://www.bloomberg.com/news/articles/2022-10-12/asml-orders-us-employees-to-stop-servicing-customers-in-china
https://www.bloomberg.com/news/articles/2022-10-12/us-chip-suppliers-pull-back-from-china-s-yangtze-after-biden-ban
https://twitter.com/jordanschnyc/status/1580889342402129921 https://twitter.com/Scholars_Stage/status/1580950956560199683
https://www.nytimes.com/2022/10/13/us/politics/biden-china-technology-semiconductors.html
https://www.chinatalk.media/p/new-chip-export-controls-explained
https://www.bloomberg.com/news/articles/2022-10-10/china-chip-stocks-drop-as-biden-tightens-rules-on-us-tech-access
https://noahpinion.substack.com/p/biden-declares-economic-war-on-the
“Apple freezes plan to use China’s YMTC chips amid political pressure: Company previously planned to put Chinese-made memory in some iPhones” (NYT on how YMTC’s Apple deal was a major target of Congress):
Up to half of all iPhone memory worldwide! That was a huge order they lost, literally billions upon billions. And that’s just one order. And further, it was the one they were counting on to bankroll their chip fab—like a shark, massive capital investments drawing on a huge international web of dependencies don’t do well when suddenly frozen in place...
“Lam Research warns of up to $2.5b revenue hit from U.S. curbs on China exports”
https://www.csis.org/analysis/choking-chinas-access-future-ai
“American technology boosts China’s hypersonic missile program”
“Analysis: China faces its “Sputnik” moment as US export curbs deal a blow to its chip ambitions”, Reuters
“China and USA Are Officially At Economic War – Technology Restriction Overview: New regulations will impact global trade by hundreds of billions of dollars annually”, Dylan Patel (SemiAnalysis)
“China Summons Chip Firms for Emergency Talks After US Curbs”, Bloomberg (emphasis added):
“TSMC Suspends Work for Chinese Chip Startup Amid US Curbs”; “China’s Largest GPU Developer Biren Slashes Headcount [by 33%] Due to U.S. Sanctions...as TSMC halts shipments” (particularly striking as this is coming well after the big meeting with the ministry and presumably Biren would know about any planned subsidies/relief, and this is only the start… it seems doubtful there are any competitors who will be hiring them, rather than firing their own, under the circumstances—will there soon be a lot of Shanghai taxi drivers peculiarly knowledgeable about CUDA programming?)
“China’s Xi Says Willing to Work With United States for Mutual Benefit”; “Top China Envoy Lashes Out at US Export Curbs in Blinken Call”
“Shares in Chinese Companies Crash After Xi Jinping Stacks Party With Allies” (tech especially)
“TSMC Cuts Down Orders By Up to 50%, Sending Shockwaves Says Report”
“US Ban on Americans Aiding China Chip Firms Narrower Than Feared”
“Will Sanctions Against Russia End the War in Ukraine? D.C. bureaucrats have worked stealthily with allies to open a financial front against Putin.” (not directly China-related but contains several examples of how chip sanctions gradually bite over time: it doesn’t make headlines, factories just go idle and then quietly shut down)
“Observers: China’s Chip Talent Hurdle Worsens After Layoffs at US Firm Marvell”
“US-China Chip War with the Chip Avengers: “It takes a thousand steps to make a semiconductor and you’re going to have to get them all right.”″, Jordan Schneider & Irene Zhang (Schneider denies that the party congress timing was deliberate, incidentally; YMMV on whether that makes the ‘giant middle finger to the Congress’ better or worse.)
“AliBaba Chinese 128-Core CPU World Record Expelled From Rankings: No Availability; Biden’s sanctions work?”
“Tech war: Nvidia offers new GPU chip [‘A800’] tailored for Chinese market as it vows to comply with US export regulations” (Reuters; gimped to 400GB/s interconnect, below the A100 600GB/s)
“US seeks to pressure allies’ chip gear makers to join export control”
“Chinese chip designers [AliBaba & Biren] slow down processors to dodge US sanctions” (specifically, gimping interconnect in the hopes that TSMC will agree to fab them)
“TSMC 7nm process capacity utilization falling rapidly”
“China’s top chip maker SMIC warns on negative impact from US export controls after posting flat third-quarter revenue”
“China’s chip executives brace for winter as US sanctions push country’s semiconductor industry to the brink of desperation”, SMCP (notable points: unusually strong language in the title; VC investment has halted; orders have fallen substantially, such as −20%; chip business failures already exceed 2021′s total; SMIC has halved investments; one VC thinks startups will need at least 18 months of cashflow in the bank to survive; there’s some hype/happy-talk about how maybe they can just pivot to photonics out of semiconductor entirely; and there is zero discussion of CCP government bailout/support.)
“A Dangerous Game Over Taiwan”
“Engineers From Taiwan Bolstered China’s Chip Industry. Now They’re Leaving.”
“US bans may not be loosened for next 5 years, which benefits Korean memory brands, says Silicon Motion”
See also: for historical background on China’s previous failed efforts to create a autarkic domestic chip industry, see ‘The Sour Past of “China Chips”’
“China’s Silicon Future: China dreams of competing with global superpowers in the semiconductor industry. Whether its efforts will succeed is far from clear.”, Karson Elmgren (CSET)
“Beijing allows US export-control checks on Chinese tech companies: Biden administration says China’s commerce ministry has allowed inspections ahead of trade blacklisting deadline” (“Desperate times calls for desperate measures.”)
Looks like they’re knuckling under.
“Japanese tech leaders warn Beijing will ride out US chip sanctions: Sony and NEC executives say progress may be slowed but China will remain a force in AI and other areas”
“US targets China’s potential chip stars with new restrictions: Companies added to trade blacklist previously flew under the radar”:
(BGI Genomics also got blacklisted, not that anyone particularly cares about Chinese genetics at this point.)
* One might think it’d be crazy to try to trigger this sort of systemic crisis in the middle of a global, and Chinese, economic crisis. But of course, like chemotherapy, the question isn’t whether it’s bad for you, but worse for the other guy, and potentially pushing the chip ecosystem into a systemic collapse will never be easier than it is today. The USA is much wealthier than China, it can handle high-priced chips better.
† Presumably this extends to updates, upgrades, replacement of consumables, repairs when things break, replacement of broken machines… (How’s Russian military manufacturing & aviation going these days?)
‡ What an utter insult to Xi Jingping, incidentally. He must be furious to have this drop literally days before. Brother Pooh is not noted for his thick skin. I wouldn’t be surprised if whoever is masterminding this in the Biden administration timed it deliberately—after all, given that it’s been several weeks since the TSMC GPU embargo was announced, they could easily have delayed it to this week & after.
So, it’s been about a month and a week. Seems like enough time to evaluate a little the link dump above (which is in semi-chronological order). Where is this chip embargo now? I’d summarize the news & expert opinion thus far as:
The embargo is still solid.
The Biden administration has not walked it back, and confirmed the more restrictive parts. I have not seen any coverage indicating that Chinese corps are trivially circumventing it, both ASML & TSMC seem to be enforcing it, and Chinese corps are biting the bullet in deliberately gimping their chip designs to comply with it. Further, major players like Apple have been canceling equally major orders. None of this would be happening if it were only on paper or could be easily circumvented with a shell corp or something.
The consequences for the Chinese chip industry are still big, and bad.
We have plenty of reports about major layoffs, large hits to revenue, cutbacks to investments/R&D, exodus of US/Taiwan-linked personnel, and a halt to VC investment. Quotes from insiders like VCs or major chip manufacturer representatives range from ‘dire’ to ‘apocalyptic’, with time-ranges in the years for when—hopefully!---things might be better again. (Much less informatively: My previous comment got some circulation on social media, and mockery aside, there weren’t any comments I saw that looked like good arguments for why the impact would be minimal, rather than vague assertions that they would just somehow be fine even if they couldn’t get any ASML gear etc. I also asked anyone who might know something on my recent SF trip why this might not be a big deal, and got nothing, and overall an impression that everyone has been too distracted by the numerous other things happening of late to really process it.)
How big? It’s unclear because these aren’t great times for the global semiconductor industry either, as they are running into the general economic malaise and the bullwhip effect where the excess COVID-induced demand & past supply shortages are fading out—but it’s worth noting that it seems like non-CCP firms like Nvidia & TSMC were well-aware that they were going to overshoot to some degree and prepared for it, and don’t seem to be in nearly as bad shape overall.
But there does not (!) seem to be any massive CCP bailout of the Chinese chip industry planned.
While there are many fiscal stimuli ongoing, including ones announced since the embargo, chip-specific ones have not been announced—as would be necessary both to coordinate and restore confidence in the ecosystem—and the reported layoffs/cutbacks are highly costly mistakes if you expect a big faucet of billions of dollars of free government money to be turned on any month now, so seem to imply that the post-embargo meetings with the government did not spur a bailout effort. If Bloomberg’s reporting is correct (and I have no problem believing that a meeting attended by that many figures had at least one person willing to recount it all near-verbatim to a Bloomberg journalist), then they already collectively told the CCP that they were ‘doomed’ without massive additional investment, and the CCP appears to’ve shrugged and told them that ‘domestic IT market demand was adequate’ for them to survive.
The prospect of a collapse, beyond merely a hard recession, remains unclear, and will be hard to evaluate.
It may be tempting to say that “well, it’s been a month and while they’ve had some painful blows they are clearly still fine”. But that’s never how systemic collapses happen—remember bubbles like the Japan bubble, the US housing bubble, fracking, the repeatedly-averted Chinese bubble popping etc, or consider cryptocurrencies right now: the industry seemed to have weathered its most recent bubble popping with surprisingly few casualties compared to the historical fallout of each cryptocurrency bubble, in part due to bailout purchases/investments by sterling household brands like FTX… and then literally overnight that fell apart, and many individuals at many entities received unwelcome surprises about what connections there were in the cryptocurrency ecosystem. “There is a great deal of ruin in a nation.”
Almost all entities involved still have runway: I mean, if you were so fragile that you had less than 1 month of expenses (in the worst case of abruptly going to zero cashflow) and could have gone bankrupt already, then you were already doomed, embargo or no embargo. It is just very little time, on an industry-wide scale. Zombie companies can stagger on for a long time before finally going bankrupt. (As the quote goes: “slowly, then suddenly.”) Cash has not run out, reality has not set in, optimism remains high, supply stockpiles are only partially depleted, complex machines have not yet broken down or reached the end of maintenance cycles or expected lifetimes, slashed orders mean capacity losses are less important… Similarly, when Putin invaded Ukraine 275 days ago, they were extensively embargoed, particularly on chips, and there have not been any dramatic consequences with screaming headlines about passenger planes falling out of the sky—but there have been consequences, such as the gradual disappearance of all their good high precision missiles/artillery like cruise missiles, the choking off supplies to their front lines, resorting to cannibalizing many units to get one working unit or using very expensive equipment in incredibly wasteful ways (hypersonic missiles on apartment blocks, anti-ship missiles on land targets, S-300 AAs as ghetto cruise missiles), obsolete equipment & reliance on imports like Iranian drones, and factories you’ve never heard of simply dying for lack of chips & other supplies because they were unable to work around so many dependencies lopped off all at once. It has taken many months for subtle signs to show up of real damage, and indeed, many people early on were quite skeptical any Russian embargoes could do any good (surely they will just import it from China---!).
Points #1/2/4 are no surprise but point #3 is a surprise. I think pretty much everyone took for granted that the basic CCP response would be to double down on all the chip subsidies, and if they’d already dumped in $100b, oh well, now they’d dump in $200b (in for a penny, in for a pound). This seems… not to have happened? That’s surprising. At least, I’m surprised. If you aren’t surprised, why not? Is there a bailout somewhere I missed in the news?
So, let’s take as a hypothesis that there is no bailout for chips, especially on a large enough scale to really matter. Why this neglect?
Let’s take further as a hypothesis that the reason for the neglect is not simply Xi deciding to invade Taiwan & therefore writing off the domestic chip industry, as this is an extreme course of action and one that most people claim to find even more improbable than, say, the idea that a Xi-like dictator like Putin would do something as absurdly self-destructive as actually invade Ukraine this year instead of just saber-rattling his massed troops along the border to intimidate them into concessions.
The remaining conclusion would seem to be that Xi has chosen to take the L: he is neither going to massively bail out the domestic chip industry nor take out their competitor, and is just going to let it take its lumps and whatever happens happens, China will just have to get along with whatever chips it can make and the gimped chips TSMC will deign to manufacture for it. (At least, I can’t think of any additional meaningful choice outside the trilemma of ‘bailout, accept defeat, or invasion’.) OK, but why? I struggled to see how that could make sense. Let’s go even further into the hinterlands of geopolitical & psychological amateur speculation and ask what Xi is thinking...
From the scaling-pilled perspective, or even just centrist AI perspective, this is an insane position: it is taking a L on one of, if not the most, important future technological capabilities, which in the long run may win or lose wars. If China wants to dominate Asia, much less surpass the obsolete American empire, or create AGI, or lead in aerospace, or create ‘5G’ or whatever, it’s hard to see how it’s going to do that while paying more for chips which are half a decade or worse out of date.
But Xi is not scaling-pilled (after all, few people are, even in the most cutting-edge AI R&D labs). So maybe we should ask: is he centrist on AI? Er… Oh—does he care about AI at all? What evidence is there that he does? There doesn’t seem to be much. Going further: what evidence is there that he even regards chips in toto as being all that important? From Xi’s perspective, what has ‘chipolitics’ looked like thus far?
Huawei: the US embargo of chips to the national champion Huawei, and Huawei’s near-death experience, is where chiplomacy started getting ugly. This involved ordinary bulk consumer chips for smartphones and equipment like 5G base stations. It did not involve high-end GPUs or future chips at all. Huawei simply needed millions of chips to sell for export of hundreds of millions of smartphones to the entire world like Africa, and couldn’t get them. But, they survived, and they may now struggle to get the chips that they would like and rely on alternate suppliers of lower-end chips (any dreams of challenging Apple on its high-end home turf are long gone), but Huawei as a whole does still sell a ton.
Russia: another embargo post-Ukraine, cutting off supply of all sorts of chips, almost all antiquated chips designed decades ago for specialized equipment: again, nothing like a H100 GPU or in any way connected to stuff like ‘4nm nodes’. Even for drones, you can usually get by with pretty old parts or improvising; it’s more important to have lots of cheap drones than geewhiz. Here too the problem is Russia needs (1) millions upon millions of specific chips to feed into existing manufacturing lines to feed the meatgrinder of its ‘eastern front’ (if you will), (2) needs them very soon, preferably a few months ago, as the Ukrainian assault will not stop and they have burned through much of their reserves, and (3) cannot get millions of those specific chips whether a few months ago or a few months from now.
The current US/TSMC chip ban: little of the damage reported thus far has much to do with failing to develop new nodes or not getting access to A100/H100s. Nvidia canceled a lot of orders of them, but I haven’t seen anything report about big corporations going out of business etc, and Chinese AI research seems more or less to be carrying on as before with its current stock. The damage is coming from losing big bulk orders like dumping memory to Apple and from catchup designs not being fabbed and suppliers of existing stuff being knocked out. As time passes, these ‘seen’ damages will be replaced by much larger ‘unseen’ damages cascading out.
Even from an AI research perspective, the damage will be subtle. You can keep on using existing clusters indefinitely; they will just put a ceiling on what you can do with reasonably trained or sized models before the communications delays kill performance, and they will hamstring your budget by costing several times more over the next few years, increasingly so. You can assemble clusters 1 GPU at a time, buying them on the gray market, eventually, years later, reaching a respectable size. You can keep doing AI research which looks like almost all AI research in 2021*, and which will look respectable, and use hand-me-down public models from the West, and do lots of small-scale optimization work (which will be, or has already been, Bitter-Lessoned but that is cold comfort to those who can’t afford the bigger irons). You can keep pumping out CS conference proceedings with low-grade small-budget research that would barely make a decent blog post. (I read a lot of IEEE papers relating to anime research, typically GANs, typically terrible, typically East Asian, and typically a few GPUs max.)
A chip ban doesn’t doom Chinese DL (or Chinese AI in general) research to becoming a cargo cult field, but it does render it increasingly prone to irrelevancy, brain drain of everyone who wants to work on the future of DL and not the past, involution / l’art pour l’art, and organizational/intellectual sunk costs and Galapagos syndrome. Considering that Chinese R&D and science are not particularly healthy or fraud-free in the best of times… But how would an outsider, such as a political leader, notice that the cargo is no longer landing?
* I realize people like to portray AI scaling as some overwhelmingly dominant paradigm. This isn’t true. You can go look at a page of NIPS abstracts and see that that is not the case, or look at the NLP survey the other month and note how few people will endorse scaling propositions on a mere anonymous survey, or note how few Arxiv submissions a day merit a /r/MLscaling submission despite spreading a broad net. AI scaling is far from the majority of AI research; scaling research is merely the majority of research that will matter.
We could add a few other points:
the senior CCP leadership is semi-famous for being ‘technical’ (typically engineering degrees like hydrology or mechanics or aerospace) but little to do with anything computer. Xi Jinping has a degree in chemical engineering from a low-rigor period 43 years ago, and then a degree in BS, both of which might just be mostly fake (pretty common). Propaganda aside, his major intellectual interest is literature, particularly Goethe. He has not overseen any major technical projects, or made any major intellectual contributions I’m aware of; indeed, reading about him, he’s always struck me as being mediocre in every aspect besides Comunist Party infighting, bureaucracy, and consolidating power.
Techlash: Xi’s reign has been marked by constant hostility towards and suppression of ‘big tech’ ie. the very people who would benefit and use cutting-edge chips the most and who would be explaining to Party officials the long-term prospects. There are innumerable angles here (for example, yesterday, it sounds like the CCP may deign to graciously allow a few video game developers to, after a year or two of it simply not being allowed, ‘release video games’), and suppression of individuals like Jack Ma come off as very personal. The rhetoric of the regime emphasizes redistribution, only thinly veiled as ‘voluntary donations’, and the Party defending the public and the ‘China Dream’ from rapacious corporations.
Conversely, his reign has been marked by an emphasis on legible atom-heavy scientific projects, and a general downplaying of everything related to bits or information, unless it has a national security angle (leading to ‘Dutch disease’ where an ultra-niche like facial recognition gets lavishly funded, crowding out more generalizable research). For all the talk of ‘data is the new oil’ or ‘China as a data superpower’ or the advantages from ‘Chinese lack of privacy’, China still drastically underperforms in making good use of it. There are large GPU clusters; but all the really important DL research still comes from the West—I’ve noted that it seems like you could trade all of Chinese DL research impact for 1 or 2 Western labs like DeepMind’s impact and still have enough change left over for coffee. There are large numbers of genetic datasets fragmented over many Chinese groups; but global genetics research remains driven by UK Biobank from 2014 etc.
Attempts at scientific reform and quality control are on uneven ground. Fraud in Chinese science seems to still be endemic, and impact low; past efforts like paying per publication merely made the problem worse. In ML, when fraudulent rings of peer review are discovered, they often involve Chinese conferences or reviewers. Publication in prestigious foreign journals like Science or Nature (rather than Chinese journals) remains a major goal, as a matter of quality control, because domestic Chinese journals cannot be trusted.
There seems to be considerable contempt for the USA and American capabilities in China among ‘wolf warriors’, taking cues from the top, and with considerable historical precendent for authoritarian countries to mistakenly gauge the USA as ‘decadent’ and ‘weak’. This may have been trimmed a bit after Ukraine and seeing what things like HIMARSs can do, but it runs deep and inside the Chinese bubble, there is little correction. (When was the last time Xi Jinping was in the USA and saw more than political flunkies? Or any Chinese, for that matter, given their multi-year near-shutdown of international travel?) The thinness of the air at the heights Emperor Xi inhabits is prone to induce altitude sickness and hallucinations. (But at least, thanks to “Zero COVID”, among his many problems, personally getting COVID is not one of them.) If the Americans are decadent because of their emphasis on software and compute, and China & Xi are superior because they aren’t decadent...
The many stimuli the CCP has used repeatedly over the past decade, and since the chip embargo was put in place, shows that they can and will and have had enough time to do so, yes, but the flip side is that the more you stimulate, the less ammunition you have left for the next stimulus, and the less credibility you have to the populace or markets. At some point your macroeconomics start looking bad. There may not be as much money left as one would assume.
So, in past chip incidents, the primary problem has been a complete absence of any chips, and not so much the advancement of the chips themselves. He has never seen anyone lose a war due to lack of AI or GPUs; he’s only seen disasters caused by lacking perfectly ordinary chips that his domestic manufacturers probably could’ve made 10 years ago. And in learning lessons from past chip incidents, what Xi brings to the table is: zero technical competence or expertise in the relevant area, a hatred of software and everything to do with it, and long-standing prioritization of heavy-industry-like stuff (which is clearly visible to the naked eye and ‘conventional’ and ‘prestigious’ and applauded by old credentialed foreigners).
Further, mistakes in this regard may be hard to see. ‘The seen and the unseen’ is a dangerous trap because it is so much easier to see the seen than it is to see the unseen. If Xi makes a mistake on chips, a military mistake, then by the nature of things military, he may never realize it. If the engineers of, say, hypersonic missiles can’t get enough high-end GPUs, their complaints will be ignored by the next layer of management and never punted all the way up to Beijing, and they will simply run their simulations at a lower resolution or take other shortcuts; and if the hypersonic missile in question turns out to be a lemon, inadequate to hit NATO units or US aircraft carriers, how will anyone ever find out short of a war over Taiwan—at which point it is far too late? Naturally, of course, given a supply of at least basic chips to work with, the establishment will assure him everything is fine, just like the Russian military assured Putin it was not a paper tiger or hopelessly undermined by corruption, and almost all the time they will be right.
That is, from Xi’s perspective, all in all, maybe it looks fairly reasonable to neglect chips right now. They aren’t that important, and don’t seem in that much worse trouble than anything else, while bailing them out to the degree where they can potentially gain, or at least near, the cutting-edge would use up a ton of an increasingly skint government’s money. Plus, as master of the currents of history piloting China to a glorious Chinese Century avenging the Century of Humiliations, he has much bigger fish to fry, like the house of cards which is real estate, and Zero COVID. There will be side effects, yes, but if gaming GPUs becoming expensive helps turn little Aiguo away from a career as a useless game programmer into a respectable hardworking fusion physicist, perhaps that’s even a feature rather than a bug?
Well, I could be wrong about all this. But now I can see at least one perspective from which the chip embargo is a big deal but also Xi’s rational response is to indeed just take it on the chin, and perhaps tone down the rhetoric and engage in a bit more biding one’s time & hiding one’s strength. (I doubt that the long-term aims have changed meaningfully just because Beijing is calibrating its rhetoric a little down from recent peaks of aggression, but in the short term, things will be superficially more peaceful.)
The scaling-pilled AI view ought to be that scaling AI kills you. Why pretend that there’s a strategic advantage here, as opposed to a loaded gun you can point at your own head if you’re stupid enough?
It’s one thing to say “given China’s actual beliefs, they ought to do X” or “if China were rationally acting on a correct understanding of the world, they would do Y”. But why criticize China for avoiding a self-destructive action that would make sense to do if they had a specific combination of definitely-true, maybe-true, and definitely-false beliefs—a specific combination they don’t in fact have?
Isn’t the “scaling AI kills you” view the conjunction of “scaling-pilled” and “alignment is extremely difficult” views, rather than being identical with the scaling-pilled view?
One could reason as something like:
If alignment is as hard as people make it out to be, we’re in all likelihood dead anyway since Westerners are going to develop AI even if we don’t.
If alignment isn’t as hard as people make it out to be, then the country that controls the most powerful AI will be the one that becomes dominant in the world.
Thus if alignment is hard it doesn’t matter what we do, and if alignment is less hard we should invest in AI. Thus, we should invest in AI.
(There’s some obvious nuance that this argument is missing, e.g. the chance of arms races increasing the difficulty of alignment, but some form of it still seems reasonable to me.)
Are you suggesting that the sane policy is for Xi to dump in as much subsidies as needed until China catches up in semiconductors with the US and its allies? I haven’t seen anyone else argue this, and it seems implausible to me, given that the latter collectively has much greater financial and scientific/engineering resources. China’s GDP is only $18T vs $58T for OECD, and as you say there’s a lot more corruption/fraud in Chinese R&D, so how can they hope to win an outright tech race (and starting from about a decade behind)?
Yes. And perhaps no one else does because they aren’t scaling proponents. But from a scaling perspective, accepting a permanent straitjacket around GPUs & a tightened noose is tantamount to admitting defeat & abandoning the future to other countries; it’d be like expelling all your Jewish scientists in 1935 & banning the mining of uranium. It’s not the beginning of any story that ends in victory & being #1, only of stories that end with you being #10, or #100. Like the Manhattan Project, you pay whatever it costs, and it costs what it costs.
Well, it’s certainly not easy. It’s not a great situation for China to be in, and yet, it is in fact the one they are in, and they have to deal with it. Reality has no obligation to make anything easy for you, nor should you expect something like ‘dethrone the global hyperpower and create a new world order around the Middle Kingdom’ to be easy. Similarly, it’s not easy to defeat a Russian or Chinese invasion, nor was it easy to develop atomic bombs, etc, but if you don’t, there will be consequences you may find unacceptable—even, existential, one might say—so you don’t get much of a choice. You miss 100% of the shots you don’t take.
China has a lot of resources yet untapped, both financial and otherwise, and is doing many other things like stimulating other sectors of the economy, so clearly it can try more than it is right now—but it appears not to be. If it’s not a capability issue, then it must be a choice. Xi seems to disagree that the consequences of choosing to accept defeat in chips & AI will be all that unacceptable, and my comment here is about thinking through what the strategic logic could be from his perspective which makes that choice an acceptable one because the the chip race game is not worth the candle.
I agree with Rob Bensinger’s response here, plus it’s just a really weird use of “insane”, like saying that Japan would have been insane not to attack Pearl Harbor after the US imposed an oil embargo on them, because “You miss 100% of the shots you don’t take.” Thinking that way only makes sense if becoming a world or regional hegemon was your one and only goal, but how did that become the standard for sanity of world leaders around here?
It’s not a weird use. It is a completely normal one: becoming the hegemon is the avowed goal and so making choices which drop the odds of success so drastically raise questions about the thought process, which in this case I find much better explained by simply mistaken beliefs and desires on Xi’s part combined with ordinary sanity. I have already explained at length why I think those beliefs are mistaken, and I also think the desires are bad: do I think China (in the sense of its population as a whole) is better off for Xi’s powerhunger, or that China would be better off for launching a scaling Manhattan Project? Certainly not: in my opinion, Xi’s reign has made the Chinese people substantially worse off than they would have been under a more status quo leader, and Xi has probably single-handedly curtailed their long-term growth prospects & condemned them to permanent middle income status as they begin to enter a Japan-style malaise, in addition to authoritarian disasters like the Uighurs. But many Chinese believe otherwise now, and endorse further questing for hegemony, and they & Xi are in charge, not you, and desire what they desire, not what you desire—you (and Rob) are projecting your own preferences and moralizing here, instead of trying to understand what is going on.
I wouldn’t say “drastically” here so maybe this is the crux. I think the chances of success if China does make an all out push for semiconductors is very low given its own resources and likely US and allies’ responses (e.g. they could collectively way outspend China on their own subsidies). I could express this as <1% chance of having a world leading semi fab 10 years from now and <5% chance 20 years from now, no matter what China chooses to do at this point. If hegemony was the only goal then even a 1% chance would be worth it, but “drastically” makes me think maybe that’s not what you’re saying. These are off the cuff numbers so I’m pretty open to changing my mind about them, but seem reasonable given what I know about China’s research capabilities and what it took for the world to reach its current level of semiconductor technology.
Separately from gwern’s argument, I say that maintaining the gap is still of vital national interest. As an example, one of the arguments in favor of nuclear testing bans is that it unilaterally favors American nuclear supremacy, because only the US has the computational resources to conduct simulations good enough to be used in engineering new weapons.
That logic was applied to Russia, but the same logic applies to China: advanced simulations are useful for almost every dimension of military competition. If they let advanced compute go, that means that the US will be multiple qualitative generations ahead in terms of our ability to simulate, predict, and test-without-risk.
This is a terrible position to be in, geopolitically.
Thanks for really interesting discussion and summary of the state of play. I’m curious what you think of Ben Thompson’s point that China could have an advantage at the “trailing edge”(28-45nm, and 45+) if TSMC were taken off the board?
https://stratechery.com/2022/chips-and-china/
Maybe the play is to focus on the trailing edge, which is easier to take control of, but still v important in the short-term at least, and temporarily let go of the cutting edge?
Whenever I try and think about Xi’s actions as rational I get hung up on the neverending Zero COVID. Many genuinely think it’s mostly about saving face but even if I try hard I can’t see how it could look anything but childish. They must have convinced themselves it’s actually a good policy. I could at least understand that!
I wonder if given the COVID and real estate crises, Xi’s government just doesn’t have the financial resources to bail out the chips industry, plus maybe they (correctly?) understand that the likelihood of building an internationally competitive chips industry is poor (given the sanctions) even if they do dump in another $200b?
Also, it seems like China is being less antagonistic towards Taiwan and other countries in the last few days. Together with the lack of chips bailout, maybe it means they’ve realized that it was too early to “go loud” and are pivoting back to a softer, more cooperative strategy.
“China is working on a more than 1 trillion yuan ($143 billion) support package for its semiconductor industry.”
“The majority of the financial assistance would be used to subsidise the purchases of domestic semiconductor equipment by Chinese firms, mainly semiconductor fabrication plants, or fabs, they said.”
“Such companies would be entitled to a 20% subsidy on the cost of purchases, the three sources said.”
https://www.reuters.com/technology/china-plans-over-143-bln-push-boost-domestic-chips-compete-with-us-sources-2022-12-13/
My impression is this is too little, too late. Does it change any of your forecasts or analysis?
I have a few questions:
What would you do if all of your investments were stuck in the Chinese market? Puts on semi-conductor-reliant industries?
How much benefit can Chinese firms get from ignoring NVIDIA GPU ML training restrictions?
Would it be possible to post a few sources?
So is this a good thing for AI x-risk? Do you support a broader policy of the US crippling Chinese AI?
Uh… it’s either a very good thing or bad thing. As the Chinese quote goes, it is too soon to tell. If I had to come down on one side, right now, I think I would come down on ‘good’; slowing down Chinese AI, which would by default hand AGI to Xi & generally pays even less attention to safety than everyone else does, is good, and while ‘CCP destroys TSMC’ is far from the ideal approach to restricting compute growth, it is at least going to make a difference—unlike almost all other proposals. The upfront cost in economics, human welfare, peace, rules-based global order etc is going to be exorbitant, however. In my lifetime, wars have not had a good track record of producing solutions at low costs. (The situation reminds me of nukes, Seoul, or EU dependence on Russian gas: it could probably have been prevented cheaply early on with the stroke of a pen, but several decades later...)
Even if TSMC is destroyed, Samsung has acquired much of TSMC’s hard-earned chip-fab/experience curves/tacit knowledge, so scaling in the US should not be set back that much.
(we will have Liang Mong-Song entirely to thank)
Oh, crap, I work in Chinese genetics. If Illumina and other suppliers embargo China, my job will go tits up. Any idea how much BGI Genomics will be affected? I was also considering jumping ship to UAV development, so this has been a really bad month. No one I work with knew any of this was happening when I brought it up earlier this week.
BGI Genomics, like the mothership in mainland China proper? I’m less familiar with the corporate structure and revenue and dependency on overseas supply chains than I would hope you are, but my understanding was that BGI didn’t have much business overseas (having failed to compete with Illumina), and was reliant on domestic demand mostly from agriculture & medicine, and didn’t depend on Illumina having spent the past decade+ trying and mostly failing to surpass Illumina (but at least it does have its own decent sequencers). So, having failed in those ways and forced into autarky, it doesn’t seem like sanctions/embargos can hurt BGI much more than it already has been? And macroeconomics-wise, sequencing seems like it would be reasonably robust a business because farmers won’t stop needing genetics-related services nor will patients stop getting sick. And politics-wise, I see no particular Xi-techlash angle for him doing things like overnight outlawing the industry or censors just refusing to approve any video game release for a year. So overall, doesn’t seem too bad.
Honestly, UAV seems like it’d be a worse place to go simply because that sounds to me like it’ll be more disrupted by random chip problems and export issues due to being extremely military-linked dual-use tech. Drones are stuffed full of all sorts of random weird little chips (see: Russian problems getting UAVs and resorts like getting them from Iran), and how much domestic Chinese demand for UAVs could there possibly be to make up for exports?
Thanks, I was just really worried because our entire sequencing pipeline uses Illumina products. But I asked around our sequencing division, and they think the difference between using Illumina and BGI products isn’t too big—what BGI lacks in quality it makes up for in lower costs. Apparently, the difference in Q30 (% of reads with errors <0.1% of bases) is ~90-95 for Illumina and 80 for BGI, which is marginal. Switching wouldn’t be a major problem according to them. BGI also uses Chinese chipsets, which means sanctions aren’t going to impact it much. I don’t think this is going to be as bad as I first thought.
How likely do you think China getting cut off from Illumina is? Do you think consumer GPUs are going to be restricted?
Yes, that was my impression. BGI sequencing is not as good as Illumina, but it’s not like it’d destroy them to switch over. And sequencers don’t use any really high-end GPUs (even if you’d like to have them for bioinformatics), so it’s not like a chip embargo is an immediate halt-production problem the way it is for building a new supercomputer or building cars stuffed full of miscellaneous chips
Based on a plain reading, consumer GPUs already are restricted: it’s not a ban on A100s/H100s, it’s a ban on any system as powerful as an A100, to quote Nvidia:
This is a static threshold, which makes no allowance for Moore’s law. It is an upheld hand: “Thus far, and no further!” And it is a threshold which may be biting already. EDIT: it looks like they are going to grandfather in consumer GPUs, ignoring their TFLOPs and focusing only on the interconnect bandwidth+latency, where I think all Nvidia consumer GPUs < A100. This may be a bad bet: people have had not much incentive to figure out how to yoke together large numbers of consumer GPUs (as most people using >1 consumer GPU are either hobbyists who generally can stick all the GPUs they can afford into a single box, or cryptominers for whom interconnect is irrelevant because the PoW mining is by design embarassingly-parallel), so if Chinese hyperscalers or major AI labs can purchase all the 4090s they want, now there will be vastly more incentive to figure out hardware hacks (will, say, TSMC ban Chinese designs for interconnect chips which do no computation...?) or low-communication strategies (even large constant factor penalties, which would render them useless to Western groups, will eventually be worthwhile to Chinese groups barred from >=A100 GPUs.)
What is the ‘peak performance’ roughly equivalent to an A100? Well, Nvidia’s product page tells me an A100 delivers 19 FP32 TFLOPs, where FP32 is probably a conservative number. (I don’t think there are very many cases in DL where you would still need FP64, and while everyone is trying to move to FP16 and lower, where an A100 does 312 TFLOPS instead, you still can’t always easily or reliably train every arch with FP/BF16 and you can encounter some severe problems especially with the important case of large Transformers.) 19 TFLOPS may sound like a lot… but the top-end gaming GPU Nvidia RTX 4090 goes as high as 86 TFLOPS (specced at 82 TFLOPS, with GeForce RTX 3090 Ti at 40 TFLOPS and GeForce RTX 2080 Ti at 14 TFLOPS).
(The H100, incidentally, is 67 TFLOPS.)
So, if USG is banning ‘peak performance roughly equivalent to A100’, and gaming GPUs are turning in TFLOPS several times that of the A100, then the obvious interpretation would seem to be that yeah, lots of consumer GPUs are already restricted. All the way back to like 2019 GPUs, possibly.
(I was really surprised by this when I went to check the numbers; I did not think that gaming GPUs were that much faster than the A100. I knew that the A100 was already kinda obsolete with the H100 in the pipeline, that there was a ‘datacenter tax’ and ‘enterprise tax’, and also that everyone has kinda understated GPU progress because they were so hard to get for so long and most people have been trucking along with GPUs years out of date—even the V100 still shows up in papers occasionally—etc, but I didn’t think it would be that much slower, like 4x. Either I am badly misunderstanding units here, or I am greatly overrating A100s & underrating recent gaming GPUs like the absurdly large 4090, or I am making a mistake in looking at FP32 because Nvidia put all of the A100 advantages into lower precisions.)
https://twitter.com/paul_scharre/status/1585684356819046400
Why are gaming GPUs faster than ML GPUs? Are the two somehow adapted to their special purposes, or should ML people just be using gaming GPUs?
>Why are gaming GPUs faster than ML GPUs? Are the two somehow adapted to their special purposes, or should ML people just be using gaming GPUs?
They aren’t really that much faster, they are basically the same chips. It’s just that the pro version is 4X as expensive. It’s mostly a datacenter tax. The gaming GPUs do generally run way hotter and power hungry, especially boosting higher, and this puts them ahead against the equivalent ML GPUs in some scenarios.
Price difference is not only a tax though—the ML GPUs do have differences but it usually swings things by 10 to 30 percent, occasionally more. Additionally the pro versions typically have 2X-4X the GPU memory which is a huge qualitative difference in capability, and they are physically smaller and run cooler so you can put a bunch of them inside one server and link them together with high speed NVLink cables into configurations that aren’t practical with 3090s. 3090s have a single NVLink port. Pro cards have three. 4090s curiously have zero—NVIDIA likely trying to stop the trend of using cheap gaming GPUs for research. Also, the ML GPUs for last gen were also on a a 7nm TSMC process, while the gaming GPUs were on Samsung 8nm process. This means the A100 using 250 watts outperforms the 3090 using 400 watts. But they are overall the same chip.
None of that accounts for a 4X or more cost multiplier, and the TLDR is the chips are not that different. If gaming GPUs came in higher memory configurations, and all supported NVLink, and were legally allowed to be sold in datacenters, nobody would pay the cost multiplier.
I just realized that H100s are still available from online vendors for ~189000 yuan (~$27000), which is international market price.
Well then. Time to cash out my savings and make some money. Do you guys think it’s feasible? How much are Chinese H100 prices likely to rise? Should I be trying to scoop up high-end GPUs instead?
Update: local computer shop says international GPU suppliers are still accepting Chinese orders and Chinese GPU prices are stable for now.
China’s top chipmaker, Semiconductor Manufacturing International Corporation, built a 7nm chip in July of this year. This was a big improvement on their previous best of 14nm and apparently surpassed expectations, though is still well behind IBM’s 2nm chip and other similarly small ones. This was particularly surprising given that the Trump administration blacklisted SMIC two years ago, meaning for years they’ve been subject to similar restrictions as Biden recently imposed on all of China. Tough to put this in the right context, but we should follow progress of China’s chipmakers going forwards.