Vladimir_Nesov

Karma: 32,261

Vladimir_Nesov Apr 4, 2025, 12:39 AM
2 points
0
in reply to: Cole Wyeth’s comment on: AI 2027: What Superintelligence Looks Like
For me a specific crux is scaling laws of R1-like training, what happens when you try to do much more of it, which inputs to this process become important constraints and how much they matter. This working out was extensively brandished but not yet described quantitatively, all the reproductions of long reasoning training only had one iteration on top of some pretrained model, even o3 isn’t currently known to be based on the same pretrained model as o1.

The AI 2027 story heavily leans into RL training taking off promptly, and it’s possible they are resonating with some insider rumors grounded in reality, but from my point of view it’s too early to tell. I guess in a few months to a year there should be enough public data to tell something, but then again a quantitative model of scaling for MoE (compared to dense) was only published in Jan 2025, even though MoE was already key to original GPT-4 trained in 2022.

Vladimir_Nesov Apr 3, 2025, 7:23 PM
28 points
0
on: AI 2027: What Superintelligence Looks Like
Non-Google models of late 2027 use Nvidia Rubin, but not yet Rubin Ultra. Rubin NVL144 racks have the same number of compute dies and chips as Blackwell NVL72 racks (change in the name is purely a marketing thing, they now count dies instead of chips). The compute dies are already almost reticle sized, can’t get bigger, but Rubin uses 3nm (~180M Tr/mm2) while Blackwell is 4nm (~130M Tr/mm2). So the number of transistors per rack goes up according to transistor density between 4nm and 3nm, by 1.4x, plus better energy efficiency enables higher clock speed, maybe another 1.4x, for the total of 2x in performance. The GTC 2025 announcement claimed 3.3x improvement for dense FP8, but based on the above argument it should still be only about 2x for the more transistor-hungry BF16 (comparing Blackwell and Rubin racks).

Abilene site of Stargate^[1] will probably have 400K-500K Blackwell chips in 2026, about 1 GW. Nvidia roadmap puts Rubin (VR200 NVL144) 1.5-2 years after Blackwell (GB200 NVL72), which is not yet in widespread use, but will get there soon. So the first models will start being trained on Rubin no earlier than late 2026, much more likely only in 2027, possibly even second half of 2027. Before that, it’s all Blackwell, and if it’s only 1 GW Blackwell training systems^[2] in 2026 for one AI company, shortly before 2x better Rubin comes out, then that’s the scale where Blackwell stops, awaiting Rubin and 2027. Which will only be built at scale a bit later still, similarly to how it’s only 100K chips in GB200 NVL72 racks in 2025 for what might be intended to be a single training system, and not yet 500K chips.

This predicts at most 1e28 BF16 FLOPs (2e28 FP8 FLOPs) models in late 2026 (trained on 2 GW of GB200/GB300 NVL72), and very unlikely more than 1e28-4e28 BF16 FLOPs models in late 2027 (1-4 GW Rubin datacenters in late 2026 to early 2027), though that’s alternatively 3e28-1e29 FP8 FLOPs given the FP8/BF16 performance ratio change with Rubin I’m expecting. Rubin Ultra is another big step ~1 year after Rubin, with 2x more compute dies per chip and 2x more chips per rack, so it’s a reason to plan pacing the scaling a bit rather than rushing it in 2026-2027. Such plans will make rushing it more difficult if there is suddenly a reason to do so, and 4 GW with non-Ultra Rubin seems a bit sudden.

So pretty similar to Agent 2 and Agent 4 at some points, keeping to the highest estimates, but with less compute than the plot suggests for months while the next generation of datacenters is being constructed (during the late 2026 to early 2027 Blackwell-Rubin gap).
1. ↩︎
  It wasn’t confirmed all of it goes to Stargate, only that Crusoe is building it on the same site as it did the first buildings that do go to Stargate.
2. ↩︎
  500K chips, 1M compute dies, 1.25M H100-equivalents, ~4e27 FLOPs for a model in BF16.

Technical Claims

Vladimir_NesovApr 3, 2025, 12:30 AM

14 points

0 comments1 min readLW link

Vladimir_Nesov Apr 2, 2025, 6:44 PM
5 points
2
on: How To Believe False Things
Beliefs held by others are a real phenomenon, so tracking them doesn’t give them unearned weight in attention, as long as they are not confused with someone else’s beliefs. You can even learn things specifically for the purpose of changing their simulated mind rather than your own (in whatever direction the winds of evidence happen to blow).

Vladimir_Nesov Apr 1, 2025, 2:24 PM
16 points
5
on: We’re not prepared for an AI market crash
The scale of training and R&D spending by AI companies can be reduced on short notice, while global inference buildout costs much more and needs years of use to pay for itself. So an AI slowdown mostly hurts clouds and makes compute cheap due to oversupply, which might be a wash for AI companies. Confusingly major AI companies are closely tied to cloud providers, but OpenAI is distancing itself from Microsoft, and Meta and xAI are not cloud providers, so wouldn’t suffer as much. In any case the tech giants will survive, it’s losing their favor that seems more likely to damage AI companies, making them no longer able to invest as much in R&D.

Vladimir_Nesov Mar 31, 2025, 9:59 PM
8 points
4
in reply to: funnyfranco’s comment on: On Downvotes, Cultural Fit, and Why I Won’t Be Posting Again
https://slatestarcodex.com/2014/07/30/meditations-on-moloch/

It’s “mainstream” here, described well many times before.

Vladimir_Nesov Mar 31, 2025, 9:23 PM
16 points
4
in reply to: Rafael Harth’s comment on: On Downvotes, Cultural Fit, and Why I Won’t Be Posting Again

if we didn’t have a capitalist system, then the entire point about profit motives, pride, and race dynamics wouldn’t apply

Presence of many nations without a central authority still contributes to race dynamics.

Vladimir_Nesov Mar 30, 2025, 10:27 AM
9 points
5
in reply to: Thane Ruthenis’s comment on: Thane Ruthenis’s Shortform
If o3 is based on GPT-4o, there is a reasoning model based on GPT-4.5 that’s better. If o3 is based on GPT-4.5 (and so the reason it’s still not out is that they are waiting for Blackwells to inference it at a reasonable speed and cost), then it was a hasty job just after the base model for GPT-4.5 was pretrained, and so by now they have something much better. Either way, there is some sort of “o4”, but it’s probably a bad look to announce it before releasing the already-announced o3.

Vladimir_Nesov Mar 29, 2025, 5:13 PM
1 point
0
in reply to: tomdekan’s comment on: AI Needs Us? Information Theory and Humans as data
Before life, there are only rocks and astronomical objects. Once new things can be created, prior world is relatively unimportant to understand in comparison, because it’s constrained to happenstance of what was there in the past, and there is no similar constraint on what can be created in the future.

Most interesting things are those that get intentionally created with the purpose of being interesting in mind. For any purpose, this or other, that doesn’t end up referencing humanity or the past, it’s possible to create more optimal things in view of that purpose than anything that already happens to exist, because things that happen to exist were never superintelligently optimized to fit that purpose. Humanity is like rocks and astronomical objects, relics that are not optimal in most respects.

Vladimir_Nesov Mar 29, 2025, 4:29 PM
5 points
0
on: AI Needs Us? Information Theory and Humans as data
The only way to know more digits of pi is to keep computing them, therefore a truly advanced AI would maintain increasingly giant computers that keep computing the digits because of its interest in maximal learning. This proves too much, doesn’t say anything about priorities. Humans are made of atoms AI could use to compute more digits of pi, which it must do because of its interest in maximal learning, and so on.

Vladimir_Nesov Mar 28, 2025, 5:31 PM
3 points
1
in reply to: Mis-Understandings’s comment on: Mis-Understandings’s Shortform
Hence “next token predictor” is a bit of a misnomer, as computation on any given token will also try to contribute to prediction of distant future tokens, not just the next one.

Vladimir_Nesov Mar 28, 2025, 4:30 PM
13 points
1
on: Gemini 2.5 is the New SoTA
Curious if it’s built on the same base model as Gemini 2.0 Pro, or on a completely new pretrained model. With 100K TPUv6e datacenters (about the same as 100K H100 in training compute), Gemini 2.0 Pro seems like it’s underperforming in its weight class, compared to GPT-4.5 and even Grok 3 (likely trained on similar compute). So it makes some sense they’d write it off as a failed run and do another one, but alternatively long reasoning post-training could’ve fixed enough to get a good reasoning model. In which case the name choice breaks the pattern of Gemini 2.0 Flash Thinking, but could be a way of distancing the success of Gemini 2.5 Pro (the reasoning model) from the mediocre performance of Gemini 2.0 Pro (the chat model).

Google’s TPUv6e systems have large scale-up world sizes, unlike Hopper systems. So there is a short window of a few months where they have the advantage in being able to more cheaply inference large reasoning models, unlike everyone else (unless they also use TPUs). Other AI companies would need access to Blackwell NVL36 or NVL72 in order to get reasonable cost and speed of inferencing large reasoning models, and it seems it’ll take another 2-5 months before they are out in force.

Vladimir_Nesov Mar 26, 2025, 6:28 PM
12 points
2
in reply to: MichaelDickens’s comment on: On (Not) Feeling the AGI
Working through a model of the future in a better-understood hypothetical refines gears applicable outside the hypothetical. Exploratory engineering for example is about designing machines that can’t be currently built in practice and often never will be worthwhile to build as designed. It still gives a sense of what’s possible.

(Attributing value to steps of a useful activity is not always practical. Research is like that, very useful that it’s happening overall, but individual efforts are hard to judge, and so acting on attempts to judge them risks goodhart curse.)

Vladimir_Nesov Mar 25, 2025, 4:28 PM
4 points
0
on: On (Not) Feeling the AGI
Many who feel the AGI are not feeling Drosophila, an intermediate level of impact between AGI and superintelligence. Biomass doubling time of 1-3 days is way more than 30% yearly growth. This kind of shortcut to scaling physical infrastructure (robots, fusion plants) doesn’t merely continue the trend of human industry supercharged with AI automation and speed advantage.

Vladimir_Nesov Mar 25, 2025, 6:16 AM
19 points
2
on: Recent AI model progress feels mostly like bullshit
With Blackwell^[1] still getting manufactured and installed, newer large models and especially their long reasoning variants remain unavailable or prohibitively expensive or too slow (GPT-4.5 is out, but not its thinking variant). In a few months Blackwell will be everywhere, and between now and then widely available frontier capabilities will significantly improve. Next year, there will be even larger models trained on Blackwell.

This kind of improvement can’t be currently created with post-training without needing long reasoning traces or larger base models, but post-training is still good at improving things under the lamppost, hence the illusionary nature of current improvement when you care about things further in the dark.
1. ↩︎
  Blackwell is an unusually impactful chip generation, because it fixes what turned out to be a major issue with Ampere and Hopper when it comes to inference of large language models on long context, by increasing scale-up world size from 8 Hopper chips to 72 Blackwell chips. Not having enough memory or compute on each higher bandwidth scale-up network was a bottleneck that made inference unnecessarily slow and expensive. Hopper was still designed before ChatGPT, and it took 2-3 years to propagate importance of LLMs as an application into working datacenters.

Vladimir_Nesov Mar 25, 2025, 5:40 AM
LW: 2 AF: 1
0
AF
in reply to: orthonormal’s comment on: orthonormal’s Shortform
It’s an essential aspect of decision making for an agent to figure out where it might be. Thought experiments try to declare the current situation, but they don’t necessarily need to be able to convincingly succeed. Algorithmic induction, such as updating from Solomonoff prior, is the basic way an agent figures out which situations it should care about, and declaring that we are working with a particular thought experiment doesn’t affect the prior. In line with updatelessness, an agent should be ready for observations in general (according to which of them it cares about more), rather than particular “fair” observations, so distinguishing observations that describe “fair” thought experiments doesn’t seem right either.

Vladimir_Nesov Mar 23, 2025, 7:51 AM
LW: 6 AF: 3
2
AF
on: Towards a scale-free theory of intelligent agency
Coalitional agency seems like an unnecessary constraint on design of a composite agent, since an individual agent could just (choose to) listen to other agents and behave the way their coalition would endorse, thereby effectively becoming a composite agent, without being composite “by construction”. The step where an agent chooses which other (hypothetical) agents to listen to makes constraints on the nature of agents unnecessary, because the choice to listen to some agents and not others can impose any constraints that particular agent cares about, and so an “agent” could be as vague as a “computation” or a program.

(Choosing to listen to a computation means choosing a computation based on considerations other than its output, committing to use its output in a particular way without yet knowing what it’s going to be, and carrying out that commitment once the output becomes available, regardless of what it turns out to be.)

This way we can get back to individual rationality, figuring out how an agent should choose to listen to which other agents/computations when coming up with its own beliefs and decisions. But actually occasionally listening to those other computations is the missing step in most decision theories, which would take care of interaction with other agents (both actual and hypothetical).

Vladimir_Nesov Mar 23, 2025, 7:30 AM
LW: 6 AF: 3
0
AF
on: Towards a scale-free theory of intelligent agency
Discussions of how to aggregate values and probabilities feel disjoint. Jeffrey-Bolker formulation of expected utility presents the preference data as two probability distributions over the same sample space, so that expected utility of an event is reconstructed as the ratio of the event’s measures given by the two priors. (The measure that goes into the numerator is “shouldness”, and the other one remains “probability”.)

This gestures at a way of reducing the problem of aggregating values to the problem of aggregating probabilities. In particular, markets seem to be easier to set up for probabilities than for expected utilities, so it might be better to set up two markets that are technically the same type of thing, one for probability and one for shouldness, than to target expected utility directly. Values of different agents are incomparable, but so are priors, any fundamental issues with aggregation seem to remain unchanged by this reformulation. These can’t be “prediction” markets since resolution is not straightforward and somewhat circular, grounded in what the coalition will settle on eventually, but logical induction has to deal with similar issues already.

Vladimir_Nesov Mar 21, 2025, 3:56 PM
37 points
1
on: Vladimir_Nesov’s Shortform
Abilene site of Stargate will host 100K-128K chips in GB200 NVL72 racks by this summer, and a total of 400K-512K chips in 2026, based on a new post by Crusoe and a reinterpretation of the recent Bloomberg post in light of the Crusoe post. For 2025, it’s less than 200K chips^[1], but more than the surprising 16K-32K chips^[2] that the Bloomberg post suggested. It can be a training system after all, but training a raw compute “GPT-5” (2e27 FLOPs) by the end of 2025 would require using FP8^[3].

The Crusoe post says “initial phase, comprising two buildings at … 200+ megawatts” and “each building is designed to operate up to 50,000 NVIDIA GB200 NVL72s”. Dylan Patel’s estimate (at 1:24:42) for all-in power per Blackwell GPU as a fraction of the datacenter was 2.0 KW (meaning per chip, or else it’s way too much). At GTC 2025, Jensen Huang showed a slide (at 1:20:52) where the estimate is 2.3 KW per chip (100 MW per 85K dies, which is 42.5K chips).

So the “50K GB200 NVL72s” per building from the Mar 2025 Crusoe post can only mean the number of chips (not dies or superchips), and the “100K GPUs” per building from the Jul 2024 Crusoe post must’ve meant 100K compute dies (which is 50K chips). It’s apparently 100-115 MW per building then, or 800-920 MW for all 8 buildings in 2026, which is notably lower than 1.2 GW the Mar 2025 Crusoe post cites.

How can the Bloomberg’s 16K “GB200 semiconductors” in 2025 and 64K in 2026 be squared with this? The Mar 2025 Crusoe post says there are 2 buildings now and 6 additional buildings in 2026, for the total of 8, so in 2026 the campus grows 4x, which fits 16K vs. 64K from Bloomberg. But the numbers themselves must be counting in the units of 8 chips. This fits counting in the units of GB200 NVL8 (see at 1:13:39), which can be referred to as a “superchip”. The Mar 2025 Crusoe post says Abilene site will be using NVL72 racks, so counting in NVL8 is wrong, but someone must’ve made that mistake on the way to the Bloomberg post.

Interpreting the Bloomberg numbers in units of 8 chips, we get 128K chips in 2025 (64K chips per building) and 512K chips in 2026 (about 7K GB200 NVL72 racks). This translates to 256-300 MW for the current 2 buildings and 1.0-1.2 GW for the 8 buildings in 2026. This fits the 1.2 GW figure from the Mar 2025 Crusoe post better, so there might be some truth to the Bloomberg post after all, even as it’s been delivered in a thoroughly misleading way.
1. ↩︎
  Crusoe’s Jul 2024 post explicitly said “each data center building will be able to operate up to 100,000 GPUs”, and in 2024 “GPU” usually meant chip/package (in 2025, it’s starting to mean “compute die”, see at 1:28:04; there are 2 compute dies per chip in GB200 systems). Which suggested 200K chips for the initial 2 buildings.
2. ↩︎
  The post said it’s the number of “coveted GB200 semiconductors”, which is highly ambiguous because of the die/chip/superchip counting issue. A “GB200 superchip” means 2 chips (plus a CPU) by default, so 16K superchips would be 32K chips.
3. ↩︎
  A GB200 chip (not die or superchip) produces 2.5e15 dense BF16 FLOP/s (2.5x more than an H100 chip). Training at 40% utilization for 3 months, 100K chips produce 8e26 FLOPs. But in FP8 it’s 1.6e27 FLOPs. Assuming GPT-4 was 2e25 FLOPs, 100x its raw compute asks “GPT-5” to need about 2e27 FLOPs. In the OpenAI’s introductory video about GPT-4.5, there was a hint it might’ve been trained in FP8 (at 7:38), so it’s not implausible that GPT-5 would be trained in FP8 as well.
What links here?
- Vladimir_Nesov's comment on AI 2027: What Superintelligence Looks Like by Daniel Kokotajlo (Apr 3, 2025, 7:23 PM; 28 points)

Vladimir_Nesov Mar 18, 2025, 9:54 PM
4 points
0
in reply to: DAL’s comment on: DAL’s Shortform
Here’s the place in the interview where he says this (at 16:16). So there were no crucial qualifiers for the 3-6 months figure, which in hindsight makes sense, since it’s near enough to likely refer to his impression of an already existing AI available at Anthropic internally^[1]. Maybe also corroborated in his mind with some knowledge about capabilities of a reasoning model based on GPT-4.5, which is almost certainly available internally at OpenAI.
1. ↩︎
  Probably a reasoning model based on a larger pretrained model than Sonnet 3.7. He recently announced in another interview that a model larger than Sonnet 3.7 is due to come out in “relatively small number of time units” (at 12:35). So probably the plan is to release in a few weeks, but something could go wrong and then it’ll take longer. Possibly long reasoning won’t be there immediately if there isn’t enough compute to run it, and the 3-6 months figure refers to when he expects enough inference compute for long reasoning to be released.

Vladimir_Nesov

Tech­ni­cal Claims

Technical Claims