Thane Ruthenis

Karma: 4,996

Thane Ruthenis 29 Jan 2025 2:18 UTC
2 points
0
in reply to: Daniel Kokotajlo’s comment on: What Indicators Should We Watch to Disambiguate AGI Timelines?
Not for math benchmarks. Here’s one way it can “cheat” at them: suppose that the CoT would involve the model generating candidate proofs/derivations, then running an internal (learned, not hard-coded) proof verifier on them, and either rejecting the candidate proof and trying to generate a new one, or outputting it. We know that this is possible, since we know that proof verifiers can be compactly specified.
This wouldn’t actually show “agency” and strategic thinking of the kinds that might generalize to open-ended domains and “true” long-horizon tasks. In particular, this would mostly fail the condition (2) from my previous comment.
Something more open-ended and requiring “research taste” would be needed. Maybe a comparable performance on METR’s benchmark would work for this (i. e., the model can beat a significantly larger fraction of it at 1 billion tokens compared to 1 million)? Or some other benchmark that comes closer to evaluating real-world performance.
Edit: Oh, math-benchmark performance would convince me if we get access to a CoT sample and it shows that the model doesn’t follow the above “cheating” approach, but instead approaches the problem strategically (in some sense). (Which would also require this CoT not to be hopelessly steganographied, obviously.)

Thane Ruthenis 29 Jan 2025 1:19 UTC
4 points
0
in reply to: Daniel Kokotajlo’s comment on: What Indicators Should We Watch to Disambiguate AGI Timelines?
Prooobably ~simultaneously, but I can maybe see it coming earlier and in a way that isn’t wholly convincing to me. In particular, it would still be a fixed-length task; much longer-length than what the contemporary models can reliably manage today, but still hackable using poorly-generalizing “agency templates” instead of fully general “compact generators of agenty behavior” (which I speculate humans to have and RL’d LLMs not to). It would be some evidence in favor of “AI can accelerate AI R&D”, but not necessarily “LLMs trained via SSL+RL are AGI-complete”.
Actually, I can also see it coming later. For example, some suppose that the capability researchers invent some method for reliably-and-indefinitely extending the amount of serial computations a reasoning model can productively make use of, but the compute or memory requirements grow very fast with the length of a CoT. Some fairly solid empirical evidence and theoretical arguments in favor of boundless scaling can appear quickly, well before the algorithms are made optimal enough to (1) handle weeks-long CoTs and/or (2) allow wide adoption (thus making it available to you).
I think the second scenario is more plausible, actually.

Thane Ruthenis 29 Jan 2025 0:29 UTC
4 points
0
in reply to: Daniel Kokotajlo’s comment on: What Indicators Should We Watch to Disambiguate AGI Timelines?
I wish we had something to bet on better than “inventing a new field of science,”
I’ve thought of one potential observable that is concrete, should be relatively low-capability, and should provoke a strong update towards your model for me:
If there is an AI model such that the complexity of R&D problems it can solve (1) scales basically boundlessly with the amount of serial compute provided to it (or to a “research fleet” based on it), (2) scales much faster with serial compute than with parallel compute, and (3) the required amount of human attention (“babysitting”) is constant or grows very slowly with the amount of serial compute.
This attempts to directly get at the “autonomous self-correction” and “ability to think about R&D problems strategically” ideas.
I’ve not fully thought through all possible ways reality could Goodhart to this benchmark, i. e. “technically” pass it but in a way I find unconvincing. For example, if I failed to include the condition (2), o3 would have probably already “passed” it (since it potentially achieved better performance on ARC-AGI and FrontierMath by sampling thousands of CoTs then outputting the most frequent answer). There might be other loopholes like this...
But it currently seems reasonable and True-Name-y to me.
What links here?
- Thane Ruthenis's comment on What Indicators Should We Watch to Disambiguate AGI Timelines? by snewman (29 Jan 2025 2:18 UTC; 7 points)

Thane Ruthenis 28 Jan 2025 22:22 UTC
2 points
0
in reply to: kwiat.dev’s comment on: DeepSeek Panic at the App Store
Well, he didn’t do it yet either, did he? His new announcement is, likewise, just that: an announcement. Manifold is still 35% on him not following through on it, for example.

Thane Ruthenis 28 Jan 2025 21:41 UTC
5 points
1
in reply to: jbash’s comment on: DeepSeek Panic at the App Store
“Intensely competent and creative”, basically, maybe with a side of “obsessed” (with whatever they’re cracked at).

Thane Ruthenis 28 Jan 2025 21:36 UTC
5 points
0
in reply to: Garrett Baker’s comment on: DeepSeek Panic at the App Store
Supposedly Trump announced that back in October, so it should already be priced in.
(Here’s my attempt at making sense of it, for what it’s worth.)

Thane Ruthenis 28 Jan 2025 21:34 UTC
LW: 5 AF: 3
−2
AF
on: Thane Ruthenis’s Shortform
Here’s a potential interpretation of the market’s apparent strange reaction to DeepSeek-R1 (shorting Nvidia).
I don’t fully endorse this explanation, and the shorting may or may not have actually been due to Trump’s tariffs + insider trading, rather than DeepSeek-R1. But I see a world in which reacting this way to R1 arguably makes sense, and I don’t think it’s an altogether implausible world.
If I recall correctly, the amount of money globally spent on inference dwarfs the amount of money spent on training. Most economic entities are not AGI labs training new models, after all. So the impact of DeepSeek-R1 on the pretraining scaling laws is irrelevant: sure, it did not show that you don’t need bigger data centers to get better base models, but that’s not where most of the money was anyway.
And my understanding is that, on the inference-time scaling paradigm, there isn’t yet any proven method of transforming arbitrary quantities of compute into better performance:
- Reasoning models are bottlenecked on the length of CoTs that they’ve been trained to productively make use of. They can’t fully utilize even their context windows; the RL pipelines just aren’t up to that task yet. And if that bottleneck were resolved, the context-window bottleneck would be next: my understanding is that infinite context/”long-term” memories haven’t been properly solved either, and it’s unknown how they’d interact with the RL stage (probably they’d interact okay, but maybe not).
  - o3 did manage to boost its ARC-AGI and (maybe?) FrontierMath performance by… generating a thousand guesses and then picking the most common one...? But who knows how that really worked, and how practically useful it is. (See e. g. this, although that paper examines a somewhat different regime.)
- Agents, from Devin to Operator to random open-source projects, are still pretty terrible. You can’t set up an ecosystem of agents in a big data center and let them rip, such that the ecosystem’s power scales boundlessly with the data center’s size. For all but the most formulaic tasks, you still need a competent human closely babysitting everything they do, which means you’re still mostly bottlenecked on competent human attention.
Suppose that you don’t expect the situation to improve: that the inference-time scaling paradigm would hit a ceiling pretty soon, or that it’d converge to distilling search into forward passes (such that the end users end up using very little compute on inference, like today), and that agents just aren’t going to work out the way the AGI labs promise.
In such a world, a given task can either be completed automatically by an AI for some fixed quantity of compute X, or it cannot be completed by an AI at all. Pouring ten times more compute on it does nothing.
In such a world, if it were shown that the compute needs of a task can be met with ten times less compute than previously expected, this would decrease the expected demand for compute.
The fact that capable models can be run locally might increase the number of people willing to use them (e. g., those very concerned about data privacy), as might the ability to automatically complete 10x as many trivial tasks. But it’s not obvious that this demand spike will be bigger than the simultaneous demand drop.
And I, at least, when researching ways to set up DeepSeek-R1 locally, found myself more drawn to the “wire a bunch of Macs together” option, compared to “wire a bunch of GPUs together” (due to the compactness). If many people are like this, it makes sense why Nvidia is down while Apple is (slightly) up. (Moreover, it’s apparently possible to run the full 671b-parameter version locally, and at a decent speed, using a pure RAM+CPU setup; indeed, it appears cheaper than mucking about with GPUs/Macs, just $6,000.)
This world doesn’t seem outright implausible to me. I’m bearish on agents and somewhat skeptical of inference-time scaling. And if inference-time scaling does deliver on its promises, it’ll likely go the way of search-and-distill.
On balance, I don’t actually expect the market to have any idea what’s going on, so I don’t know that its reasoning is this specific flavor of “well-informed but skeptical”. And again, it’s possible the drop was due to Trump, nothing to do with DeepSeek at all.
But as I’d said, this reaction to DeepSeek-R1 does not seem necessarily irrational/incoherent to me.

Thane Ruthenis 27 Jan 2025 20:59 UTC
4 points
0
in reply to: johnswentworth’s comment on: johnswentworth’s Shortform
I don’t have deep expertise in the subject, but I’m inclined to concur with the people saying that the widely broadcast signals don’t actually represent one consistent thing, despite your plausible argument to the contrary.
Here’s a Scott Alexander post speculating why that might be the case. In short: there was an optimization pressure towards making internal biological signals very difficult to decode, because easily decodable signals were easy target for parasites evolving to exploit them. As the result, the actual signals are probably represented as “unnecessarily” complicated, timing-based combinations of various “basic” chemical, electrical, etc. signals, and they’re somewhat individualized to boot. You can’t decode them just by looking at any one spatially isolated chunk of the body, by design.
Basically: separate chemical substances (and other components that look “simple” locally/from the outside) are not the privileged basis for decoding internal signals. They’re the anti-privileged basis, if anything.

Thane Ruthenis 26 Jan 2025 19:44 UTC
8 points
0
on: Instrumental Goals Are A Different And Friendlier Kind Of Thing Than Terminal Goals
Hmm. This does have the feel of gesturing at something important, but I don’t see it clearly yet...
Free association: geometric rationality.
MIRI’s old results argue that “corrigibility via uncertainty regarding the utility function” doesn’t work, because if the agent maximizes expected utility anyway, it doesn’t matter one whit whether we’re taking expectation over actions or over utility functions. However, the corrigibility-via-instrumental-goals does have the feel of “make the agent uncertain regarding what goals it will want to pursue next”. Is there, therefore, some way to implement something-like-this while avoiding MIRI’s counterexample?
Loophole: the counterexample works in the arithmetically-expected utility regime. What if we instead do it in the geometric one? I. e., have an agent take actions that maximize the geometrically-expected product of candidate utility functions? This is a more conservative/egalitarian regime: any one utility function flipping to negative or going to zero wipes out all value, unlike with sums (which are more tolerant of ignoring/pessimizing some terms, and can have “utility monsters”). So it might potentially make the agent actually hesitant to introduce potentially destructive changes to its environment...
(This is a very quick take and it potentially completely misunderstands the concepts involved. But I figure it’s better to post than not, in case the connection turns out obvious to anyone else.)

Thane Ruthenis 25 Jan 2025 20:19 UTC
LW: 3 AF: 1
0
AF
in reply to: Vladimir_Nesov’s comment on: Thane Ruthenis’s Shortform
Coming back to this in the wake of DeepSeek r1...
I don’t think the cumulative compute multiplier since GPT-4 is that high, I’m guessing 3x, except perhaps for DeepSeek-V3, which wasn’t trained compute optimally and didn’t use a lot of compute, and so it remains unknown what happens if its recipe is used compute optimally with more compute.
How did DeepSeek accidentally happen to invest precisely the amount of compute into V3 and r1 that would get them into the capability region of GPT-4/o1, despite using training methods that clearly have wildly different returns on compute investment?
Like, GPT-4 was supposedly trained for $100 million, and V3 for $5.5 million. Yet, they’re roughly at the same level. That should be very surprising. Investing a very different amount of money into V3′s training should’ve resulted in it either massively underperforming GPT-4, or massively overperforming, not landing precisely in its neighbourhood!
Consider this graph. If we find some training method A, and discover that investing $100 million in it lands us at just above “dumb human”, and then find some other method B with a very different ROI, and invest $5.5 million in it, the last thing we should expect is to again land near “dumb human”.
Or consider this trivial toy model: You have two linear functions, f(x) = Ax and g(x) = Bx, where x is the compute invested, output is the intelligence of the model, and f and g are different training methods. You pick some x effectively at random (whatever amount of money you happened to have lying around), plug it into f, and get, say, 120. Then you pick a different random value of x, plug it into g, and get… 120 again. Despite the fact that the multipliers A and B are likely very different, and you used very different x-values as well. How come?
The explanations that come to mind are:
- It actually is just that much of a freaky coincidence.
- DeepSeek have a superintelligent GPT-6 equivalent that they trained for $10 million in their basement, and V3/r1 are just flexes that they specifically engineered to match GPT-4-ish level.
- DeepSeek directly trained on GPT-4 outputs, effectively just distilling GPT-4 into their model, hence the anchoring.
- DeepSeek kept investing and tinkering until getting to GPT-4ish level, and then stopped immediately after attaining it.
- GPT-4ish neighbourhood is where LLM pretraining plateaus, which is why this capability level acts as a sort of “attractor” into which all training runs, no matter how different, fall.

Thane Ruthenis 23 Jan 2025 18:54 UTC
18 points
4
in reply to: Lao Mein’s comment on: Lao Mein’s Shortform
Hm, that’s not how it looks like to me? Look at the 6-months plot:
The latest growth spurt started January 14th, well before the Stargate news went public, and this seems like just its straight-line continuation. The graph past the Stargate announcement doesn’t look “special” at all. Arguably, we can interpret it as Stargate being part of the reference class of events that are currently driving GE Vernova up – not an outside-context event as such, but still relevant...
Except for the fact that the market indeed did not respond to Stargate immediately. Which makes me think it didn’t react at all, and the next bump you’re seeing is some completely different thing from the reference class of GE-Vernova-raising events (to which Stargate itself does not in fact belong).
Which is probably just something to do with Trump’s expected policies. Note that there was also a jump on November 4th. Might be some general expectation that he’s going to compete with China regarding energy, or that he’d cut the red tape on building plants for his tech backers.
In which case, perhaps it’s because Stargate was already priced-in – or at least whichever fraction of Stargate ($30-$100 billion?) is real and is actually going to be realized. We already knew datacenters of this scale were going to be built this year, after all.

Thane Ruthenis 22 Jan 2025 19:05 UTC
5 points
1
in reply to: Robert Cousineau’s comment on: Thane Ruthenis’s Shortform
Overall, I think that the new Stargate numbers published may (call it 40%) be true, but I also think there is a decent chance this is new administration trump-esque propoganda/bluster (call it 45%),
I think it’s definitely bluster, the question is how much of a done deal it is to turn this bluster into at least $100 billion.
I don’t this this changes the prior expected path of datacenter investment at all. It’s precisely how the expected path was going to look like, the only change is how relatively high-profile/flashy this is being. (Like, if they invest $30-100 billion into the next generation of pretrained models in 2025-2026, and that generation fails, they ain’t seeing the remaining $400 billion no matter what they’re promising now. Next generation makes or breaks the investment into the subsequent one, just as was expected before this.)
Satya Nadella was just asked about how funding looks for stargate, and said “Microsoft is good for investing 80b”
He does very explicitly say “I am going to spend 80 billion dollars building Azure”. Which I think has nothing to do with Stargate.
(I think the overall vibe he gives off is “I don’t care about this Stargate thing, I’m going to scale my data centers and that’s that”. I don’t put much stock into this vibe, but it does fit with OpenAI/Microsoft being on the outs.)

Thane Ruthenis 22 Jan 2025 17:53 UTC
LW: 23 AF: 10
2
AF
on: Thane Ruthenis’s Shortform
So, Project Stargate. Is it real, or is it another “Sam Altman wants $7 trillion”? Some points:
- The USG invested nothing in it. Some news outlets are being misleading about this. Trump just stood next to them and looked pretty, maybe indicated he’d cut some red tape. It is not an “AI Manhattan Project”, at least, as of now.
- Elon Musk claims that they don’t have the money and that SoftBank (stated to have “financial responsibility” in the announcement) has less than $10 billion secured. If true, while this doesn’t mean they can’t secure an order of magnitude more by tomorrow, this does directly clash with “deploying $100 billion immediately” statement.
  - But Sam Altman counters that Musk’s statement is “wrong”, as Musk “surely knows”.
  - I… don’t know which claim I distrust more. Hm, I’d say Altman feeling the need to “correct the narrative” here, instead of just ignoring Musk, seems like a sign of weakness? He doesn’t seem like the type to naturally get into petty squabbles like this, otherwise.
  - (And why, yes, this is how an interaction between two Serious People building world-changing existentially dangerous megaprojects looks like. Apparently.)
- Some people try to counter Musk’s claim by citing Satya Nadella’s statement that Satya’s “good for his $80 billion”. But that’s not referring to Stargate, that’s about Azure. Microsoft is not listed as investing in Stargate at all, it’s only a “technology partner”.
- Here’s a brief analysis from the CIO of some investment firm. He thinks it’s plausible that the stated “initial group of investors” (SoftBank, OpenAI, Oracle, MGX) may invest fifty billion dollars into it over the next four years; not five hundred billion.
  - They don’t seem to have the raw cash for even $100 billion – and if SoftBank secured the missing funding from some other set of entities, why aren’t they listed as “initial equity funders” in the announcement?
Overall, I’m inclined to fall back on @Vladimir_Nesov’s analysis here. $30-50 billion this year seems plausible. But $500 billion is, for now, just Altman doing door-in-the-face as he had with his $7 trillion.
I haven’t looked very deeply into it, though. Additions/corrections welcome!

Thane Ruthenis 21 Jan 2025 2:57 UTC
3 points
0
in reply to: gwern’s comment on: Implications of the inference scaling paradigm for AI safety
McLaughlin announced 2025-01-13 that he had joined OpenAI
He gets onboarded only on January 28th, for clarity.

Thane Ruthenis 21 Jan 2025 0:57 UTC
5 points
−1
in reply to: green_leaf’s comment on: Thane Ruthenis’s Shortform
I think GPT-4 to o3 represent non-incremental narrow progress, but only, at best, incremental general progress.
(It’s possible that o3 does “unlock” transfer learning, or that o4 will do that, etc., but we’ve seen no indication of that so far.)

Thane Ruthenis 21 Jan 2025 0:04 UTC
5 points
0
in reply to: Seth Herd’s comment on: Thane Ruthenis’s Shortform
I’m sure he wants hype, but he doesn’t want high expectations that are very quickly falsified
There’s a possibility that this was a clown attack on OpenAI instead...

Thane Ruthenis 21 Jan 2025 0:02 UTC
6 points
4
in reply to: Joseph Miller’s comment on: Thane Ruthenis’s Shortform
Valid, I was split on whether it’s worth posting vs. it’d be just me taking my part in spreading this nonsense. But it’d seemed to me that a lot of people, including LW regulars, might’ve been fooled, so I erred on the side of posting.

Thane Ruthenis 20 Jan 2025 23:59 UTC
4 points
2
in reply to: Cervera’s comment on: Thane Ruthenis’s Shortform
As I’d said, I think he’s right about the o-series’ theoretic potential. I don’t think there is, as of yet, any actual indication that this potential has already been harnessed, and therefore that it works as well as the theory predicts. (And of course, the o-series scaling quickly at math is probably not even an omnicide threat. There’s an argument for why it might be – that the performance boost will transfer to arbitrary domains – but that doesn’t seem to be happening. I guess we’ll see once o3 is public.)

Thane Ruthenis 20 Jan 2025 23:53 UTC
6 points
0
in reply to: Alexander Gietelink Oldenziel’s comment on: Thane Ruthenis’s Shortform
The thing is—last time I heard about OpenAI rumors it was Strawberry.
That was part of my reasoning as well, why I thought it might be worth engaging with!
But I don’t think this is the same case. Strawberry/Q* was being leaked-about from more reputable sources, and it was concurrent with dramatic events (the coup) that were definitely happening.
In this case, all evidence we have is these 2-3 accounts shitposting.

Thane Ruthenis 20 Jan 2025 15:40 UTC
LW: 120 AF: 37
8
AF
on: Thane Ruthenis’s Shortform
Alright, so I’ve been following the latest OpenAI Twitter freakout, and here’s some urgent information about the latest closed-doors developments that I’ve managed to piece together:
- Following OpenAI Twitter freakouts is a colossal, utterly pointless waste of your time and you shouldn’t do it ever.
- If you saw this comment of Gwern’s going around and were incredibly alarmed, you should probably undo the associated update regarding AI timelines (at least partially, see below).
- OpenAI may be running some galaxy-brained psyops nowadays.
Here’s the sequence of events, as far as I can tell:
1. Some Twitter accounts that are (claiming, without proof, to be?) associated with OpenAI are being very hype about some internal OpenAI developments.
2. Gwern posts this comment suggesting an explanation for point 1.
3. Several accounts (e. g., one, two) claiming (without proof) to be OpenAI insiders start to imply that:
  1. An AI model recently finished training.
  2. Its capabilities surprised and scared OpenAI researchers.
  3. It produced some innovation/is related to OpenAI’s “Level 4: Innovators” stage of AGI development.
4. Gwern’s comment goes viral on Twitter (example).
5. A news story about GPT-4b micro comes out, indeed confirming a novel OpenAI-produced innovation in biotech. (But it is not actually an “innovator AI”.)
6. The stories told by the accounts above start to mention that the new breakthrough is similar to GPT-4b: that it’s some AI model that produced an innovation in “health and longevity”. But also, that it’s broader than GPT-4b, and that the full breadth of this new model’s surprising emergent capabilities is unclear. (One, two, three.)
7. Noam Brown, an actual confirmed OpenAI researcher, complains about “vague AI hype on social media”, and states they haven’t yet actually achieved superintelligence.
8. The Axios story comes out, implying that OpenAI has developed “PhD-level superagents” and that Sam Altman is going to brief Trump on them. Of note:
  1. Axios is partnered with OpenAI.
  2. If you put on Bounded Distrust lens, you can see that the “PhD-level superagents” claim is entirely divorced from any actual statements made by OpenAI people. The article ties-in a Mark Zuckerberg quote instead, etc. Overall, the article weaves the impression it wants to create out of vibes (with which it’s free to lie) and not concrete factual statements.
9. The “OpenAI insiders” gradually ramp up the intensity of their story all the while, suggesting that the new breakthrough would allow ASI in “weeks, not years”, and also that OpenAI won’t release this “o4-alpha” until 2026 because they have a years-long Master Plan, et cetera. Example, example.
10. Sam Altman complains about “twitter hype” being “out of control again”.
11. OpenAI hype accounts deflate.
What the hell was all that?
First, let’s dispel any notion that the hype accounts are actual OpenAI insiders who know what they are talking about:
- “Satoshi” claims to be blackmailing OpenAI higher-ups in order to be allowed to shitpost classified information on Twitter. I am a bit skeptical of this claim, to put it mildly.
- “Riley Coyote” has a different backstory which is about as convincing by itself, and which also suggests that “Satoshi” is “Riley”’s actual source.
As far as I can tell digging into the timeline, both accounts just started acting as if they are OpenAI associates posting leaks. Not even, like, saying that they’re OpenAI associates posting leaks, much less proving that. Just starting to act as if they’re OpenAI associates and that everyone knows this. Their tweets then went viral. (There’s also the strawberry guy, who also implies to be an OpenAI insider, who also joined in on the above hype-posting, and who seems to have been playing this same game for a year now. But I’m tired of looking up the links, and the contents are intensely unpleasant. Go dig through that account yourself if you want.)
In addition, none of the OpenAI employee accounts with real names that I’ve been able to find have been participating in this hype cycle. So if OpenAI allowed its employees to talk about what happened/is happening, why weren’t any confirmed-identity accounts talking about it (except Noam’s, deflating it)? Why only the anonymous Twitter people?
Well, because this isn’t real.
That said, the timing is a bit suspect. This hype starting up, followed by the GPT-4b micro release and the Axios piece, all in the span of ~3 days? And the hype men’s claims at least partially predicting the GPT-4b micro thing?
There’s three possibilities:
- A coincidence. (The predictions weren’t very precise, just “innovators are coming”. The details about health-and-longevity and the innovative output got added after the GPT-4b piece, as far as I can tell.)
- A leak in one of the newspapers working on the GPT-4b story (which the grifters then built a false narrative around).
- Coordinated action by OpenAI.
One notable point is, the Axios story was surely coordinated with OpenAI, and it’s both full of shenanigans and references the Twitter hype (“several OpenAI staff have been telling friends they are both jazzed and spooked by recent progress”). So OpenAI was doing shenanigans. So I’m slightly inclined to believe it was all an OpenAI-orchestrated psyop.
Let’s examine this possibility.
Regarding the truth value of the claims: I think nothing has happened, even if the people involved are OpenAI-affiliated (in a different sense from how they claim). Maybe there was some slight unexpected breakthrough on an obscure research direction, at most, to lend an air of technical truth to those claims. But I think it’s all smoke and mirrors.
However, the psyop itself (if it were one) has been mildly effective. I think tons of people actually ended up believing that something might be happening (e. g., janus, the AI Notkilleveryoneism Memes guy, myself for a bit, maybe gwern, if his comment referenced the pattern of posting related to the early stages of this same event).
That said, as Eliezer points out here, it’s advantageous for OpenAI to be crying wolf: both to drive up/maintain hype among their allies, and to frog-boil the skeptics into instinctively dismissing any alarming claims. Such that, say, if there ever are actual whistleblowers pseudonymously freaking out about unexpected breakthroughs on Twitter, nobody believes them.
That said, I can’t help but think that if OpenAI were actually secure in their position and making insane progress, they would not have needed to do any of this stuff. If you’re closing your fingers around agents capable of displacing the workforce en masse, if you see a straight shot to AGI, why engage in this childishness? (Again, if Satoshi and Riley aren’t just random trolls.)
Bottom line, one of the following seems to be the case:
- There’s a new type of guy, which is to AI/OpenAI what shitcoin-shills are to cryptocurrency.
- OpenAI is engaging in galaxy-brained media psyops.
Oh, and what’s definitely true is that paying attention to what’s going viral on Twitter is a severe mistake. I’ve committed it for the first and last time.
I also suggest that you unroll the update you might’ve made based on Gwern’s comment. Not the part describing to the o-series’ potential – that’s of course plausible and compelling. The part where that potential seems to have already been confirmed and realized according to ostensible OpenAI leaks – because those leaks seem to be fake. (Unless Gwern was talking about some other demographic of OpenAI accounts being euphorically optimistic on Twitter, which I’ve somehow missed?)^[1]
(Oh, as to Sam Altman meeting with Trump? Well, that’s probably because Trump’s Sinister Vizier, Sam Altman’s sworn nemesis, Elon Musk, is whispering in Trump ear ²⁴⁄₇ suggesting to crush OpenAI, and if Altman doesn’t seduce Trump ASAP, Trump will do that. Especially since OpenAI is currently vulnerable due to their legally dubious for-profit transition.
This planet is a clown show.)
I’m currently interested in:
- Arguments for actually taking the AI hype people’s claims seriously. (In particular, were any actual OpenAI employees provably involved, and did I somehow miss them?)
- Arguments regarding whether this was an OpenAI psyop vs. some random trolls.
Also, pinging @Zvi in case any of those events showed up on his radar and he plans to cover them in his newsletter.
1. ^
  Also, I can’t help but note that the people passing the comment around (such as this, this) are distorting it. The Gwern-stated claim isn’t that OpenAI are close to superintelligence, it’s that they may feel as if they’re close to superintelligence. Pretty big difference!
  Though, again, even that is predicated on actual OpenAI employees posting actual insider information about actual internal developments. Which I am not convinced is a thing that is actually happening.
What links here?
- Thane Ruthenis's comment on Thane Ruthenis’s Shortform by Thane Ruthenis (17 Jan 2025 22:57 UTC; 34 points)