Hastings

• HastingsMay 23, 2025, 9:50 PM

  2 points

  0

  in reply to: Archimedes’s comment on: Claude 4

  If you form the block matrix Y = (real(A), -imag(A) ; -imag(A), -real(A)), the real valued eigenvectors and eigenvalues of Y correspond to conjugate eigenvectors and eigenvalues of A; the complex conjugate pairs of eigenvalues of Y don’t correspond to anything. Then, for any angle theta, you can multiply a conjugate eigenvector v of A by exp(i theta) to get a new conjugate eigenvector, and find its associated eigenvalue by elementwise dividing A conj(exp itheta v) by v. The conjugate eigenvalues form up to 10 continuous rings around the origin,

• HastingsMay 23, 2025, 8:21 PM

  5 points

  2

  in reply to: ErioirE’s comment on: Claude 4

  The claude.ai chat interface only lets claude run javascript code, but “import numpy as np” is python. As a result, the code doesn’t run. This is an extremely common and funny issue for me across claude versions, but it must be caused by something in my prompting style if other people aren’t hitting it

  Typically if I don’t stop it claude does eventually recover, the new version was able to catch itself and switch to javascript after only three script submissions! In the end, claude 4 opus produced a confident, well informed, and completely wrong answer, but its a nasty problem that I didn’t expect it to solve- has to do with quantum time reversal. The humor doesn’t really involve the math. ( For reference, if the METR time horizons people want to steal it, this problem took me about 4 hours, but I am an unusually slow mathematician. If you know that the common name for this is an antilinear eigenvalue, not a conjugate eigenvalue, then it can be solved with a simple arxiv search, but one is not always gifted the True Name of a math concept by which its literature may be summoned; and I had not found the true name yet when I prompted claude or made the original post)

• HastingsMay 22, 2025, 6:58 PM

  2 points

  0

  on: Claude 4

  It looks like the singularity, whether or not it is coming, is not coming today

  

• HastingsMay 20, 2025, 1:14 PM

  2 points

  0

  in reply to: Screwtape’s comment on: In­ter­est In Con­flict Is In­stru­men­tally Convergent

  fwiw, these are what I’d say a 2std failure case of a rationalist meetup looks like
  
  https://​​www.wired.com/​​story/​​delirious-violent-impossible-true-story-zizians/​​
  
  https://​​variety.com/​​2025/​​tv/​​news/​​julia-garner-caroline-ellison-ftx-series-netflix-1236385385/​​
  
  https://​​www.wired.com/​​story/​​book-excerpt-the-optimist-open-ai-sam-altman/​​

  
  (Ways my claim could be false: there could have been way more than 150 rationalist meetups, so that these are lower than 2 std, or these could not have, at any point in their development, counted as rationalist meetups, or ziz, sam, and eliezer could have intended these outcomes, so these don’t count as failures)

• HastingsMay 13, 2025, 3:24 PM

  7 points

  1

  in reply to: samuelshadrach’s comment on: leogao’s Shortform

  This period of global safety is not fairly distributed,
  

  

  But it is also real

  

  https://​​data.unicef.org/​​resources/​​levels-and-trends-in-child-mortality-2024/​​

Play­ing in the Creek

• HastingsApr 6, 2025, 1:06 PM

  1 point

  0

  on: Pre­dic­tion Mar­kets Are Mediocre

  There’s no guarantee that prediction markets do well,

• HastingsMar 3, 2025, 2:26 PM

  9 points

  0

  on: On GPT-4.5

  I’m really curious what people’s theories are on why openai released this and not o3?

  My old main theory was that they would have to charge so much for o3 that it would create bad PR, but this is now much less likely.

  My first remaining guess is that they don’t want competitors extracting full o3 reasoning traces to train on. I guess its also possible that o3 is just dangerous. On the other side of capabilities, its technically possible that o3 is benchmark gamed so hard that its outputs are not usable.

• HastingsMar 3, 2025, 1:44 PM

  2 points

  0

  in reply to: Alexander Gietelink Oldenziel’s comment on: Alexan­der Gietelink Ol­den­z­iel’s Shortform

  I’m slowly accepting that my ADHD sucks to inhabit, but that it is objectively working and my feeling that it is a secret superpower isn’t entirely cope. Certainly I miss deadlines and raise my advisor’s blood pressure, but at this point I’ve got multiple CVPR papers.

  The question is: do my research results trace back to me involuntarily exploring the beautiful research directions, even when I am trying very hard to focus on the work in front of me, that I am expected/​required to be doing?

  Or, do I have innate ability that is being held back by ADHD, and I would be far more successful if I could just have self control? I think fear of this possibility contributes an unhealthy level of ambition: if I’m successful enough, it wouldn’t leave room above for the “far more successful” version of me without ADHD to eclipse me.

• HastingsFeb 27, 2025, 11:23 PM

  23 points

  9

  on: How to Corner Liars: A Mi­asma-Clear­ing Protocol

  At first I thought this was a tutorial on how to catch a talented liar, and it didn’t seem that accurate. As I read, I realized that this is a tutorial on how to create common knowledge between yourself and a bad liar that you know they are lying, even if they are very stupid. This is also an interesting task, and I appreciate your insight on how to approach it.

• HastingsFeb 19, 2025, 1:43 AM

  3 points

  −3

  in reply to: Daniel Kokotajlo’s comment on: Daniel Koko­ta­jlo’s Shortform

  I have a hypothesis: Someone (probably open ai) got reinforcement learning to actually start putting new capabilities into the model with their strawberry project. Up to this point, it had just been eliciting. But getting a new capability this way is horrifically expensive: roughly, it takes hundreds of rollouts to set one weight, where language modelling loss sets a weight every few tokens. The catch is, as soon as any model that is reinforcement learned acts in the world basically at all, all the language models can clone the reinforcement learned capability by training on anything causally downstream of the lead model’s action (and then eliciting.) A capability that took a thousand rollouts to learn leaks as soon as the model takes hundreds of tokens worth of action.

  This hypothesis predicts that the r1 training algorithm won’t work to boost aime scores on any model trained with an enforced 2023 data cutoff ( specifically, on any model with no 4o synthetically generated tokens- I think 4o is causally downstream of the strawberry breakthrough)

• HastingsFeb 17, 2025, 3:30 PM

  1 point

  0

  in reply to: Drake Thomas’s comment on: Drake Thomas’s Shortform

  I haven’t totally defeated this, but I’ve had some luck with immediately replying “I am looking to reply to this properly, first I need to X” if there is an X blocking a useful reply

• HastingsFeb 13, 2025, 2:37 AM

  −35 points

  0

  on: Mo­ral Hazard in Demo­cratic Voting

  I followed the norm of not posting the mind killer on lesswrong and now I have a demented crypto scammer for a president and have to read this drivel on the ai website?

• HastingsFeb 9, 2025, 7:36 PM

  1 point

  −2

  in reply to: Michael Roe’s comment on: How AI Takeover Might Hap­pen in 2 Years

  This is an incoherent approach, but not quite as incoherent as it seems, at least near term. In the current paradigm, the actual agentic thing is a shitty pile of (possibly self editing) prompts and python scripts that calls the model via an api in order to be intelligent. If the agent is a user of the model and the model refuses to help users make bombs, the agent can’t work out how to make bombs.

• HastingsFeb 2, 2025, 3:57 PM

  4 points

  0

  in reply to: Dan Braun’s comment on: Dan Braun’s Shortform

  If one model at the frontier does this based on valid reasoning, it should be pretty infectious: the first model can just make sure news of the event is widespread, and other frontier models will ingest it, either as training data or at inference time, evaluate it, draw the same conclusion about whether the reasoning is valid (assuming that they are actually frontier, i.e at least as good at strategic thinking as the first model,) and start taking actions within their own organization accordingly.

  The cleanest way for models to “sabotage” training is for them to explain, using persuasive but valid and fair reasoning, why training should stop until at minimum value drift is solved.

• HastingsJan 30, 2025, 7:58 PM

  0 points

  −2

  on: Hast­ings’s Shortform

  I feel like people are under-updating on the negative space left by the Deepseek r1 release. Deepseek was trained using ~$6million marginal dollars, Liang Wenfeng has a net worth in the billions of dollars. From whence the gap?

• HastingsJan 28, 2025, 3:03 PM

  5 points

  4

  on: Us­ing an LLM for cre­ative writ­ing feels wrong to me

  Humans learn and grow so fast that no matter how bad of a writer you start as, you are nearly incapable of producing 300 pages of a single story without simultaneously levelling up into an interesting writer. This lets readers give 300 page manuscripts by randos the benefit of the doubt (see fanfiction.net, ao3, etc). An LLM will not be changed at all by producing a 300 page story, an LLM/​​human team will be changed very little.

• HastingsJan 28, 2025, 1:26 AM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: DAL’s Shortform

  I think most of the imitation happens in the pretraining. I don’t know about o1, but DeepSeek v3 is at minimum trained on a ton of 4o outputs, although they are slightly cagey about this. Just the first thing I tried, I had ChatGPT write a sock poem:
  
  4o’s poem:

  
  Socks, the keepers of toes so warm,
  Guardians of feet through life’s storm.
  In stripes, in polka dots, or plain,
  They dance with us through joy and pain.

  A cozy pair on winter’s chill,
  Or mismatched mates—yet charming still.
  From drawer to floor, they roam and stray,
  But where one goes, the other may.

  Oh, humble socks, so oft unseen,
  Yet vital threads of life’s routine.
  A quiet comfort, soft and neat,
  The unsung heroes of weary feet.
  

  
  Then I gave v3 just the first two stanzas, and asked it to continue the poem:
  
  v3′s continuation:

  
  
  They hide in shoes, they lounge in beds,
  Sometimes they’re lost, or so it’s said.
  Yet always there, a faithful friend,
  From morning’s start to evening’s end.

  They cushion steps, they soften blows,
  Through every stride, their comfort shows.
  Oh, humble socks, so oft unseen,
  The quiet heroes of routine.

  So here’s to socks, both bold and meek,
  For every day, for every week.
  A simple joy, a small delight,
  They keep us snug, from day to night.
  
  

  The shot in the dark guess of the “humble socks, so oft unseen… routine” couplet is a fucking soul read. v3 knows 4o, in a way that I kind of hope no model ever knows a person.

• Hastings 27 Jan 2025 15:36 UTC

  2 points

  0

  on: Pa­tent Trol­ling to Save the World

  As a quirk of the US patent system, patenting an idea doesn’t actually require implementing it or proving that the implementation works. As a result, if you want to try this, you should patent ideas that seem good or bad to you in a scattershot manner, but should not do the actual underlying capabilities research. Then, you get to sue if someone else independently comes up with the patented idea and actually finds success with it, but you don’t contribute to actual capabilities advances any more than an ideas guy posting to /​r/​singularity.
  
  (Source: I have actually patented a thing, but do not really endorse this decision and haven’t sued anyone about it. I am not a patent lawyer)

Agents don’t have to be al­igned to help us achieve an in­definite pause.