Dan H

AISN #49: Su­per­in­tel­li­gence Strategy

In­tro­duc­ing MASK: A Bench­mark for Mea­sur­ing Hon­esty in AI Systems

On the Ra­tion­al­ity of Deter­ring ASI

AISN #48: Utility Eng­ineer­ing and EnigmaEval

• Dan HFeb 13, 2025, 12:18 AM

  16 points

  3

  in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Per­l­man’s Shortform

  capability thresholds be vague or extremely high

  xAI’s thresholds are entirely concrete and not extremely high.

  evaluation be unspecified or low-quality

  They are specified and as high-quality as you can get. (If there are better datasets let me know.)

  I’m not saying it’s perfect, but I wouldn’t but them all in the same bucket. Meta’s is very different from DeepMind’s or xAI’s.

• Dan HFeb 10, 2025, 9:28 PM

  14 points

  12

  in reply to: Drake Thomas’s comment on: Drake Thomas’s Shortform

  though I don’t think xAI took an official position one way or the other

  I assumed most of everybody assumed xAI supported it since Elon did. I didn’t bother pushing for an additional xAI endorsement given that Elon endorsed it.

AISN #47: Rea­son­ing Models

AISN #46: The Transition

• Dan HJan 19, 2025, 1:37 AM

  32 points

  0

  in reply to: meemi’s comment on: meemi’s Shortform

  It’s probably worth them mentioning for completeness that Nat Friedman funded an earlier version of the dataset too. (I was advising at that time and provided the main recommendation that it needs to be research-level because they were focusing on Olympiad level.)

  Also can confirm they aren’t giving access to the mathematicians’ questions to AI companies other than OpenAI like xAI.

AISN #45: Cen­ter for AI Safety 2024 Year in Review

• Dan HDec 3, 2024, 4:45 AM

  10 points

  2

  on: (The) Light­cone is noth­ing with­out its peo­ple: LW + Lighthaven’s big fundraiser

  and have clearly been read a non-trivial amount by Elon Musk

  Nit: He heard this idea in conversation with an employee AFAICT.

AISN #44: The Trump Cir­cle on AI Safety Plus, Chi­nese re­searchers used Llama to cre­ate a mil­i­tary tool for the PLA, a Google AI sys­tem dis­cov­ered a zero-day cy­ber­se­cu­rity vuln­er­a­bil­ity, and Com­plex Sys­tems

AI Safety Newslet­ter #43: White House Is­sues First Na­tional Se­cu­rity Memo on AI Plus, AI and Job Dis­place­ment, and AI Takes Over the Nobels

AI Safety Newslet­ter #42: New­som Ve­toes SB 1047 Plus, OpenAI’s o1, and AI Gover­nance Summary

AI Safety Newslet­ter #41: The Next Gen­er­a­tion of Com­pute Scale Plus, Rank­ing Models by Sus­cep­ti­bil­ity to Jailbreak­ing, and Ma­chine Ethics

AI fore­cast­ing bots incoming

• Dan HAug 26, 2024, 1:03 PM

  4 points

  −14

  on: Dar­wi­nian Traps and Ex­is­ten­tial Risks

  Relevant: Natural Selection Favors AIs over Humans

  universal optimization algorithm

  Evolution is not an optimization algorithm (this is a common misconception discussed in Okasha, Agents and Goals in Evolution).

AI Safety Newslet­ter #40: Cal­ifor­nia AI Leg­is­la­tion Plus, NVIDIA De­lays Chip Pro­duc­tion, and Do AI Safety Bench­marks Ac­tu­ally Mea­sure Safety?

The Bit­ter Les­son for AI Safety Research

• Dan HAug 2, 2024, 3:27 PM

  3 points

  0

  on: Un­learn­ing via RMU is mostly shallow

  We have been working for months on this issue and have made substantial progress on it: Tamper-Resistant Safeguards for Open-Weight LLMs

  General article about it: https://​​www.wired.com/​​story/​​center-for-ai-safety-open-source-llm-safeguards/​​