Vaniver

Karma: 40,685

Vaniver Jul 25, 2025, 10:50 PM
20 points
3
on: HPMOR: The (Probably) Untold Lore
Hermione’s mask does not, so far as I noticed, move to Dumbledore.
What about the mask of
captured person offstage to be rescued?

Vaniver Jul 19, 2025, 10:26 PM
9 points
0
in reply to: the gears to ascension’s comment on: Comment on “Four Layers of Intellectual Conversation”
I expect there are plenty of possible and actual critiques whose responses should include a sentence like “it would have been a more useful critique if the author had read my point properly”
This reminded me of @transhumanist_atom_understander’s commentary on tumblr about the Smalley-Drexler debate:
Smalley, though, didn’t read Nanosystems. I’m pretty sure of this. I don’t think you can tell from his Scientific American article. But it becomes pretty clear in the “debate”. Drexler wrote an open letter to Smalley, and Smalley’s response includes this revealing paragraph:
...
So when I say that Smalley’s objections are at least addressed (convincingly or not) in Nanosystems, I don’t infer that Smalley must have read this and made the objections anyway. He didn’t read it.

Vaniver Jul 19, 2025, 2:53 PM
12 points
2
on: IMO challenge bet with Eliezer
https://x.com/alexwei_/status/1946477742855532918

This doesn’t count yet for the standards of the bet, I think, but this seems pretty close to “EY wins” to me? (The manifold market has jumped up to 93%.)

Vaniver Jul 19, 2025, 1:08 AM
2 points
0
on: Critic Contributions Are Logically Irrelevant
At the beginning of your post, you talk about “the value” of comments in a way that seems like it’s purely connected to their information content. Why not view them as speech acts?

Vaniver Jul 19, 2025, 12:56 AM
2 points
0
in reply to: Benquo’s comment on: Three Missing Cakes, or One Turbulent Critic?
I think I agree with your statement; I assume that this happened, though? Or, at least, in a mirror of the ‘improvements visible from the outside’ comment earlier, the question is whether MIRI is now operating in a way that leads to successfully opposing their adversaries, rather than whether they’ve exposed their reasoning about this to the public.

Vaniver Jul 18, 2025, 3:58 PM
9 points
2
in reply to: Benquo’s comment on: Three Missing Cakes, or One Turbulent Critic?
I can’t comment on why you weren’t invited [to the CFAR postmortem], because I was not involved with the decision-making for who would be invited; I just showed up to the event. Naively, I would’ve guessed it was because you didn’t work at CFAR (unless you did and I missed it?); I think only one attendee wasn’t in that category, for a broad definition of ‘work at’.
I have to rate all the time spent that didn’t result in improvements visible from the outside as nothing but costs paid to sustain internal narcissistic supply
This seems fair to me.
The uniformly positive things I’ve heard about “Don’t Create the Torment Nexus II: If Anyone Builds It, Everyone Dies” implies not much in the way of new perspective or even consensus that one is needed.
I think the main difference between MIRI pre-2022 and post-2022 is that pre-2022 had much more willingness to play along with AI companies and EAs, and post-2022 is much more willing to be openly critical.
There are other differences, and also I think we might be focusing on totally different parts of MIRI. Would you care to say more about where you think there needs to be new perspective?

Vaniver Jul 16, 2025, 4:42 AM
6 points
0
in reply to: Benquo’s comment on: Three Missing Cakes, or One Turbulent Critic?
Hmm. I HMCFed after that, I think, but I don’t remember why I didn’t talk much about it publicly. (Also I think there was a CFAR postmortem that I don’t recall getting written up and discussed online, tho there was lots of in-person discussion.)

Vaniver Jul 12, 2025, 2:03 AM
4 points
0
in reply to: MalcolmOcean’s comment on: IABIED: Advertisement design competition
It does in #1 but not #4--I should’ve been clearer which one I was referring to.

Vaniver Jul 10, 2025, 5:33 PM
7 points
1
in reply to: Mikhail Samin’s comment on: IABIED: Advertisement design competition
I like “this is not a metaphor”.
I think referring to Emmett as “former OpenAI CEO” is a stretch? Or, like, I don’t think it passes the onion test well enough.

Vaniver Jul 10, 2025, 5:17 PM
8 points
0
in reply to: Karl von Wendt’s comment on: IABIED: Advertisement design competition
However, I still think it is a good idea to create a sense of urgency, both in the ad and in books about AI safety.
Personally, I would rather stake my chips on ‘important’ and let urgent handle itself. The title of the book is a narrow claim—if anyone builds it, everyone dies—with the clarifying details conveniently swept into the ‘it’. Adding more inferential steps makes it more challenging to convey clearly and more challenging to hear (since each step could lose some of the audience).
There’s some further complicated arguments about urgency—you don’t want to have gone out on too much of a limb about saying it’s close, because of costs when it’s far—but I think I most want to make a specialization of labor argument, where it’s good that the AI 2027 people, who are focused on forecasting, are making forecasting claims, and good that MIRI, who are focused on alignment, are making alignment difficulty / stakes claims.
I see your point about how a weak claim can water down the whole story. But if I could choose between a 100 people convinced that ASI would kill us all, but with no sense of urgency, and 50 or even 20 who believe both the danger and that we must act immediately, I’d choose the latter.
Hmm I think I might agree with this value tradeoff but I don’t think I agree with the underlying prediction of what the world is offering us.
I think also MIRI has tried for a while to recruit people who can make progress on alignment and thought it was important to start work now, and the current push is on trying to get broad attention and support. The people writing blurbs for the book are just saying “yes, this is a serious book and a serious concern” and not signing on to “and it might happen in two years”—tho probably some of them also believe that—and I think that gives enough cover for the people who are acting on two-year timelines to operate.

Vaniver Jul 8, 2025, 6:34 AM
5 points
0
in reply to: habryka’s comment on: The Cult of Pain
And ultimately the only thing that matters here is power consumption,
Why? I think this is measuring exterior temperature, not the average of exterior and interior temperature. If cooling is set to a comfortable temperature and only run on heat wave days, then you should expect the heat wave days to also have a boost from the thermal mass of interior temperature, and there could be other indirect effects.
[Like, I would buy that power consumption dominates. But the only thing? Seems premature.]
I would be surprised if AC ends up more than 50% of power consumption
It does in Texas during heat waves (focusing only on peak demand, which seems fair). Texas is, of course, hotter than Europe (and places even hotter than Texas have even higher cooling costs).

Vaniver Jun 29, 2025, 4:54 PM
17 points
10
on: Support for bedrock liberal principles seems to be in pretty bad shape these days
I think I have a somewhat different diagnosis.
For example, take ‘property rights’. As a category, this mixes together lots of liberal and illiberal things: houses, hammers, and taxi medallions are all ‘property’ but the first two are productive capital and the last one is a pretty different form of capital. I’d go so far as to say NIMBYism is mostly downstream of an expansive view of property rights—my ownership of my house is not just the volume and physical objects on it, but also more indirect things like the noises and smells that impinge on it and the view out from it.
I think the core problem for classical liberalism in the 2020s is something like “figuring out a modern theory of regulation”. That is, increased population density has increased the indirect costs of action (more people now see and are inconvenienced by your ugly building) and increased economic sophistication has increased a bunch of burdens (more complicated varieties of products require more complicated regulations) but the main answers for how to deal with this have come from anti-liberals. Like, consider Wolf Ladejinski, who helped influence land reform in Asia because he understood the popularity of communism came from (largely correct!) hatred of landlords, and free enterprise also does not like landlords strangling the economy. I think the returns to figuring out things like this are pretty high, and am moderately optimistic about ‘abundance’ types managing to do a similar thing, but I think there’s still lots of fertile ground here.

Vaniver Jun 27, 2025, 7:44 PM
4 points
0
in reply to: johnswentworth’s comment on: johnswentworth’s Shortform
Did you ever try Circling? I wonder some if there’s a conversational context that’s very “get to the interesting stuff” which would work better for you. (Or, even if it’s boring, it might be because it’s foregrounding relational aspects of the conversation which are much less central for you than they are for most people.)

Vaniver Jun 23, 2025, 1:17 AM
6 points
2
in reply to: Valentine’s comment on: Consider chilling out in 2028
E.g., why did folk write AI 2027? Did they honestly think the timeline was that short?
Isn’t it more like “I think there’s a 10% chance of transformative AI by 2027, and that is like 100x higher than what it looks like most people think, so people really need to think thru that timeline”?
Like, I generally put my median year at 2030-2032; if we make it to 2028, the situation will still feel like “oh jeez we probably only have a few years left”, unless we made it to 2028 thru a mechanism that clearly blocks transformative AI showing up in 2032. (Like, a lot is hinging on what “feels basically like today” means.)

Vaniver Jun 22, 2025, 6:33 AM
16 points
0
in reply to: Lumpyproletariat’s comment on: New Endorsements for “If Anyone Builds It, Everyone Dies”
Done, we’ll see how it goes.

Vaniver Jun 20, 2025, 3:36 AM
78 points
12
in reply to: Malo’s comment on: New Endorsements for “If Anyone Builds It, Everyone Dies”
IMO the real story here for ‘how’ is “the book is persuasive to a general audience.” (People have made claims about the overton window shifting—and I don’t think there’s 0 of that—but my guess is the book would have gotten roughly the same blurbs in 2021, and maybe even 2016.)
But the social story for how is that I grew up in the DC area, one of my childhood friends is the son of an economist who is not a household name but is prominent enough that all of the household name economists know him. (This is an interesting position to be in—I feel like I’m sort of in this boat with the rationality community.) We play board games online every week and so when the blurb hunt started I got him an advance copy, and he was hooked enough to recommend it to Ben (and others, I think).
(I share this story in part b/c I think it’s cool, but also because I think lots of people are ~2 hops away from some cool people and could discover this with a bit of thought and effort.)
What links here?
- A case for courage, when speaking of AI danger by So8res (Jun 27, 2025, 2:15 AM; 508 points)
- AI #123: Moratorium Moratorium by Zvi (Jul 3, 2025, 3:40 PM; 33 points)

Vaniver Jun 19, 2025, 5:49 PM
30 points
5
in reply to: quetzal_rainbow’s comment on: New Endorsements for “If Anyone Builds It, Everyone Dies”
I’ve suggested a pathway or two for this; if you have independent pathways, please try them / coordinate with Rob Bensinger about trying them.

Vaniver Jun 11, 2025, 6:28 AM
9 points
2
in reply to: habryka’s comment on: Mikhail Samin’s Shortform
I think Anthropic leadership should feel free to propose a plan to do something that is not “ship SOTA tech like every other lab”. In the absence of such a plan, seems like “stop shipping SOTA tech” is the obvious alternative plan.
Note that Anthropic, for the early years, did have a plan to not ship SOTA tech like every other lab, and changed their minds. (Maybe they needed the revenue to get the investment to keep up; maybe they needed the data for training; maybe they thought the first mover effects would be large and getting lots of enterprise clients or w/e was a critical step in some of their mid-game plans.) But I think many plans here fail once considered in enough detail.

Vaniver Jun 11, 2025, 6:24 AM
23 points
14
in reply to: Thane Ruthenis’s comment on: Mikhail Samin’s Shortform
I think more than this, when you look at the labs you will often see the breakthru work was done by a small handful of people or a small team, whose direction was not popular before their success. If just those people had decided to retire to the tropics, and everyone else had stayed, I think that would have made a huge difference to the trajectory. (What does it look like if Alec Radford had decided to not pursue GPT? Maybe the idea was ‘obvious’ and someone else gets it a month later, but I don’t think so.)

Vaniver Jun 11, 2025, 6:22 AM
26 points
10
in reply to: Neel Nanda’s comment on: Mikhail Samin’s Shortform
I do think there’s a Virtue of Silence problem here.
Like—I was a ML expert who, roughly ten years ago, decided to not advance capabilities and instead work on safety-related things, and when the returns to that seemed too dismal stopped doing that also. How much did my ‘unilateral stopping’ change things? It’s really hard to estimate the counterfactual of how much I would have actually shifted progress; on the capabilities front I had several ‘good ideas’ years early but maybe my execution would’ve sucked, or I would’ve been focused on my bad ideas instead. (Or maybe me being at the OpenAI lunch table and asking people good questions would have sped the company up by 2%, or w/e, independent of my direct work.)
How many people are there like me? Also not obvious, but probably not that many. (I would guess most of them ended up in the MIRI orbit and I know them, but maybe there are lurkers—one of my friends in SF works for generic tech companies but is highly suspicious of working for AI companies, for reasons roughly downstream of MIRI, and there might easily be hundreds of people in that boat. But maybe the AI companies would only actually have wanted to hire ten of them, and the others objecting to AI work didn’t actually matter.)