Vaniver

Karma: 40,302

Vaniver Apr 1, 2025, 7:22 PM
20 points
0
in reply to: aphyer’s comment on: Rafael Harth’s Shortform
The grass that can be touched is not the true grass.

Vaniver Apr 1, 2025, 6:11 PM
46 points
0
on: LessWrong has been acquired by EA
What convinced me this made sense?
- One of EA’s most popular and profitable games is The Sims, which famously benefits from Sim irrationality. In The Sims 5, there will be bold and new exciting ways for your Sims to behave, and they’ll be able to use our memetic virality model to have controversies and factional alignment. (Generating scissor statements is ethical so long as you’re doing it in Simlish.)
- EA is investing in the hypothesis that bad writing drives underperformance. Having ratfic writers and philosophers look at Mass Effect 3 could have turned that from a disappointing series-ender (did you play Andromeda?) to a resounding triumph, and Dragon Age: Veilguard, despite being positively reviewed in general, was panned for its weak writing and became inflamed in culture war controversy. We’ve thought a lot about how misbehaving gods would act, in a way that I think would have made for a more compelling story and user experience.
- I didn’t expect we could do anything relating to EA’s flagship sports games (FIFA, NHL, Madden, etc.), but what astonished me was the potential to do the reverse. I don’t know if we’ll be able to get Gwern 2025 out in time, but look forward to Gwern 2026. They were practically salivating at the idea of being able to take a normally annual product, tied to sports schedules that won’t be adjusted by advancing AI progress, and adapt it to a domain which, as part of an overall hyperbolic growth curve, will generate enough new content for a new release in ~half the time every new release.

Vaniver Mar 20, 2025, 4:02 AM
2 points
0
in reply to: JesperO’s comment on: The Failed Strategy of Artificial Intelligence Doomers
The short version is they’re more used to adversarial thinking and security mindset, and don’t have a culture of “fake it until you make it” or “move fast and break things”.
I don’t think it’s obvious that it goes that way, but I think it’s not obvious that it goes the other way.

Vaniver Mar 16, 2025, 5:02 AM
7 points
0
on: Help make the orca language experiment happen
This project is extremely neglected, since normal people don’t seriously consider whether orcas might be that smart.
Ok, but matters is not what normal people are doing, but what specialists are doing. Why not try to do this as part of Project CETI?

Vaniver Feb 15, 2025, 4:56 AM
2 points
0
on: Celtic Knots on a hex lattice
It looks like you only have pieces with 2 connections and 6 connections, which works for maximal density. But I think you need some slack space to create pieces without the six axial lines. I think you should include the tiles with 4 connections also (and maybe even the 0-connection tile!) and the other 2-connection tiles; it increases the number by quite a bit but I think will let you make complete knots.

Vaniver Feb 3, 2025, 6:45 PM
2 points
0
in reply to: Alexander Gietelink Oldenziel’s comment on: Alexander Gietelink Oldenziel’s Shortform
I haven’t thought deeply about this specific case, but I think you should consider this like any other ablation study—like, what happens if you replace the SAE with a linear probe?

Vaniver Feb 3, 2025, 7:39 AM
4 points
0
in reply to: habryka’s comment on: The Failed Strategy of Artificial Intelligence Doomers
And then a lot of the post seems to make really quite bad arguments against forecasting AI timelines and other technologies, doing so with… I really don’t know, a rejection of bayesianism? A random invocation of an asymmetric burden of proof?
I think the position Ben (the author) has on timelines is really not that different from Eliezer’s; consider pieces like this one, which is not just about the perils of biological anchors.
I think the piece spends less time than I would like on what to do in a position of uncertainty—like, if the core problem is that we are approaching a cliff of uncertain distance, how should we proceed?--but I think it’s not particularly asymmetric.
[And—there’s something I like about realism in plans? If people are putting heroic efforts into a plan that Will Not Work, I am on the side of the person on the sidelines trying to save them their effort, or direct them towards a plan that has a chance of working. If the core uncertainty is whether or not we can get human intelligence advancement in 25 years—I’m on your side of thinking it’s plausible—then it seems worth diverting what attention we can from other things towards making that happen, and being loud about doing that.]

Vaniver Feb 3, 2025, 7:30 AM
15 points
8
on: The Failed Strategy of Artificial Intelligence Doomers
Instead, the U.S. government will do what it has done every time it’s been convinced of the importance of a powerful new technology in the past hundred years: it will drive research and development for military purposes.
I think this is my biggest disagreement with the piece. I think this is the belief I most wish 10-years-ago-us didn’t have, so that we would try something else, which might have worked better than what we got.
Or—in shopping the message around to Silicon Valley types, thinking more about the ways that Silicon Valley is the child of the US military-industrial complex, and will overestimate their ability to control what they create (or lack of desire to!). Like, I think many more ‘smart nerds’ than military-types believe that human replacement is good.

Vaniver Feb 3, 2025, 7:25 AM
4 points
0
in reply to: jimrandomh’s comment on: The Failed Strategy of Artificial Intelligence Doomers
The article seems to assume that the primary motivation for wanting to slow down AI is to buy time for institutional progress. Which seems incorrect as an interpretation of the motivation. Most people that I hear talk about buying time are talking about buying time for technical progress in alignment.
I think you need both? That is—I think you need both technical progress in alignment, and agreements and surveillance and enforcement such that people don’t accidentally (or deliberately) create rogue AIs that cause lots of problems.
I think historically many people imagined “we’ll make a generally intelligent system and ask it to figure out a way to defend the Earth” in a way that I think seems less plausible to me now. It seems more like we need to have systems in place already playing defense, which ramp up faster than the systems playing offense.

Vaniver Jan 16, 2025, 8:10 PM
4 points
0
in reply to: Zack_M_Davis’s comment on: Shutting Down the Lightcone Offices
My understanding is that the Lightcone Offices and Lighthaven have 1) overlapping but distinct audiences, with Lightcone Offices being more ‘EA’ in a way that seemed bad, and 2) distinct use cases, where Lighthaven is more of a conference venue with a bit of coworking whereas Lightcone Offices was basically just coworking.

Vaniver Jan 11, 2025, 6:04 AM
31 points
18
on: Human takeover might be worse than AI takeover
By contrast, today’s AIs are really nice and ethical. They’re humble, open-minded, cooperative, kind. Yes, they care about some things that could give them instrumental reasons to seek power (eg being helpful, human welfare), but their values are great
They also aren’t facing the same incentive landscape humans are. You talk later about evolution to be selfish; not only is the story for humans is far more complicated (why do humans often offer an even split in the ultimatum game?), but also humans talk a nicer game than they act (see construal level theory, or social-desirability bias). Once you start looking at AI agents who have similar affordances and incentives that humans have, I think you’ll see a lot of the same behaviors.
(There are structural differences here between humans and AIs. As an analogy, consider the difference between large corporations and individual human actors. Giant corporate chain restaurants often have better customer service than individual proprietors because they have more reputation on the line, and so are willing to pay more to not have things blow up on them. One might imagine that AIs trained by large corporations will similarly face larger reputational costs for misbehavior and so behave better than individual humans would. I think the overall picture is unclear and nuanced and doesn’t clearly point to AI superiority.)
though there’s a big question mark over how much we’ll unintentionally reward selfish superhuman AI behaviour during training
Is it a big question mark? It currently seems quite unlikely to me that we will have oversight systems able to actually detect and punish superhuman selfishness on the part of the AI.

Vaniver Dec 3, 2024, 12:45 AM
4 points
0
in reply to: Czynski’s comment on: (The) Lightcone is nothing without its people: LW + Lighthaven’s big fundraiser
I think it’s hard to evaluate the counterfactual where I made a blog earlier, but I think I always found the built-in audience of LessWrong significantly motivating, and never made my own blog in part because I could just post everything here. (There’s some stuff that ends up on my Tumblr or w/e instead of LW, even after ShortForm, but almost all of the nonfiction ended up here.)

Vaniver Nov 2, 2024, 12:35 AM
12 points
2
in reply to: habryka’s comment on: JargonBot Beta Test
Consider the reaction my comment from three months ago got.

Vaniver Oct 28, 2024, 6:41 PM
6 points
0
in reply to: Elizabeth’s comment on: Why I quit effective altruism, and why Timothy Telleen-Lawton is staying (for now)
I think being a Catholic with no connection to living leaders makes more sense than being an EA who doesn’t have a leader they trust and respect, because Catholicism has a longer tradition
As an additional comment, few organizations have splintered more publicly than Catholicism; it seems sort of surreal to me to not check whether or not you ended up on the right side of the splintering. [This is probably more about theological questions than it is about leadership, but as you say, the leadership is relevant!]

Vaniver Sep 7, 2024, 2:24 AM
33 points
21
on: Perhaps Try a Little Therapy, As a Treat?
I don’t think Duncan knows what “a boundary” is.
General Semantics has a neat technology, where they can split out different words that normally land on top of each other. If boundary_duncan is different from boundary_segfault, we can just make each of the words more specific, and not have to worry about whether or not they’re the same.
I’ve read thru your explainer of boundary_segfault, and I don’t see how Duncan’s behavior is mismatched. It’s a limit that he set for himself that defines how he interacts with himself, others, and his environment. My guess is that the disagreement here is that under boundary_segfault, describing you as having “poor boundaries” is saying that your limits are poorly set. (Duncan may very well believe this! Tho the claim that you set them for yourself makes judging the limits more questionable. )
That said, “poor boundaries” is sometimes used to describe a poor understanding or respect of other people’s boundaries. It seems to me like you are not correctly predicting how Duncan (or other people in your life!) will react to your messages and behavior, in a way that aligns with you not accurately predicting their boundaries (or predicting them accurately, and then deciding to violate them anyway).
This isn’t something that I do. This is something that I have done
I don’t understand this combination of sentences. Isn’t he describing the same observations you’re describing?
There is a point here that he’s describing it as a tendency you have, instead of an action that happened. But it sure seems like you agree that it’s an action that happened, and I think he’s licensed to believe that it might happen again. As inferences go, this doesn’t seem like an outlandish one to make.
The friends who know me well know that I am a safe person. Those who have spent even a day around me know this, too!
The comments here seem to suggest otherwise.
You talk about consent as being important to you; let’s leave aside questions of sexual consent and focus just on the questions: did Duncan consent to these interactions? Did Duncan ask you to leave him alone? Did you leave him alone?

Vaniver Jul 26, 2024, 4:03 PM
36 points
50
in reply to: Kaj_Sotala’s comment on: Universal Basic Income and Poverty
I wasn’t sure what search term to use to find a good source on this but Claude gave me this:
I… wish people wouldn’t do this? Or, like, maybe you should ask Claude for the search terms to use, but going to a grounded source seems pretty important to staying grounded.

Vaniver Jun 17, 2024, 8:15 PM
3 points
0
in reply to: Ebenezer Dukakis’s comment on: Ebenezer Dukakis’s Shortform
I think Six Dimensions of Operational Adequacy was in this direction; I wish we had been more willing to, like, issue scorecards earlier (like publishing that document in 2017 instead of 2022). The most recent scorecard-ish thing was commentary on the AI Safety Summit responses.
I also have the sense that the time to talk about unpausing is while creating the pause; this is why I generally am in favor of things like RSPs and RDPs. (I think others think that this is a bit premature / too easy to capture, and we are more likely to get a real pause by targeting a halt.)

Vaniver May 31, 2024, 2:08 AM
LW: 6 AF: 2
−2
AF
on: There Should Be More Alignment-Driven Startups!
While the coauthors broadly agree about points listed in the post, I wanted to stick my neck out a bit more and assign some numbers to one of the core points. I think on present margins, voluntary restraint slows down capabilities progress by at most 5% while probably halving safety progress, and this doesn’t seem like a good trade. [The numbers seem like they were different in the past, but the counterfactuals here are hard to estimate.] I think if you measure by the number of people involved, the effect of restraint is substantially lower; here I’m assuming that people who are most interested in AI safety are probably most focused on the sorts of research directions that I think could be transformative, and so have an outsized impact.

There Should Be More Alignment-Driven Startups

Vaniver, Judd Rosenblatt, Cameron Berg and phgubbins

May 31, 2024, 2:05 AM

62 points

14 comments11 min readLW link

Vaniver May 15, 2024, 6:35 PM
2 points
0
in reply to: Jeffrey Heninger’s comment on: Environmentalism in the United States Is Unusually Partisan
Similarly for the Sierra Club, I think their transition from an anti-immigration org to a pro-immigration org seems like an interesting political turning point that could have failed to happen in another timeline.

Vaniver

There Should Be More Align­ment-Driven Startups

There Should Be More Alignment-Driven Startups