Ben Pace

Karma: 34,754

I’m an admin of LessWrong. Here are a few things about me.

I generally feel more hopeful about a situation when I understand it better.
I have signed no contracts nor made any agreements whose existence I cannot mention.
I believe it is good take responsibility for accurately and honestly informing people of what you believe in all conversations; and also good to cultivate an active recklessness for the social consequences of doing so.
It is wrong to directly cause the end of the world. Even if you are fatalistic about what is going to happen.

(Longer bio.)

Ben Pace Jun 12, 2025, 1:09 AM
2 points
0
in reply to: Said Achmiz’s comment on: β-redex’s Shortform
One answer is that people who genuinely have a hard time understanding other ppl are selected for heavily.
To be clear I regularly have a hard time understanding other people, including on the internet, I don’t think it’s that unusual a state of affairs to be explaining.

Ben Pace Jun 12, 2025, 12:57 AM
2 points
0
in reply to: MondSemmel’s comment on: Mikhail Samin’s Shortform
I make the case here for stopping based on deontological rather than consequentialist reasons.

Ben Pace Jun 12, 2025, 12:51 AM
2 points
0
in reply to: Said Achmiz’s comment on: β-redex’s Shortform
A couple of notes:
1. I think the position I most want to defend is that you can post something you know will get a lot of engagement, and it can also be true. I think when someone writes something like “I literally don’t understand why people choose to lie regularly in their daily lives” they might be aware that it could be a toxoplasma-kind of scissor-statement, and also they still don’t understand why people do it so much.
2. I think it’s a very common experience of writers that they have a very poor ability to predict what gets a lot of attention and what does not. I think this is true of Aella and her tweets, in that she is often confident and wrong, or very surprised. So I don’t expect that all of her popular twits are expected to be.

Ben Pace Jun 11, 2025, 11:13 PM
24 points
23
in reply to: Neel Nanda’s comment on: Mikhail Samin’s Shortform
I too believe we should ultimately judge things based on their consequences. I believe that having deontological lines against certain actions is something that leads humans to make decisions with better consequences, partly because we are bounded agents that cannot well-compute the consequences of all of our actions.
For instance, I think you would agree that it would be wrong to kill someone in order to prevent more deaths, today here in the Western world. Like, if an assassin is going to kill two people, but says if you kill one then he won’t kill the other, if you kill that person you should still be prosecuted for murder. It is actually good to not cross these lines even if the local consequentialist argument seems to check out. I make the same sort of argument for being first in the race toward an extinction-level event. Building an extinction-machine is wrong, and arguing you’ll be slightly more likely to pull back first does not stop it from being something you should not do.

I think when you look back at a civilization that raced to the precipice and committed auto-genocide, and ask where the lines in the sand should’ve been drawn, the most natural one will be “building the extinction machine, and competing to be first to do so”. So it is wrong to cross this line, even for locally net positive tradeoffs.
What links here?
- Ben Pace's comment on Mikhail Samin’s Shortform by Mikhail Samin (Jun 12, 2025, 12:57 AM; 2 points)

Ben Pace Jun 11, 2025, 11:03 PM
4 points
−6
in reply to: Said Achmiz’s comment on: β-redex’s Shortform
Actually, I confidently believe she (and other autistic rationalists) who ask these questions often are earnestly confused, and do not easily empathize with the other minds on these topics. I think there’s still selection effects on why such people’s comments are selected for, but it is not conscious “engagement-bait”.

Ben Pace Jun 11, 2025, 2:41 AM
2 points
−4
in reply to: Neel Nanda’s comment on: Mikhail Samin’s Shortform
I’m not saying that it’s implausible that the consequences might seem better. I’m stating it’s still morally wrong to race toward causing a likely extinction-level event as that’s a pretty schelling place for a deontological lines against action.

Ben Pace Jun 11, 2025, 1:18 AM
5 points
0
in reply to: sunwillrise’s comment on: Mikhail Samin’s Shortform
I am not quite sure what an overheating gun refers to, I am guessing the idea is that it has some chance of going off without being fired.
Anyhow, if that’s accurate, it’s acceptable to decide to be the person holding an overheating gun, but it’s not acceptable to (for example) accept a contract to assassinate someone so that you get to have the overheating gun, or to promise to kill slightly fewer people with the gun than the next guy. Like, I understand consequentially fewer deaths happen, but our society has deontological lines against committing murder even given consequentialist arguments, which are good. You’re not allowed to commit murder even if you have a good reason.

Ben Pace Jun 10, 2025, 6:37 PM
1 point
0
in reply to: MondSemmel’s comment on: Mikhail Samin’s Shortform
I just want to clarify that Anthropic doesn’t have the social authority of a governmental firing squad to kill people.

Ben Pace Jun 10, 2025, 6:35 PM
6 points
4
in reply to: sunwillrise’s comment on: Mikhail Samin’s Shortform
Just because someone else is going to kill me, doesn’t mean we don’t have an important societal norm against murder. You’re not allowed to kill old people just because they’ve only got a few years left, or kill people with terminal diseases.

Ben Pace Jun 10, 2025, 4:17 PM
3 points
3
in reply to: Neel Nanda’s comment on: Mikhail Samin’s Shortform
If you are killing me and my friends because you think it better that you do the killing than someone else, then actually I will still ask you to stop, because I draw a hard line around killing me and my friends. Naturally, I have a similar line around developing tech that will likely kill me and my friends.

Ben Pace May 30, 2025, 11:26 PM
7 points
0
in reply to: Czynski’s comment on: Czynski’s Shortform
I heard someone (who I respect) say why they don’t post on LessWrong more. They said when they talk about their thoughts and ideas with their friends, the friends won’t question their basic frame / sanity, and so won’t undermine their trust in themself. They said that this is acceptable on LessWrong, which is a more uncomfortable experience.
Ever since then I’ve tried a bit harder to make sure to question my friends’ basic frames and sanity, so that I’m not encouraging self-blinding in the people around me, and so they will know that they’re welcome to do the same to me.

Ben Pace May 30, 2025, 5:06 AM
9 points
4
on: CFAR is running an experimental mini-workshop (June 2-6, Berkeley CA)!
Yay! I’m happy to see this happening. I would strongly consider going if I wasn’t running LessOnline immediately before it (after which I will both be exhausted AND on call for festival-season work).

Ben Pace May 29, 2025, 2:53 PM
29 points
17
in reply to: Joseph Miller’s comment on: Zac Hatfield Dodds’s Shortform
I’d guess it looks more stable to investors. Unlike if you have a bunch of EAs on the OpenAI board and they confusingly try to fire the CEO for as unimportant a crime as lying; that’s quite hard for investors to predict.

Ben Pace May 26, 2025, 3:48 AM
LW: 15 AF: 6
10
AF
in reply to: ryan_greenblatt’s comment on: ryan_greenblatt’s Shortform
I think the main thing I want to convey is that I think you’re saying that LWers (of which I am one) have a very low opinion of the integrity of people at Anthropic, but what I’m actually saying that their integrity is no match for the forces that they are being tested with.
I don’t need to be able to predict a lot of fine details about individuals’ decision-making in order to be able to have good estimates of these two quantities, and comparing them is the second-most question relating to whether it’s good to work on capabilities at Anthropic. (The first one is a basic ethical question about working on a potentially extinction-causing technology that is not much related to the details of which capabilities company you’re working on.)

Ben Pace May 25, 2025, 8:05 PM
LW: 2 AF: 2
0
AF
in reply to: ryan_greenblatt’s comment on: ryan_greenblatt’s Shortform
What’s an example decision or two where you would want to ask yourself whether they should get more or less open-ended power? I’m not sure what you’re thinking of.

Ben Pace May 25, 2025, 7:06 PM
LW: 11 AF: 5
9
AF
in reply to: ryan_greenblatt’s comment on: ryan_greenblatt’s Shortform
Not the main thrust of the thread, but for what it’s worth, I find it somewhat anti-helpful to flatten things into a single variable of “how much you trust Anthropic leadership to make decisions which are good from your perspective”, and then ask how optimistic/pessimistic you are about this variable.
I think I am much more optimistic about Anthropic leadership on many axis relative to an overall survey of the US population or Western population – I expect them to be more libertarian, more in favor of free speech, more pro economic growth, more literate, more self-aware, higher IQ, and a bunch of things.
I am more pessimistic about their ability to withstand the pressures of a trillion dollar industry to shape their incentives than the people who are at Anthropic.
I believe the people working there are siloing themselves intellectually into an institution facing incredible financial incentives for certain bottom lines like “rapid AI progress is inevitable” and “it’s reasonably likely we can solve alignment” and “beating China in the race is a top priority”, and aren’t allowed to talk to outsiders about most details of their work, and this is a key reason that I expect them to screw up their decision-making.
I am optimistic about their relative-ability to have a sensible conversation about the next 5 years and what alignment failures look like, relative to most people on earth. This is not the standard I require to expect people to not do ML training runs that lead to human extinction, but nonetheless I predict they will do relatively quite well on this axis.
I don’t have a single variable here, I have a much more complicated model than this. It looks to me that collapsing questions of trust about people or groups into a single varibale of how optimistic I am about them making decisions which are good from my values has been a common question-substitution in the Effective Altruism scene, where I think people have been repeatedly hoodwinked by sociopaths due to not moving toward a more detailed model that predicts exactly where and when someone will make good vs bad decisions.

Ben Pace May 21, 2025, 4:14 PM
4 points
0
in reply to: leogao’s comment on: leogao’s Shortform
(I affirm that I don’t believe you were being knowingly dishonest or deceptive at any point in this thread.)

Ben Pace May 20, 2025, 6:39 AM
4 points
0
in reply to: Veedrac’s comment on: leogao’s Shortform
But not in full generality! This is a fine question to raise in this context, but in general the correct thing to do in basically all situations is to consider the object level, and then also let yourself notice if people are unusually insane around a subject, or insane for a particular reason. Sometimes that is the decisive factor, but for all questions, the best first pass is to think about how that part of the world works, rather than to think about the other monkeys who have talked about it in the past.

Ben Pace May 20, 2025, 6:38 AM
2 points
0
in reply to: leogao’s comment on: leogao’s Shortform
ou’re I am confused why you think my claims are only semi related. to me my claim is very straightforward, and the things i’m saying are straightforwardly converying a world model that seems to me to explain why i believe my claim. i’m trying to explain in good faith, not trying to say random things. i’m claiming a theory of how people parse information, to justify my opening statement,
Thank you for all this. I still think your quick take is wrong on the matter of epistemology.
I acknowledge that you make a fine point about persuasion, that someone who is primarily running the heuristic that “claims about the end of the world are probably crack-pots or scammers” will not be persuaded by someone arguing that actually 20:1 against and 20:1 in favor of a claim are equally extreme beliefs.
A version of the quick take that I would’ve felt was just fine would read:
Some people have basically only heard claims of human extinction coming from crackpots and scammers, and will not have thought much about the AI extinction idea on the object level. To them, this sort of argument I’ve discussed is unpersuasive at moving beyond the “is this a crackpot/scam” part of the dialogue. In this quick take I’ll outline my model of how they’re thinking about it, and give recommendations for how you should argue instead.
But your quick take doesn’t confine itself to discussing those people in those situations. It flatly says it’s true as a matter of epistemology that you should “use bigness of claim as a heuristic for how much evidence you need before you’re satisfied”, that you should “use reference classes that have consistently made good decisions irl” and that the crackpots/scammers one is the correct one to use here otherwise you’ll risk “getting pwned ideologically”.
These aren’t always the right heuristics (e.g. on this issue they are not for you and for me) and you shouldn’t say that they are just so that some people on Twitter will stop using rhetoric that isn’t working.
I believe you’re trying to do your best to empathize with people who are unpersuaded by an unsuccessful rhetorical move, a move that people who believe your position are making in public discourse. That is commendable. I think you are attempting to cause other people who hold your position to stop using that rhetorical move, by telling them off for using it, but to acheive this aim you are repeatedly saying the people who do not hold your position are doing normatively correct epistemology, and you’re justifying it with Occam’s razor and reference class forecasting, and this is all wrong. In some situations for some people it is reasonable to primarily use theses heuristics, and in other situations for other people it is not. I’m not arguing that the people unpersuaded are being unreasonable, but (for example) your opening sentence makes fully-general statements about how to reason about this issue that I believe are false. Rule number of one of good discourse: don’t make false statements about epistemology in order to win an object level point.
Yep, seems fine to drop this here; I make no bid of you to reply further.

Ben Pace May 19, 2025, 1:35 AM
2 points
0
in reply to: leogao’s comment on: leogao’s Shortform
You’re writing lots of things here but as far as I can tell you aren’t defending your opening statement, which I believe is mistaken.
I claim it is a lot more reasonable to use the reference class of “people claiming the end of the world” than “more powerful intelligences emerging and competing with less intelligent beings” when thinking about AI x-risk. further, we should not try to convince people to adopt the latter reference class—this sets off alarm bells, and rightly so (as I will argue in short order) - but rather to bite the bullet, start from the former reference class, and provide arguments and evidence for why this case is different from all the other cases.
Firstly, it’s just not more reasonable. When you ask yourself “Is a machine learning run going to lead to human extinction?” you should not first say “How trustworthy are people who have historically claimed the world is ending?”, you should of course primarily bring your attention to questions about what sorts of machine is being built, what sort of thinking capacities it has, what sorts of actions it can take in the world, what sorts of optimization it runs, how it would behave around humans if it were more powerful than them, and so on. We can go back to discussing epistemology 101 if need be (e.g. “Hug the Query!”).
Secondly, insofar as someone believes you are a huckster or a crackpot, you should leave the conversation, communication here has broken down and you should look for other communication opportunities. However, insofar as someone is only evaluating this tentatively as one of many possible hypotheses about you then you should open yourself up to auditing / questioning by them about why you believe what you believe and your past history and your memetic influences. Being frank is the only way through this! But you shouldn’t say to them “Actually, I think you should treat me like a huckster/scammer/serf-of-a-corrupt-empire.” This feels analogous to a man on a date with a woman saying “Actually I think you should strongly privilege the hypothesis that I am willing to rape you, and now I’ll try to provide evidence for you that this is not true.” It would be genuinely a bad sign about a man that he thinks that about himself, and also he has moved the situation into a much more adversarial frame.
I suspect you could write some more narrow quick-take such as “Here is some communication advice I find helpful when talking with friends and colleagues about how AI can lead to human extinction”, but in generalizing it all the way to making dictates about basic epistemology you are making basic mistakes and getting it wrong.
Please either (1) defend and/or clarify the original statement, or (2) concede that it was mistaken, rather than writing more semi-related paragraphs about memetic immune systems.