Ben Pace

• Ben Pace 4 Apr 2026 6:25 UTC

  6 points

  1

  in reply to: Domenic’s comment on: “You Have Not Been a Good User” (LessWrong’s sec­ond album)

  I am so happy to read you enjoyed Dance of the Doomsday Clock! I have rarely been able to stomach the dozens of hours Habryka spends to iterate and iterate and iterate on lyrics and genre and generations. Both of my songs that made it (this and Litany of Gendlin from the first album) were me thinking about what I wanted for a little while and ~largely one-shotting it.

  Doomsday Clock really makes me laugh, it’s so melodramatic. I always imagine faux-Petrov turning to the audience and delivering Eliezer’s (sincere) line: “Wherever you are, whatever you’re doing, take a minute to not destroy the world.”

Lighthaven Se­quences Read­ing Group #76 (Tues­day 3/​31)

• Ben Pace 26 Mar 2026 19:06 UTC

  2 points

  0

  in reply to: Mateusz Bagiński’s comment on: 5 Things I Learned After 10 Days of Inkhaven

  This time I have purchased a slightly larger ballpit.

Lighthaven Se­quences Read­ing Group #75 (Tues­day 3/​24)

• Ben Pace 19 Mar 2026 18:39 UTC

  15 points

  0

  in reply to: Michaël Trazzi’s comment on: Michaël Trazzi’s Shortform

  I’ll be there!

• Ben Pace 18 Mar 2026 20:23 UTC

  5 points

  0

  in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Per­l­man’s Shortform

  I got a lot of my impression of credit assignment from Abram’s Post “The Credit Assignment Problem”, which opens with lots of examples:

  I’ve occasionally said “Everything boils down to credit assignment problems.”

  What I really mean is that credit assignment pops up in a wide range of scenarios, and improvements to credit assignment algorithms have broad implications. For example:

  • Politics.

  • • When politics focuses on (re-)electing candidates based on their track records, it’s about credit assignment. The practice is sometimes derogatorily called “finger pointing”, but the basic computation makes sense: figure out good and bad qualities via previous performance, and vote accordingly.

    • When politics instead focuses on policy, it is still (to a degree) about credit assignment. Was raising the minimum wage responsible for reduced employment? Was it responsible for improved life outcomes? Etc.

  • Economics.

  • • Money acts as a kind of distributed credit-assignment algorithm, and questions of how to handle money, such as how to compensate employees, often involve credit assignment.

    • In particular, mechanism design (a subfield of economics and game theory) can often be thought of as a credit-assignment problem.

  • Law.

  • • Both criminal law and civil law involve concepts of fault and compensation/​retribution—these at least resemble elements of a credit assignment process.

  • Sociology.

  • • The distributed computation which determines social norms involves a heavy element of credit assignment: identifying failure states and success states, determining which actions are responsible for those states and who is responsible, assigning blame and praise.

  • Biology.

  • • Evolution can be thought of as a (relatively dumb) credit assignment algorithm.

  • Ethics.

  • • Justice, fairness, contractualism, issues in utilitarianism.

  • Epistemology.

  • • Bayesian updates are a credit assignment algorithm, intended to make high-quality hypotheses rise to the top.

    • Beyond the basics of Bayesianism, building good theories realistically involves identifying which concepts are responsible for successes and failures. This is credit assignment.

  Another big area which I’ll claim is “basically credit assignment” is artificial intelligence.

  He also says:

  I’ve said that classifier systems faced a credit assignment problem. What does that mean, exactly?

  The definition I want to use for this essay is:

  • you’re engaged in some sort of task;

  • you use some kind of strategy, which can be broken into interacting pieces (such as a set of rules, a set of people, a neural network, etc);

  • you receive some kind of feedback about how well you’re doing (such as money, loss-function evaluations, or a reward signal);

  • you want to use that feedback to adjust your strategy.

  So, credit assignment is the problem of turning feedback into strategy improvements.

  Personally I consider breaking credit assignment to be breaking the epistemology of the system, which is an epistemic sin, and often parasitic. I think honorable people should not mislead people or whatever system they’re in about credit assignment.

• Ben Pace 18 Mar 2026 20:02 UTC

  3 points

  0

  in reply to: JohnofCharleston’s comment on: LessOn­line ticket sales are live! (Ear­ly­bird pric­ing un­til April 7)

  Is unchanged! I should add an FAQ to that effect, thanks for the ping.

LessOn­line ticket sales are live! (Ear­ly­bird pric­ing un­til April 7)

• Ben Pace 17 Mar 2026 23:26 UTC

  11 points

  8

  on: The New LessWrong LLM Policy is Worse Than You Think

  1.) Boilerplate:

  For sections of posts that are more about the craft of writing itself instead of ideas (Introduction, Conclusion), having an LLM expand upon a template saves a bunch of time while not really changing the message much.

  In my experience, LLMs are pretty good at filling in an argument if you give it a decent amount to work with

  In discussions of such things, I would like to request examples of this from folks proposing that AIs are equally good or better at writing than humans. My life as a LW mod is filled with reading slop/​bullshit, so flat assertions to the contrary are not persuasive to me.

  What links here?

  • papetoast's comment on pa­petoast’s Shortforms by papetoast (12 Mar 2026 3:45 UTC; 0 points)

• Ben Pace 15 Mar 2026 23:57 UTC

  4 points

  1

  in reply to: Raemon’s comment on: lilkim2025′s Shortform

  I was on the home page (www.lesswrong.com) when I got it.

• Ben Pace 14 Mar 2026 22:22 UTC

  12 points

  12

  in reply to: Adele Lopez’s comment on: New LessWrong Edi­tor! (Also, an up­date to our LLM policy.)

  I certainly don’t trust the daily deluge of new users who have this in their posts yet are substantially producing slop.

Lighthaven Se­quences Read­ing Group #74 (Tues­day 3/​17)

• Ben Pace 9 Mar 2026 20:48 UTC

  1 point

  3

  in reply to: clone of saturn’s comment on: Pro­mot­ing en­mity and bad vibes around AI safety

  If someone is currently attempting murder, it’s not reasonable to look for mutually beneficial arrangements with them, they need to be restrained and put in prison.

  This is locally true, but also I want to point out similar situations where the reasoning isn’t quite the same:

  1. If you’re at war with a country, of course you’re behaving as though “they need to be restrained and put in prison” or otherwise met with force. But I think often diplomacy and occasionally finding mutually-beneficial treaties is a good thing to do as well.

  2. Often, political leaders are pushing for and against different policies on topics like war, terrorism, healthcare, foreign aid and more. Changes in policies here definitely cause deaths, often in their thousands (e.g. reallocating money from current life-saving efforts into R&D can look like this on paper), but it’s critical to the health of a nation for people to be able to disagree and talk about these things and advocate for different positions without facing force or violence, even if you know that a certain idea will have horrendous consequences.

  I’m not open to peaceful coexistence with people who insist on building something that will likely destroy the human race.

  I myself would support laws imprisoning people selling out humanity and getting rich in the process (by building AGI via ML scaling), but I think violence is a really important civilizational schelling line and very dangerous to cross, and, while I admit I am still confused about the lines here, I am uncomfortable reading this. Your comment pattern matches to me with a pretty naive escalation across ethical lines, rather than a considered and mature one. You’re vaguely threatening violence almost reactively, rather than making any case that it would improve things or that this is one of the rare times where it’s okay to cross the line. This rhymes for me with a childish, impotent desire to have power over people who are doing terrible things, which is understandable, but doesn’t just any means of doing it.

Lighthaven Se­quences Read­ing Group #73 (Tues­day 3/​10)

• Ben Pace 6 Mar 2026 23:01 UTC

  7 points

  0

  on: Thoughts on the Pause AI protest

  “Pause AI! ’Cause we don’t wanna die!” Even better.

  I have a vague recollection I might’ve been the one who came up with that, at I think one of the early Pause AI protests in SF with Liron.

  Of course, it’s simple enough that perhaps it was reinvented.

• Ben Pace 4 Mar 2026 21:56 UTC

  5 points

  6

  in reply to: gbtw’s comment on: Saul Munn’s Shortform

  Gosh this is important context.

• Ben Pace 4 Mar 2026 6:48 UTC

  15 points

  0

  on: Lie To Me, But At Least Don’t Bullshit

  Following the oath of Bene Gesserit Truthsayers won’t get you lie-detection powers

  FWIW I do get a lot of mileage out of noticing when someone says something that, in their shoes, I could never imagine myself saying. In the act of trying to say it, I stop myself, for it trips over heuristics I have of “How could I even know that if it were true?” or “I feel like I’m slipping something past someone here.”

• Ben Pace 4 Mar 2026 5:39 UTC

  6 points

  3

  in reply to: Thomas Kwa’s comment on: OpenAI em­ploy­ees: Now is the time to stop do­ing good work.

  When hired by an employer, we agree to do certain work in exchange for compensation, not to optimize for the employer’s interests or what the CEO thinks the employer’s interests are.

  Actually it’s common for great companies to have visions that the people believe in that they make part of the hiring and onboarding process, and explicitly label and talk through (e.g. SpaceX’s “we’re going to Mars” or Stripe’s “Increase the GDP of the internet”). I think this is good, and I strongly expect that this is part of the culture Altman has set at OpenAI, so I expect it is much more of an implicit agreement there than it is if you (say) work at a restaurant as a waiter.

  There are many equilibriums about how much people expect others to believe in the company that they work at. I guess I am coming at this from a culture where people work at a place because they believe in it, and I think this is a better equilibrium.

  It’s something of an empirical question how good the existing companies are and how feasible it is to only work at places where you believe you’re improving the world. But it does seem to me that, if you’re at OpenAI but think it’s harmful to the world, you can just leave and make decent money elsewhere, I don’t think anyone is particularly trapped at the job there.

• Ben Pace 4 Mar 2026 5:19 UTC

  2 points

  0

  in reply to: Thomas Kwa’s comment on: OpenAI em­ploy­ees: Now is the time to stop do­ing good work.

  I may be over-reading into the list in the OP.

  • Did you see a bug? No you didn’t.

  • OpenAI leadership says vibe coding is fine, so why review AI code? (You can pretend to spend time on it if you want tho.)

  • Are you annoyed by unnecessary meetings? Why? Just relax.

  • Unplugged cable somewhere? Water leak? Not your job.

  • Lots of bad programmers have succeeded by spending their time on office politics instead. Office politics is a valuable skill! You should get some practice with it!

  The first one struck me as dishonest, but I think it’s fair to read the main thrust as “don’t do work you don’t have to”.

• Ben Pace 4 Mar 2026 1:12 UTC

  3 points

  3

  in reply to: bhauth’s comment on: OpenAI em­ploy­ees: Now is the time to stop do­ing good work.

  That seems like a legit point! It’s just very important to include information like that in your post when you’re proposing corporate sabotage. In political and murky domains where there’s so much noise and adversarial spreading of false narratives, it’s really scary to see posts that just say “I’m sure we all agree <thing> is bad, now let us do underhanded and dishonorable things to attack <thing>”. You can write that independent of whether something is good or bad, and instead write it whenever something is unpopular. It’s both more truth-tracking and less scary to see someone write “I encourage you to do underhanded things to <thing> because of <list of underhanded things that we know they did>.”

  You open the post with a list of things that, while bad, are at best reason to quit and protest the company, not reasons to be dishonorable. This section, as far as I can tell, was about as much as you spent actually justifying corporate sabotage:

  And then, Sam Altman had the nerve to say this:

  1. There is more open debate than I thought ther ewould be, at least in this part of Twitter, about whether we should prefer a democratically elected government or unelected private companies to have more power. I guess this is something people disagree on, but…I don’t. This seems like an important area for more discussion.

  Let’s be clear: this was not about Anthropic telling the US military not to work on autonomous weapons on its own. Altman is advocating for the government being able to require private companies (and their employees) to provide whatever services it wants, even if they don’t currently do that thing. I know the term “fascism” has been thrown around a lot, but that is Actual Fascism. Here are some other ways to use that argument:

  • “Why should Sam Altman decide what should be done with that billion dollars instead of the government, which reflects the will of the people?”

  • “Why should a private citizen get to decide they don’t want to spy on their neighbors and report any hidden jews? That should be the decision of the government, which reflects the will of the people!”

  This jump to ‘fascism’ is just cheap. Are you aware that Altman has repeatedly and publicly stated:

  In my conversations over the weekend, I reiterated that Anthropci should not be designated as a SCR, and that we hope the DOW offers them the same terms we’ve agreed to.

  Insofar as him endorsing the government’s threat against a private company is the ‘fascism’ you’re accusing, he has spoken out against it. You may wish to make some argument about why these words are not representative of the actions he will take, but instead you decided it was fine to label someone ‘Actual Fascism’ with capitals. Please hold yourself to higher standards than this.

  The biggest problem that I have with the post is that quitting your job is considered a bigger deal than giving up on being an honorable person. Seems very far out from what I consider good behavior. I myself have gone and protested outside of OpenAI due to them racing to develop AGI while being well aware that this poses an extinction-level threat to humanity, so I know is quite possible to oppose a company without acting dishonorably in the process. If you work at OpenAI and no longer believe in the ethics of the company, you can just do the decent thing and quit.