Ben Pace

Lighthaven Se­quences Read­ing Group #25 (Tues­day 03/​11)

• Ben PaceMar 9, 2025, 5:20 PM

  2 points

  0

  in reply to: Guive’s comment on: Ben Pace’s Short­form Feed

  Edited, should be working fine now, thx!

• Ben PaceMar 9, 2025, 8:16 AM

  LW: 4 AF: 3

  0

  AF

  on: Ben Pace’s Short­form Feed

  Something a little different: Today I turn 28. If you might be open to do something nice for me for my birthday, I would like to request the gift of data. I have made a 2-4 min anonymous survey about me as a person, and if you have a distinct sense of me as a person (even just from reading my LW posts/​comments) I would greatly appreciate you filling it out and letting me know how you see me!

  Here’s the survey.

  It’s an anonymous survey where you rate me on lots of attributes like “anxious”, “honorable”, “wise” and more. All multiple-choice. Two years ago I also shared a birthday survey amongst people who know me and ~70 people filled it out, and I learned a lot from it. I am very excited to see how the perception of me amongst the people I know has *changed*, and also to find out how people on LessWrong see me, so the core of this survey is ~20 of the same attributes.

  In return for your kind gift, if you complete it, you get to see the aggregate ratings of me from last time!

  This survey helps me understand how people see me, and recognize my blindspots, and I’m very grateful to anyone who takes a few mins to complete it. Two people completed it already and it took them 2 mins and 4 mins to complete it. (There are many further optional questions but it says clearly when the main bit is complete.)

  I of course intend to publish the (aggregate) data in a LW post and talk about what I’ve learned from it :-)

• Ben PaceMar 7, 2025, 9:07 PM

  5 points

  2

  in reply to: Thane Ruthenis’s comment on: A Bear Case: My Pre­dic­tions Re­gard­ing AI Progress

  I’m never sure if it makes sense to add that clause every time I talk about the future.

• Ben PaceMar 7, 2025, 8:23 PM

  10 points

  0

  on: A Bear Case: My Pre­dic­tions Re­gard­ing AI Progress

  Curated. Some more detailed predictions of the future, different from others, and one of the best bear cases I’ve read.

  This feels a bit less timeless than many posts we curate but my guess is that (a) it’ll be quite interesting to re-read this in 2 years, and (b) it makes sense to record good and detailed predictions like this more regularly in the field of AI which is moving so much faster than most of the rest of the world.

• Ben PaceMar 4, 2025, 10:51 PM

  4 points

  1

  on: The Semi-Ra­tional Mili­tar Firefighter

  Thanks for this short story! I have so many questions.

  1. This was during training camp? How many days/​weeks/​month in was this?

  2. How many people went through this training camp with you? Are you still friends with any of them?

  3. How long was training, and then how long did you serve as a professional?

  4. I encourage any links you have to content about these folks in future stories. I had to check the Wikipedia page before I fully believed that the logo was a skull with knife through it.

  Yes, I would be interested in reading another story about your time there. This story gave me some insight into the context, I think I would like something heavier on the logos, but I’d be happy with either.

• Ben PaceMar 4, 2025, 6:38 PM

  2 points

  0

  in reply to: Huera’s comment on: Open Thread Spring 2025

  Intercom please! Helps for us to have back and forth like “What device /​ operating system /​ browser?” and other relevant q’s.

Lighthaven Se­quences Read­ing Group #24 (Tues­day 03/​04)

• Ben PaceMar 2, 2025, 2:39 AM

  2 points

  0

  in reply to: KvmanThinking’s comment on: help, my self image as ra­tio­nal is af­fect­ing my abil­ity to em­pathize with others

  That sounds good to me i.e. draft this post, and then make it a comment in one of those places instead (my weak guess is a quick take is better, but whatever you like).

• Ben PaceMar 2, 2025, 2:36 AM

  2 points

  0

  in reply to: KvmanThinking’s comment on: help, my self image as ra­tio­nal is af­fect­ing my abil­ity to em­pathize with others

  Posted either as a comment on the seasonal open thread or using the quick takes /​ shortform feature, which posts it in your shortform (e.g. here is my shortform).

  

  I’m saying that this seems to me not on the level of substance of a post, so it’d be better as a comment of one of the above two types, and also that it’s plausible to me you’d probably get more engagement as a comment in the open thread.

Open Thread Spring 2025

• Ben PaceMar 2, 2025, 2:27 AM

  2 points

  0

  on: help, my self image as ra­tio­nal is af­fect­ing my abil­ity to em­pathize with others

  FWIW this feels like it should be a shortform/​open thread than a post.

• Ben PaceFeb 28, 2025, 7:35 PM

  6 points

  0

  in reply to: niplav’s comment on: Will_Pear­son’s Shortform

  I have used my admin powers to put it into a collapsible section so that people who expand this in recent discussion do not have to scroll for 5 seconds to get past it.

• Ben PaceFeb 25, 2025, 10:24 PM

  4 points

  0

  in reply to: MondSemmel’s comment on: Ben Pace’s Short­form Feed

  Though if the text changes, then it degrades gracefully to just linking to the right webpage, which is the current norm.

• Ben PaceFeb 25, 2025, 9:00 PM

  21 points

  6

  on: Ben Pace’s Short­form Feed

  I have a general belief that internet epistemic hygiene norms should include that, when you quote someone, not only should you link to the source, but you should link to the highlight of that source. In general, if you highlight text on a webpage and right-click, you can “copy link to highlight” which when opened scrolls to and highlights that text. (Random example on Wikipedia.)

  Further on this theme, archive.is has the interesting feature of constantly altering the URL to point to the currently highlighted bit of text, making this even easier. (Example, and you can highlight other bits of text to see it change.) Currently I overall don’t like it because I constantly highlight text while I’m reading it, and so am v annoyed by the URL constantly changing, but it’s plausible I’d get over this in time, and it’d be a good feature to add to LW.

  The archive.is feature is also better because the normal “copy link to highlight” can often be unwieldily and long. Also I recall it sometimes not working, probably because the highlight is too short or too long (I don’t quite understand the rules). On archive.is it just has a start and end number for where in the text is highlighted, making it always work and never be unwieldily.

  Sadly, I just tried the normal “copy link to highlight” on LW, and when I clicked through the page auto-refreshes, so the highlighted text flashes purple then disappears quickly after. It would be good for us to change that, and maybe implement this feature.

• Ben PaceFeb 23, 2025, 6:53 AM

  17 points

  2

  in reply to: Holly_Elmore’s comment on: The Sorry State of AI X-Risk Ad­vo­cacy, and Thoughts on Do­ing Better

  The point that “small protests are the only way to get big protests” may be directionally accurate, but I want to note that there have been large protests that happened without that. Here’s a shoggoth listing a bunch, including the 1989 Tiananmen Square Protests, the 2019 Hong Kong Anti-Extradition Protests, the 2020 George Floyd Protests, and more.

  The shoggoth says spontaneous large protests tends to be in response to triggering events and does rely on pre-existing movements that are ready to mobilize, the latter of which your work is helping build.

Lighthaven Se­quences Read­ing Group #23 (Tues­day 02/​25)

• Ben PaceFeb 21, 2025, 7:28 AM

  33 points

  4

  on: Ben Pace’s Short­form Feed

  I want to contrast two perspectives on human epistemology I’ve been thinking about for over a year.

  There’s one school of thought about how to do reasoning about the future which is about naming a bunch of variables, putting probability distributions over them, multiplying them together, and doing bayesian updates when you get new evidence. This lets you assign probabilities, and also to lots of outcomes. “What probability do I assign that the S&P goes down, and the Ukraine/​Russia war continues, and I find a new romantic partner?” I’ll call this the “spreadsheets” model of epistemology.

  There’s another perspective I’ve been ruminating on which is about visualizing detailed and concrete worlds, in a similar way to if you hold a ball and ask me to visualize how it’ll fall if you drop it, that I can see the world in full detail. This is more about loading a full hypothesis into your head, and into your anticipations. It’s more related to Privileging the Hypothesis /​ Believing In /​ Radical Probabilism^[1]. I’ll call this the “cognitive visualization” model of epistemology.

  These visualizations hook much more directly to my anticipations and motivations. When I am running after you to remind you that you forgot to take your adderall today, it is not because I had a spreadsheet simulate lots of variables and in a lot of worlds the distribution said it was of high utility to you. I’m doing it because I have experienced you getting very upset and overwhelmed on many days when you forgot and those experiences flashed through my mind as likely outcomes that I am acting hurriedly do divert the future from. When I imagine a great event that I want to run, I am also visualizing a certain scene, a certain freeze frame, a certain mood, that I desire and I believe it is attainable, and I am pushing and pulling on reality to line it up so that it’s a direct walk from here to there.

  Now I’m going to say that these visualizations are working memory bottlenecked, and stylize that idea more than is accurate. Similar to the idea that there are only ~7 working memory slots in the brain^[2], I feel that for many important parts of my life I can only fit a handful of detailed visualizations of the future easily accessible to my mind to use to orient with. This isn’t true in full generality – at any time or day if you ask me to visualize what happens if you drop a ball, I have an immediate anticipation – but if you constantly ask me to visualize a world in great detail with the S&P 500 goes up and the war continues, versus down and the war stops, and lots of other permutations with other variables changed, then I start to get fatigued. And this is true for life broadly, that I’m only loading up so many detailed visualizations of specific worlds.

  Given this assumption – if indeed one perhaps only has a handful of future-states that one can load into one’s brain – the rules of how to update your beliefs and anticipations are radically changed from the spreadsheets model of epistemology. Adding a visualization to your mind means removing one of your precious few; this means you will be better equipped to deal with worlds like the one you’re adding, and less well-equipped to deal with worlds like the one you’ve removed. This includes both taking useful action and making accurate predictions; which ones you load into your mind are a function of accuracy and usefulness. It can help to add many worlds into your cognitive state that you wish to constantly fight to not happen, causing the pathways to those worlds to loom higher when making your predictions. Yet being in your mind is evidence that they will not happen because you are optimizing so. Alternatively, when it is very hard to achieve something, it is often good to load (in great detail) world states that you wish to move towards, such that with every motion and action you have checked whether it’s hewing in the direction of that world, and made the adjustments to the world as required.

  This model gives an explanation for why people who are very successful often say they cannot imagine failure. They have loaded into their brain the world they are moving toward, in great detail, and in every situation they are connecting it to the world they desire and making the adjustments to setup reality to move in the right way. It is sometimes actively unhelpful to constantly be comparing reality to lots of much worse worlds and asking yourself what actions you could take to make those more likely. My sense is that this mostly helps you guide reality toward those worlds.

  And yet, I value having true beliefs, and being able to give accurate answers to questions that aren’t predictably wrong. If I don’t load a certain world-model into my brain, or if I load a set of biased ones (which I undoubtedly will in the story where can only pick ~7), I may intuitively give inaccurate answers to questions. I think this is probably what happens in the startup founders who give inaccurately high probabilities of success – their head is filled entirely with cognitive visualizations of worlds that succeed and are focused on how to get there, relative to the person with the spreadsheet that is calculating all of the variables and optimizing for accuracy far above human-functionality.

  In contrast, when founders semi-regularly slide into despair, I think this is about adding a concrete visualization of total failure to their cognitive workhouse. Suddenly lots of the situations and actions you are in are filled with fear and pain as you see them moving toward a world you desire strongly not to be in. While it is not healthy to be constantly focused on asking yourself how you could make things worse and to notice those pathways, it is helpful to boot up that visualization sometimes in order to check that’s not what’s currently happening. I have personally found it is useful to visualize in detail what it would look like if I were to be acting very stupidly, or actively self-sabotaging, in order to later make sure I behave in ways that definitely don’t come close to that. Despair is a common consequence of booting up those perspectives.

  I am still confused about what exactly counts as a cognitive visualization – in some sense I’m producing hundreds of cognitive visualizations per day, so how could I be working memory bottlenecked? I also still have more to learn in the art of human rationality, of choosing when to change the set of cognitive visualizations to have loaded in at any given time, for which I cannot simply rely on Bayes’ theorem. For now I will say that I endeavor to be able to produce the spreadsheet answers, and to often use them as my betting odds, even while it is not the answer I get when I run my cognitive visualizations or where my mind is when I take actions. I endeavor to sometimes say “I literally cannot imagine this failing. Naturally, I give it greater than 1:1 odds that it indeed does so.”

  1. ^

     More specifically (and this will make sense later in this quick take) when you’re switching out which visualizations are in your working memory, the updates you make to your probabilities will decidedly not be Bayesian, but perhaps more like the fluid updates /​ Jeffrey updating discussed by Abram.

  2. ^

     I don’t really know what a “slot” means here, so I don’t know that “7″ meaningfully maps onto some discrete thing, but the notion that the brain has a finite amount of working memory is hard to argue with.

Ar­bital has been im­ported to LessWrong

• Ben PaceFeb 20, 2025, 12:31 AM

  LW: 16 AF: 6

  5

  AF

  in reply to: Buck’s comment on: How might we safely pass the buck to AI?

  Further detail on this: Cotra has more recently updated at least 5x against her original 2020 model in the direction of faster timelines.

  Greenblatt writes:

  Here are my predictions for this outcome:

  • 25th percentile: 2 year (Jan 2027)

  • 50th percentile: 5 year (Jan 2030)

  Cotra replies:

  My timelines are now roughly similar on the object level (maybe a year slower for 25th and 1-2 years slower for 50th)

  This means 25th percentile for 2028 and 50th percentile for 2031-2.

  The original 2020 model assigns 5.23% by 2028 and 9.13% | 10.64% by 2031 | 2032 respectively. Each time a factor of ~5x.

  However, the original model predicted the date by which it was affordable to train a transformative AI model. This is a leading a variable on such a model actually being built and trained, pushing back the date by some further number of years, so view the 5x as bounding, not pinpointing, the AI timelines update Cotra has made.