LessWrong team member / moderator. I’ve been a LessWrong organizer since 2011, with roughly equal focus on the cultural, practical and intellectual aspects of the community. My first project was creating the Secular Solstice and helping groups across the world run their own version of it. More recently I’ve been interested in improving my own epistemic standards and helping others to do so as well.
Raemon
Karma: 55,955
FYI this was an April Fools joke.
Curated. I’ve been following this project for awhile (you can see some of the earlier process in Daniel’s review of his “What 2026 looks like” post, and on his comment on Tom Davidon’s What a Compute-centric framework says about AI takeoff). I’ve participated in one of the wargames that helped inform what sort of non-obvious things might happen along the path of AI takeoff.
(disclosure, Lightcone did a lot of work on the website of this project, although I was only briefly involved)
Like others have said, I appreciate this for both having a lot of research behind it, and for laying out something concrete enough to visualize and disagree with. Debating individual “event X will happen” predictions isn’t exactly the point, since some of them are merely illustrative of “something similar that might happen.” But, it’s helpful for debating underlying models about what sort-of-events are likely to happen.
One of the central, obvious debates here is “does it actually make sense to just extrapolate the trends the way this way, or is AGI takeoff dependent on some unrelated progress?”. Recent posts like A Bear Case and Have LLMs Generated Novel Insights?[1] have argued the opposite view). I lean towards “the obvious trends will continue and the obvious AGI approaches will basically work”, but only put it at bit over 50%. I think it’s reasonable to have a lower credence there. But one thought I’ve had this week is: perhaps longer-time-folk (with some credence on this) should to spend the next year-or-so focusing more on plans that help in short-timeline worlds, and then return to longer time-horizon plans if a year from now, it seems like progress has slowed and there’s some missing sauce.[2]
I think it would have been nicer if a third scenario was presented – I think the current two-scenario setup comes across as more of a rhetorical device, i.e. “if y’all don’t change your actions you will end up on the doomy racing scenario.” I believe Daniel-et-al that that wasn’t their intent, but I think a third scenario that highlighted some orthogonal axis of concern would have been helpful for getting people into the mindset of actually “rolling the simulation forward” rather than picking and arguing for a side.
- ^
Notably, written before AI 2027 came out, although I think they were reacting to an intellectual scene that was nontrivially informed by earlier drafts of it.
- ^
On the other hand, if most of your probability-mass is on mediumish timelines, and you have a mainline plan you think you could barely pull off in 10 years, such that taking a year off seems likely to make the difference,
- ^
Yeah I introduced Baba is You more as a counterbalance to empiricism-leaning fields. I think ‘practice forms of thinking that don’t come natural to you’ is generally valuable so you don’t get in ruts.
I’m curious how it went in terms of ‘do you think you learned anything useful?’
I do think the thing you describe here is great. I think I hadn’t actually tried really leveraging the current zeitgeist to actively get better at it, and it does seem like a skill you could improve at and that seems cool.
But I’d bet it’s not what was happening for most people. I think the value-transfer is somewhat automatic, but most people won’t actually be attuned to it enough. (might be neat to operationalize some kind of bet about this, if you disagree).
I do think it’s plausible, if people put more deliberate effort it, to create a zeitgeist where the value transfer is more real for more people.
A thing that gave me creeping horror about the Ghiblification is that the I don’t think the masses actually particularly understand Ghibli. And the result is an uneven simulacrum-mask that gives the impression of “rendered with love and care” without actually being so.
The Ghibli aesthetic is historically pretty valuable to me, and in particular important as a counterbalanacing force against “the things I expect to happen by default with AI.”
Some things I like about Ghibli:
The “cinematic lens” emphasizes a kind of “see everything with wonder and reverence” but not in a way that papers over ugly or bad things. Ugliness and even horror are somehow straightforwardly depicted, but in a way that somehow makes both seem very normal and down to earth, and also supernaturally majestic. (See On green, and The Expanding Moral Cinematic Universe).
The main characters are generally “low-ish neuroticism.” (This youtube analysis I like argues that the women in particular are ‘non-neurotic’, and the men tend to be “low compared to modern city-dwelling standards.”)
There’s a bit of awkwardness where Miyazaki is particularly anti-transhumanist, where I disagree with him. But I feel like I could argue with him about it on his terms – I have an easy time imagining how to depict spirits of technology and capitalism and bureaucracy as supernatural forces that have the kind of alien grandeur, not on humanity’s side or the “natural world’s side”, but still ultimately part of the world.
For years, I have sometimes walked down the street and metaphorically put on “Miyazaki goggles”, where I choose to lean into a feeling of tranquility, and I choose to see everything through that “normal but reverent” stance. I imagine the people that live in each house doing their day to day things to survive and make money and live life. And seeing the slightly broken down things (a deteriorating fence, a crumbling sidewalk) as part of a natural ebb and flow of the local ecosystem. And seeing occasional more “naturally epic” things as particularly majestic and important.
So, the wave of “ghiblify everything” was something I appreciated, and renewed a felt-desire to live more often in a ghibli-ish world. But, also, when I imagine how this naturally plays out, I don’t think it really gets us anything like a persistent reality transfer the way you describe. Mostly we get a cheap simulacra that may create some emotion / meaning at first, but will quickly fade into “oh, here’s another cheap filter.”
...
That all said, I do feel some intrigue at your concept here. I’m still generally wrapping my mind around what futures are plausible, and then desirable. I feel like I will have more to say about this after thinking more.
Seems at odds with longhairism.
Curated. I think this is a pretty important point. I appreciate Neel’s willigness to use himself as an example.
I do think this leaves us with the important followup questions of “okay, but, how actually DO we evaluate strategic takes?”. A lot of people who are in a position to have demonstrated some kind of strategic awareness are people who are also some kind of “player” on the gameboard with an agenda, which means you can’t necessarily take their statements at face value as an epistemic claim.
I think I agree with a lot of stuff here but don’t find this post itself particularly compelling for the point.
I also don’t think “be virtuous” is really sufficient to know “what to actually do.” It matters a lot which virtues. Like I think environmentalism’s problems wasn’t “insufficiently virtue-ethics oriented”, it’s problem was that it didn’t have some particular virtues that were important.
Or: when the current policy stops making sense, we can figure out a new policy.
In particular, when the current policy stops making sense, AI moderation tools may also be more powerful and can enable a wider range of policies.
I mean, the sanctions are ‘if we think your content looks LLM generated, we’ll reject it and/or give a warning and/or eventually delete or ban.’ We do this for several users a day.
That may get harder someday but it’s certainly not unenforceable now.
I agree it’ll get harder to validate, but I think having something like this policy is, like, a prerequisite (or at least helpful grounding) for the mindset change.
Curated. I think figuring out whether and how we can apply AI to AI safety is one of the most important questions, and I like this post for exploring this through many more different angles than we’d historically seen.
A thing I both like and dislike about this post is that it’s more focused on laying out the questions than giving answers. This makes it easier for me the post to “help me think it through myself” (rather than just telling me a “we should do X” style answer).
But it lays out a dizzying enough array of different concerns that I found it sort of hard to translate this into “okay what actually should I actually think about next?”. I’d have found it helpful if the post ended with some kind of recap of “here’s the areas that seem most important to be tracking, for me.”
(note: This is Raemon’s random take rather than considered Team Consensus)
Part of the question here is “what sort of engine is overall maintainable, from a moderation perspective?”.
LLMs make it easy for tons of people to be submitting content to LessWrong without really checking whether it’s true and relevant. It’s not enough for a given piece to be true. It needs to be reliably true, with low cost to moderator attention.
Right now, basically LLMs don’t produce anywhere near good enough content. So, presently, letting people submit AI generated content without adding significant additional value is a recipe for LW admins to spend a bunch of extra time each day deciding whether to moderate a bunch of content that we’re realistically going to say “no” to.
(Some of the content is ~on par with the bottom 25% of LW content, but the bottom 25% of LW content is honestly below the quality bar we prefer the site to be at, and the reason we let those comments/posts in at all is because it’s too expensive to really check if it’s reasonable, and when we’re unsure, we sometimes to default to “let it in, and let the automatic rate limits handle it”. But, the automated rate limits would not be sufficient to handle an influx of LLM slop)
But, even when we imagine content that should theoretically be “just over the bar”, there are secondorder effects of LW being a site with a potentially large amount of AI content that nobody is really sure if it’s accurate or whether anyone endorses it and whether we are entering into some slow rolling epistemic disaster.
So, my guess for the bar for “how good quality do we need to be talking about for AI content to be net-positive” is more at least top-50% and maybe top-25% of baseline LW users. And when we get to that point probably the world looks pretty different.
My lived experience is that AI-assisted-coding hasn’t actually improved my workflow much since o1-preview, although other people I know have reported differently.
It seems like my workshops would generally work better if they were spaced out over 3 Saturdays, instead of crammed into 2.5 days in one weekend.
This would give people more time to try applying the skills in their day to day, and see what strategic problems they actually run into each week. Then on each Saturday, they could spend some time reviewing last week, thinking about what they want to get out of this workshop day, and then making a plan for next week.
My main hesitation is I kind of expect people to flake more when it’s spread out over 3 weeks, or for it to be harder to find 3 Saturdays in a row that work as opposed to 1 full weekend in a row.
I also think there is a bit of a special workshop container that you get when there’s 3 days in a row, and it’s a bit sad to lose that container.
But, two ideas I’ve considered so far are:
Charge more, and people get a partial refund if they attend all three sessions.
Have there be 4 days instead of 3, and design it such that if people miss a day it’s not that big a deal.
I’ve also been thinking about a more immersive-program experience, where for 3-4 weeks, people are living/working onsite at Lighthaven, mostly working on some ambitious-but-confusing project, but with periodic lessons and checkins about practical metastrategy. (This is basically a different product than “the current workshop”, and much higher commitment, but it’s closer to what I originally wanted with Feedbackloop-first Rationality, and is what I most expect to actually work)
I’m curious to hear what people think about these.
Also, have you tracked the previous discussion on Old Scott Alexander and LessWrong about generally “mysterious straight lines” being a surprisingly common phenomenon in economics. i.e. On an old AI post Oli noted:
This is one of my major go-to examples of this really weird linear phenomenon:
150 years of a completely straight line! There were two world wars in there, the development of artificial fertilizer, the broad industrialization of society, the invention of the car. And all throughout the line just carries one, with no significant perturbations.
This doesn’t mean we should automatically take new proposed Straight Line Phenomena at face value, I don’t actually know if this is more like “pretty common actually” or “there are a few notable times it was true that are drawing undue attention.” But I’m at least not like “this is a never-before-seen anomaly”)
I think it’s also “My Little Pony Fanfics are more cringe than Harry Potter fanfics, and there is something about the combo of My Little Pony and AIs taking over the world that is extra cringe.”
I’m here from the future trying to decide how much to believe in and how common are Gods of Straight Lines, and curious if you could say more arguing about this.
FYI I think there are a set of cues that move you from ‘pretty unlikely to be interested’ to ‘maybe interested’, but not that get you above like 25% likely.