(This is not your typical factual-answer question, but I think it makes sense to format this as a question rather than a post.)
TLDR: Recommend some posts for a “practice of rationality” sequence I want to curate! Proposing posts that should exist but don’t is also cool.
I’ve been thinking recently that it would be nice if rationality were more associated with a practice—a set of skills which you can keep grinding and leveling up. Testable rationality skills (like accuracy or calibration in forecasting) are obviously a plus, but I’m not referring exclusively to this—some very real things can be hard to evaluate externally, such as emotional wellness.
A model I have in mind is meditation: meditation is easy to “grind” because the meditator gets constant immediate feedback about how well they’re focusing (or at least, they get that feedback if they meet a minimum of focus required to keep track of whether they are focusing). Yet it’s quite difficult to evaluate progress from the outside.
(In fact, when I mentioned this desire for a “practice” of rationality to one friend, they were like “I agree, and in fact I think the practice should just be insight meditation.”)
This is basically reiterating Brienne’s call for tortoise skills (see also), except what I want to do is collect proposed things which could be part of a practice.
Obviously, some CFAR content could already qualify. CFAR doesn’t exactly teach it that way—as far as I’ve observed, CFAR’s focus is on mindset interventions. “Mindset intervention” is the fancy psychology term for getting someone to think differently by having them do something once. For example, the point of “growth mindset” interventions is that you explain it once and this has long-lasting impact on someone’s behavior. Another mindset intervention is: you ask people to write about what matters to them. Doing this once has shown long-term results.
In my first CFAR experience (which was an MSFP, fwiw), the phrase “It’s not about the exercises!” was kind of a motto. It was explained at the beginning that CFAR teaches exercises not because people learn the exercises and then go out and use the exercises, but rather, going through the exercises a few times changes how you think about things. (The story was that people often go to a CFAR workshop and then improve a bunch of things in their life, but say “but I haven’t been doing the exercises!”.)
But many of the things CFAR teaches could be used as a practice, and (again referring to my first CFAR experience) CFAR does do some things which encourage you to look at them that way, like the follow-up emails which encourage you to overlearn one exercise per week (practicing that one thing a bunch so that it becomes an automatic mental motion).
Another example pointing at what I want here is bewelltuned.com. The content may or may not be right, but the sort of thing seems exactly right to me—actionable skills you can keep working on regularly after getting simple explanations of how to do it. And furthermore, the presentation seems exactly right. LessWrong has a tendency to focus on wordy explanations of intellectual topics, which is great, but the bewelltuned style seems like an excellent counterbalance.
I’m using the “question” format so that answers can recommend specific things (perhaps represented by existing LW posts, perhaps not), whereas comments can discuss this more broadly (such as what more general criteria should be applied to filter suggestions, or whether this is even a good idea). The answer list here could serve as a big repository. I’ll probably create a sequence which can be my own highly opinionated curation of the suggestions here, plus my own writing on the subject.
I originally intended Becoming Unusually Truth Oriented to be the start of a sequence on the subject written entirely by me. However, some resulting discussion made me question my approach (hence the motivation for this question).
One friend of mine (going off of some of the discussion in comments to that post) voiced a concern about the rationality community falling into the same pitfalls as martial arts. Several articles about this have been written on LW. (I’m not finding all the ones I remember! If you put links to more of them in the comments I’ll probably edit this to add them.) The concern is that a martial art of rationality could lead to the same kinds of epistemic viciousness which are seen in literal martial arts—a practice divorced from reality due to the constraints and incentives of training/teaching.
That same friend suggested that the solution was to focus on empirically verifiable skills, namely forecasting. But in the in-person rationalist community in the bay area, I’ve encountered some criticism of extreme focus on forecasting which suggests that it’s making the very mistake we’re afraid of here—Goodharting on the problem. One person asked me to give any examples of Superforecasting-like skills resulting in actual accomplishments, suggesting that planning is the far more valuable skill and varies significantly from forecasting. Another person recounted their experience sitting down with several other rationalists to learn superforecasting skills. It was a group of rather committed and also individually competent rationalists, but they quickly came to the conclusion that while they could put in the effort to become much better at forecasting, the actual skills they’d learn would be highly specific to the task of winning points in prediction tasks, and they abandoned the project, concluding that it would not meaningfully improve their general capability to accomplish things!!
So, this seems like a hard problem.
What could/should be a part of a ‘practice’ of rationality?
I started writing out some notes on my current impressions of the “rationality skill tree”. Then I had a vague sense of having written it before. It turned out to be background thoughts on why doublecrux is hard to learn, which (surprise!) I also thought were key background skills for many other rationality practices.
I haven’t rewritten this yet to be non-double-crux-centric, but think that’d be good to do someday. (The LW team has been chatting about wikis lately, and this feels like something I’d eventually want written up in a way it could be easily collaboratively added to)
Getting oriented fast in complex/messy real world situations in fields in which you are not an expert
For example, now, one topic to get oriented in would be COVID; I think for a good thinker, it should be achievable to have big-picture understanding of the situation comparable to a median epidemiologist after few days of research
Where the point isn’t to get an accurate forecast of some global variable which is asked on metaculus, but gears-level model of what’s going on / what are the current ‘critical points’ which will have outsized impact / …
In my impression, compared to some of the ‘LessWrong-style rationality’, this is more heavily dependent on ‘doing bounded rationality well’ - that is, finding the most important bits / efficiently ignoring almost all information, in contrast to carefully weighting several hypothesis which you already have
Actually trying to change something in the world where the system you are interacting with has significant level of complexity & somewhat fast feedback loop (&it’s not super-high-stakes)
Few examples of seemingly stupid things of this type I did
filled a lawsuit without the aid of a lawyer (in low-stakes case)
repaired various devices with value much lower than value of my time
tinkering with code in a language I don’t know
trying to moderate Wikipedia article on highly controversial topic about which two groups of editors are fighting
One thing I’m a bit worried about in some versions of LW rationality & someone should write a post about is something like … ’missing opportunities to actually fight in non-super-high-stakes matters″, in the martial arts metaphor.
I would add active and empathic listening, and nonviolent communication. By improving our skills at communicating and connecting with others, we improve both our effectiveness in cooperation as well as the quality of our relationships.
+1 exploring a technical topic with another person involves a lot of soft skills, and this might be one of the anti correlations that makes dramatic progress rare.
Prediction
Abram pointed out concerns about focusing rationality of prediction. I agree with those concerns, and have said before that many of the skills involved in prediction can be Goodharted well past the point of usefulness more generally. For example, frequently updating, tracking source of information to quickly capture when a question should have been considered resolved, or tracking the group aggregate are all effective strategies that are minimally helpful for other parts of rationality.
On the other hand, to argue via analogy, while martial arts are clearly too narrowly focused on form, and adapted to constraints rather than optimizing for winning, the best mixed martial artists, and I suspect many of the best hand-to-hand fighters in general, are experts in one or several martial arts. That’s because even if the practice of any martial art was Goodharted well past the point of usefulness, and waste time because of that, fighters still need all of the skills that martial arts teach.
Similarly, I think that the best rationalists will need to be really good forecasters. The return will drop as you optimize too much for forecasting, obviously, but I imagine that there’s a huge return on being in the top, say, 1% of humans at forecasting. That’s not an incredibly high bar, since most people are really bad at this. I’ll guess that the top 1% level would probably fall somewhere below the median of Good Judgement Open’s active participant rankings—but I think it’s worth having people interested in honing their rationality skills participate and improve enough to get to at least that level.
I nominate this thing johnswentworth did. In addition to the reasons he gives, I’ll add that being able to learn on your own, quickly, seems like a good skill to have, and related to (though maybe not the same thing as) rationality.
The usual caveats of “what do you mean by rationality?” seem likely to crop up immediately here. (i.e. epistemic vs instrumental). “Being able to form accurate beliefs” and “Being able to form good plans in confusing domains” seem like two main things you might want to train.
I think it’s plausible that superforecasting (and “forming accurate beliefs” in general) doesn’t lead to overwhelmingly great good life outcomes, but is still, like, a skill that is worth gaining for the same reason many other skills are: it’s valuable to other people, and you might get paid for it (either by being directly economically valuable, or longterm-public-good valuable so philanthropists would subsidize it).
How to Measure Anything seems to lay out one particular set of skills that fit at the intersection of epistemic and instrumental rationality. It doesn’t give “exercises” but I think is designed for an environment (i.e. making decisions for organizations) where you have a reasonable stream of actions+feedback loops, albeit on a slower timescale.
The Hammer Time sequence is the obvious LessWrong place to start.
My light review of the pedagogical literature suggests four things with large effect size: deliberate practice, test taking, elaboration of context (cross-linking knowledge), and teaching the material to others.
I also suspect debate would make the cut if tested. I think there’s too little of the good kind of fighting in a lot of discourse and I sort of blame California culture for not being a good influence here. I think the intuition of comparing to sparring is right as a sort of collaborative fight, also that fighting can and should be playful and exploratory. This is less scalable since it requires skill matched real time collaboration.
On the object level I’ll reiterate that we’re still failing to engage with korzybski’s assertion that we’d be radically less confused if we trained up in noticing type errors in language/representation.
More speculatively: most people are excessively tense most of the time. Example: right now check your brow, jaw, throat, shoulders, gut, pelvis. Given the interaction between physiology and mindset, and given the need for exploratory research, this winds up being of deceptive importance. Relaxation is a trainable skill.