Let’s Rename Ourselves The “Metacognitive Movement”
Associations, Definitions, Prosody
The name “rationality movement,” or “rationalism,” has several problems. It sounds self-congratulatory. It’s the name of a 17th-century anti-empirical movement. It’s defined as “the quality of being based on or in accordance with reason or logic,” which seems to oppose intuition, tradition, and emotion, whereas we’re open to those sources of information.
Sometimes, people try to use names like “aspiring rationalist,” “rationalish,” or “LessWrong community.” None are great. “LessWrong” community doesn’t explain much to outsiders. “Aspiring rationalist” comes with the baggage of the word “rationalist.” And “rationalish” fails to speak in the positive.
“Metacognition” is defined as “thinking about thinking.” That’s exactly what we do. It’s untainted by history or politics, and intuitively seems more attractive and accessible to outsiders. It has a more humble vibe. And it’s as, if not more popular than the word “rationality” on Google Trends.
One review article defines metacognition as a combination of knowledge and regulation. Knowledge is about oneself as a learner, factors affecting performance, strategies for meta-cognition, and use cases for those strategies. Regulation is about self-monitoring one’s own thinking, including planning, self-awareness, and evaluation.
Mapping “rationality” words to “metacognition” words shows that metacognition is mostly either the same length, or shorter.
| “Rationality” word | ”Metacognition” word | Effect of switch on prosody |
| Rationalist | Meta-thinker | Same length. Less aggressive, more academic. Could be good or bad. |
| Rationalism, Rationality | Metacognition, Meta-Thinking, Meta-Thought | Same length or shorter. Uses a common word (“think”). |
| Rationalist Movement | Metacognitive Movement, Meta-Thinking Movement | Longer or same length. Creates alliteration. |
One further factor is that “rationalists” can be shortened to “rats,” which is nifty, though occasionally used/confused as an insult. I don’t have an equivalently cute shortening for “meta-thinkers.” Maybe “cogs?”
Building Links To Scientific Literature
Metacognition has an evidence base, including findings that metacognition can be trained, causes performance improvements, and is independent of IQ. One review article (below) also links metacognition to “cumulative culture,” which seems to be a term for “cultural transmission of knowledge,” i.e. The Secret of Our Success.
I’ve linked below to a range of popular books, textbooks, and scientific review articles on metacognition. Among the popular books, some of them are foundational to our movement (Thinking, Fast and Slow, Antifragile, The Elephant In The Brain), while others are new and might be valuable. There is a note about how to get the scientific review articles without institutional access.
This lets us interface more easily with literature and professionals working on our interests. Being able to import and export useful concepts from the academic literature has always been one of the motors of this movement.
One potential disadvantage of using the term “metacognitive movement” is that it might be too specific. After all, most people don’t associate “metacognition” with our community’s interest in economics, X-risk, politics, statistics, and computation. On the other hand, “rationalism” doesn’t do an obviously better job of pointing to those issues. And on the positive side, this lets us understand what we’re doing as an intersection between metacognition and a variety of other academic disciplines, which helps explain why it’s a productive area for original insights.
But It’s Already Our Name!
We’ve already been using the “rationality” or “rationalist” movement for a while. It has brand name recognition. At this point, probably far more of the people we talk to associate “rationalist movement” as pointing to us as opposed to an antiquated 17th-century philosophical movement.
Paul Graham thinks that companies should be more open to changing their name, especially if someone else owns the .com URL. All the one-word .com URLs related to “rational” and “metacognition” are taken. More generally, if what we’re trying to do is set ourselves apart from other movements and academic trends—an argument I’ll dig into below—it might be better to name ourselves after an obscure movement and tendentious term, rather than a popular academic field.
Also, “rationalist” movement is only one of our names. We also use LessWrong, “rationalish,” “skeptics,” and probably others, not to mention the slew of institutional names associated with effective altruism.
But if “metacognitive movement” is a better name, then we should consider adopting it. If we can’t solve that coordination problem, then I’m worried about our ability to solve the more important ones.
“Metacognition” Might Be Too Specific Or Too General
Our community and movement isn’t just about adopting academic metacognitive techniques used to teach grade schoolers. It’s a particular approach to primarily adult metacognition. We focus on statistics, X-risk, politics, moral frameworks, and scientific skepticism and interpretation, partly because we think these are important general meta-cognitive considerations. We also are a niche community that attracts a certain type of mind, from a certain type of social, cultural, and economic background.
If we want to preserve the particularity of our movement, as well we might want to resist a name-change that connects us with an academic literature that is both broader and narrower than our own concerns. If that were desirable, we could get more specific with terms like “X-risk-related metacognition,” “Yudkowskian metacognition,” or “Bay Area metacognition.” But those are longer, and might be unattractive names for other reasons.
By setting ourselves apart from mainstream education-focused research on metacognition, we create a sort of intellectual island effect. This promotes a more unique evolution of our own ideas (though I wonder if it more often promotes insular dwarfish or gigantism).
However, I also suspect that many people in this community see themselves as interested in thinking about thinking in a more general sense. Many of us might actually be able to benefit greatly from the basic metacognitive training that the standard academic literature focuses on. Some might see a name that links us to a more mainstream academic and social world as attractive.
Crux: “Rationalism” Is An Activist Term
Many in this community are focused not just on developing their personal metacognitive skills, but on an activist agenda that includes X-risk mitigation and social/political issues. “Rationalism” could be a useful name for this agenda precisely because of its signaling and status implications, the whiff of superiority that some people see in it. Maybe the name is meant to suggest, “these are urgent issues, and if you’re not working on them, you’re not being rational.”
The reticence we feel in some cases about calling ourselves “rationalist” might reflect a genuine discomfort with the same. But maybe terms like “aspiring rationalist” and “rationalish” are better understood as a tactical move in certain situations, a way of making ourselves small when need be.
It helps define an outward-facing activist movement, rather than an inward-facing cultural community. Poetically, it sounds more aggressive. “Rational” inner-rhymes with “slash,” and sounds more energetic than the academic-sounding “metacognitive.” If the issues are as serious as we tend to think they are, and if the name is doing work to convince people of that, then more power to it. If I had to give one reason why this name change should fail, this would be it.
Could “Metacognitive Movement” be better than “Rationalist Movement” as an activist term?
First, renaming things doesn’t always change their associations much. That’s a bad thing if the name change is to escape a bad reputation. But it’s good if you’re trying to preserve a reputation, as we would be here. Renaming ourselves might help us export ideas to the mainstream, not just be taken over by it. Since the idea of promoting metacognition is pretty popular both inside and outside academia, mainstreaming these ideas could usefully push the Overton window on ideas like EA and X-risk and free up some weirdness points.
List Of Metacognition Literature
Popular Books
Goodreads and Amazon both have plenty of books on metacognition. Some are familiar to this community, while others aren’t discussed much.
Textbooks (in order of publication year)
Cognition, Metacognition, and Culture in STEM Education: Learning, Teaching and Assessment (2017): “We explore theoretical background and cutting-edge research about how various forms of cognitive and metacognitive instruction may enhance learning and thinking in STEM classrooms from K-12 to university and in different cultures and countries.” $17, eBook
Improving Reading Comprehension through Metacognitive Reading Strategies Instruction (2016): “This book addresses the need to help all students, including English learners, improve their ability to read with understanding so that they can succeed not just in their language and literacy classes, but also in their subject area classrooms.” $46 eBook, $75-$100 paper book
Metacognition, 1st edition (Dunlosky and Metcalfe, 2009): “Metacognition is the first textbook to focus on people′s extraordinary ability to evaluate and control their cognitive processes. This comprehensive text covers both theoretical and empirical metacognitive research in educational, developmental, cognitive and applied psychology.” ~$60 (eBook, paper book)
Applied Metacognition (2002): “This study overviews the relationship between theories in metacognition and their real-world applications. In addition to a theoretical overview, chapters cover metacognition in three areas: education, everyday life memory and in diverse populations.” $21 on Amazon
Emotional Disorders and Metacognition: Innovative Cognitive Therapy (2002): “The clinical experience of cognitive therapies is adding to the understanding of emotional disorders. Based on clinical experience and evidence, this groundbreaking book represents a development of cognitive therapy through the concept of metacognition.” $90, paperback
Metacognition in Learning and Instruction (Hartman, 2001): “Contributions by leading experts and others to understanding the crucial role of metacognition in relation to broad areas of education make this collection a uniquely stimulating book. It encompasses metacognition in both the neglected area of teaching and the more well-established area of learning.” $120 eBook, $160 paper book
Metacognition (Metcalfe and Shimamura, 1996, MIT press): “Metacognition offers an up-to-date compendium of major scientific issues involved in metacognition. The twelve original contributions provide a concise statement of theoretical and empirical research on self-reflective processes or knowing about what we know.” ~$35 (soft cover)
Selected scientific review articles (ordered by year)
Note: www.sci-hub.se is a wonderful, convenient pirate website for the academic literature your tax dollars paid to produce and publish. None of the links below directly link to sci-hub, but if you paste the URLs of the closed-access articles into the sci-hub.se search bar, you should be able to download them.
34 pages, closed-access
“Metacognition weakly correlated with both academic performance and intelligence, and that these relationships were moderated by the type of measurement of metacognition. Furthermore, it was found that metacognition predicted academic performance when controlling for intelligence.”
11 pages, open-access
Relates metacognition to cumulative culture, which is related to cultural transmission of knowledge (“the secret of our success”)
“Cumulative cultural evolution is the process by which cultural traits (including behaviours, artefacts and tools) change over multiple episodes of social transmission to become more effective and beneficial to their users.”
“In the current article, we examine the view, recently proposed by Heyes (2016) and Shea et al. (2014), that distinctively human cultural evolution is attributable to capacities for explicit (or Type/System 2) metacognition.”
Winne, P. H., & Azevedo, R. (2014). Metacognition.
25 pages, closed-access
Definition of metacognition, review of empirical work, future research directions.
41 pages, open-access
Knowledge and regulation
Knowledge is about: self, factors affecting performance, strategies, when and why to use strategies
Regulation is: self-monitoring (planning, awareness, evaluation)
Metacognition improves with instruction
Assessment
Challenges: construct complexity, unobservable, confounding (w/ verbal ability, working memory capacity), narrow/decontextualized measures
6 pages, closed-access
Reviews “theoretical and educational structure of the concept of metacognition,” determining the difference between cognition and metacognition, difficulties in measurement.
38 pages, closed-access
From the abstract: “I discuss… aspects of cognition about cognition and the results concerning them from the point of view of the psychologist and the computer scientist, and I attempt to place them in the context of computational theories.”
50 pages, open-access
Summarizing research on metacognition about reading, focusing on research from 1980-1988. Common research types, training studies, measurement challenges.
I have never liked the “rat” nickname. I’m not a filthy rodent. I’ve never heard the term “rationalish” before now.
I’ve always resisted using the term “rationalism”. I feel like “-ism” is a misstep into politics (and already the name of the 17th-century anti-empericists). We practice “epistemic rationality” and “instrumental rationality”, together, “rationality”, not “rationalism”.
I think the demonym for ‘metacognitive movement’ is ‘metacognizer’; I don’t think ‘meta-thinker’ is similar-looking enough.
I like ‘metacognizer’ enough that I would maybe be happy seeing it spring up now and then. But I don’t think it captures the core thing ‘rationalist’ does. Thinking about thinking is a certain strategy we use to achieve the goals of truth and winning, but we don’t want to get too wedded to that strategy at the expense of the goals. (E.g., maybe rationalists should sometimes deliberate and reflect less, and be more ‘in the moment’ or reflex-oriented; if so, naming ourselves ‘metacognizers’ could make it harder to update.) ‘Rationality’ is nice because ‘systematized accuracy and winning’ is closer to the goals.
I don’t like ‘rats’, but I think ‘cogs’ is probably even worse.
Some terms I think get at the Thing better:
rationalists
cognitive rationalists / cograts—disambiguates that we’re named after the cognitive-science jargon ‘rationality’, and aren’t e.g. the old philosophy movement
optimizists
epistemists / epistemicists
aspiring Bayesians
Bayesianists
expected utilitarians
error-reductionists
I think referencing Bayes in the name would be a mistake for the same reason as metacognition—it’s a tool, and it’s a law, but it’s not an end and it’s not The Thing.
Maybe if we were “error-reductionists,” we’d call ourselves ducks! I like the term as a fancy spin on Less Wrong, and I think it helps show that we include anything that lets us be less in error, including non-stereotypically “rational” forms of information.
On the other hand, it doesn’t point to action any better than “rationality” or “metacognition.” And it doesn’t suggest what you’d actually do, as a human being, in order to reduce your error more (a flaw in the term “rationality” as well).
Weirdly enough, for seeming like the most passive of the one-word options, “metacognition” is the only one that points to a specific and well-defined set of actionable teaching and learning techniques.
That still doesn’t mean it’s optimal. It might just mean that we’ve failed as a civilization (except at our paradise here on the blog!) to attach a set of actionable strategies to the word “rationality” or “error reduction.” So maybe we just need to correct that.
I imagine “metacogs” might gain more purchase than either “cogs” or “metacognizers” since the former is too short and silly and the latter is a mouthful to say, but “metacogs” strikes a pretty good balance.
I resisted using the word “rationalist” for a long time. But after posting lots of writing on Less Wrong I eventually needed a shorthand to explain what I’m doing to outsiders. Saying “I write on Less Wrong” doesn’t mean anything to someone who doesn’t know what Less Wrong is.
Anyone with a classical liberal education knows what rationalism is. It refers to the Enlightenment philosophers who felt that reason, as opposed to divine providence, was the chief source of knowledge.
Did rationalists differ from empiricists on the importance of data? Maybe but that isn’t important. The early Enlightenment philosophers were frequently wrong. I like the perspective of historical hindsight. It is a warning not to get too confident. Rationalism is watching slave ships from West Africa come into port, looking up at the stars and figuring out how to make a better world. With centuries of hindsight, what jumps into someone’s mind when you say “Rationalist” is the Enlightenment—a clash between science and the establishment. A collision between high ideals and muddy reality. Using an old name reminds us science is a centuries-long project.
I resisted using the word “community” for a long time too. The word “community” is overused by commercial interests attempting to manipulate people. But during over the COVID lockdown of this last year I’ve made more friends on Less Wrong than in meatspace. And the friends I’ve made are really valuable. Talking to quants I met on Less Wrong was the first time I’ve ever felt like I didn’t have to dumb down my ideas about computer science. I get better job offers here than I get at networking events.
So yeah, I’m part of the rationalist community.
Metacognitive Movement
There are two things I don’t like about “metacognitive movement”: the word “metacognitive” and the word “movement”.
The problem with “metacognitive” is it lacks the historical context of the word “rationalist”. “Rationalist” triggers the image of starry-eyed idealists living in a barbarous world. Metacognitive triggers the image of a yogi on a mountaintop or an academic in an ivory tower. I prefer the former. I’d rather be a rat than a cog.
The problem with “movement” is that Rationalism isn’t a movement. A movement is “group of people with a common ideology, esp a political or religious one” or “the organized action of such a group”. Less Wrong doesn’t have a common ideology. I don’t want it to have one. All ideologies are wrong in some way. Ideologies suppress dissent. Suppressing dissent, when you’re wrong, obstructs the pursuit of truth. Become less wrong is the opposite of following an ideology.
Are we a metacognitive movement? I don’t think so. Making a vaccine isn’t about cognition. It’s about action. Are we a movement? If we are then I ought to be kicked out because I frequently disagree with the prevailing ideology.
Kinda like it. But names are kind of sticky. But maybe it’s worth it. But I kinda like rationality. “Rational Agency” is kind of the core of the thing IMO.
If rational agency is supposed to be the core of the movement, then I think that should be our pitch. Right now, I think of rational agency as a niche subtopic within our broader focus on thinking how to think. I’ve never spent much time specifically investigating that topic, though of course I’m familiar with many of the concepts.
That said, it does seem helpful to clarify and find common ground by centering around “rational agency” as the through-line for what we do. I’ll spend more time investigating this and see if it feels like the reason I’m called to participate in this community.
I’m a bit surprised this came across as a niche subtopic (I feel like it’s covered a bunch in the sequences from different angles)
I guess I felt enough “spiritual alignment” with LessWrong when I first discovered it that the sequences weren’t my entrance point. I just kind of saw what people were talking about, and knew it was a conversation I wanted to participate in. I think that, as core to the community as the sequences are from a historical perspectie, they’re probably not everybody’s touchstone (at least not as much as they were for the “old-timers”).
It’s also kinda interesting that the “agency” tag on this site is so sparsely populated. Maybe that’s best explained by everything else being a subtopic of “agency,” and it being more valuable to talk about the sub-topics than the phenomenon as a whole?
I agree metacognition describes something we do, but I don’t think it captures it as well as a Rationality does (I don’t like “rationalism” and kinda frown whenever people use it, though gladly it’s not used a lot).
When I hear “Metacognition” I think about “Thinking about thinking”, but in any particular way. Rationality to me is almost like saying “Rational Metacognition”, meaning it has a direction, it strives to be successful, to do well (and so on). It doesn’t give as much freedom as Metacognition in a way that I like.
Put another way, Metacognition sounds like a phenomenon or a category of phenomena, while rationality sounds like a technique, an approach or a philosophy.
I am familiar with worrying that talking about rationality would feel awkward or pretentious, but I think finding a good way to introduce it could go a long way to help before we consider changing the name. Perhaps something like “I’m a rationalist, which means I learn and think about how to think well, so I can apply these lesson and be more effective and make better decisions”
Anyway, upvoted for an interesting topic and a well made argument.
Thanks for your thoughts on the subject! You’re right that metacognition has a passive ring to it, which is strange since it’s very much something we do, and that educators try to teach as a way to help students achieve instrumental goals. It feels somehow like a “thing that brains do” rather than a “thing you do with your brain,” which is sad, since one of the central metaphors for metacognition is that it’s about “learning to drive your brain.”
Maybe the problem is really that it doesn’t seem argumentative. You and I can both be engaged in meta-cognition, even if we totally disagree and aren’t making any progress toward Aumann agreement. By contrast, we can accuse each other of irrationality, and fear it in ourselves. A “failure to use metacognition” doesn’t sound as dangerous as a “failure to be rational.”
And yet I do strongly believe that thought is what drives action, and is what drives agency. In that light, “thinking about thinking” is also a way of controlling action and setting goals.
After considering the objections in the comments, I am more confident than I was when I published this post that the activist, tendentious implications of “rationality” are the point of the term. Terms like “aspiring rationalist,” “rationalish,” and other softenings are ways of “making ourselves small” to gain tactical advantage and navigate social status issues, rather than something most members of the movement really believe.
Alternatively, such softenings are ways to emphasize the rationality of the movement, since many people, including many rationalists, would agree that intuition, emotion, and tradition are valid sources of information. “Rationality” can have an implication of wasting good information. So we use “rationalish” to keep the activist spirit while showing that we’re up-to-date in our thinking.
Better would be if everybody could just somehow realize that “rationality” is working in an updated paradigm, and see it as a straightforward way to empower ordinary people and make the world a better place. That way, we wouldn’t need the softenings and deviations.
So we seem then to have an issue of creating common knowledge. Right now, we’re in a suboptimal equilibrium, where not everybody realizes that “rationality” has a capacity for an updated meaning, or we don’t all know that everybody knows it. So we’re stuck with circumlocutions and deviations until we somehow succeed in confirming that we’ve created a new common knowledge.
Just for context, I personally never used any of the softenings (I didn’t even hear about rationalish till this post). They’re both cute but ultimately meh in my opinion.
Aspiring rationalists is pretty much redundant, since a rationalist isn’t someone who claims to be rational, but indeed someome who aspires to be. So unless you aspire to aspire, there’s no point to the term.
I like Rationalish as a pun, but I think it solves the same nonexistent problem.
One person I’m 99% certain I’ve heard use “rationalish” and “aspiring rationalist” repeatedly is Julia Galef. She does a lot of communication with outside experts from other fields, and I imagine that fielding the implied status questions with grace is a persistent challenge in her line of work.
I do think it’s well worth meditating on the idea that there is no problem with “rationalist,” and cultivating a sense of benign confusion and simple explanation of the term if/when people object.
“I’m a rationalist.”
″Uh, so you think you’re smarter than everyone or something?”
“It just means a formal study of how to pick the right goals and decision-making procedures, and pursuing them effectively.”
“Oh, ok.”
Thinking about what it would take to actually pull off a rebranding, I think you’ve got to get a few people on board who will both announce the rebrand and then actually use the new term in place of the old one. This means something like you need to get the following folks on board:
LW
CFAR
MIRI
probably some EA orgs like 80k and CEA
Scott and maybe whoever the top 10 most followed rationalists are after him
If you can do that it’s probably enough to both prevent an annoying schism of folks hanging on to the old term and create common knowledge that the name changed. People who stick to the old name are then stragglers who will be seen as left behind by virtue of the actions of prestigious people and organizations within the movement.
This seems doable if you can convince all the right people. Probably means convincing them individually and then making it common knowledge that they all agree and then pushing a specific, actionable plan they can carry out.
Strategically, I’m not sure if the right first move is to try and appeal to leadership. I think this is an idea that needs some organic support. So I see this post as a seed, rather than an announcement of an agenda.
Let’s play a game. We’ll use LW’s about page description of what we mean by “rationality.” I will give points for whether “rational agent,” “agency,” or “metacognition” seem like the best label, or whether it’s a tie.
Since “rationality” sort of has the connotation of “agency,” maybe this tilts me more in favor of keeping the name we already have, at least for most of the established organizations?
On the other hand, I really do think there’s a strong contingent of people in this space who are more interested in metacognition, and have a skepticism of attempts to become a rational agent. Maybe this helps explain the discomfort with the name—there really are two different groups, in dialog, but with diverging values around agency.
Sure, you gotta figure out what you want to ask these people to do and why you want to do it first, and if you hope to succeed it’s gotta be a solid enough argument that they buy it. The usual heuristic I’ve seen is that something has to be at least 3x better for people to pay whatever costs are associated with adopting something better (for some arbitrary version of adopting or replacing an old thing).
I don’t know that you have to wait for organic support, but organic support would be an indicator that you’ve got something good that you might be able to convince folks to switch to.
fwiw I don’t think rationality has much connotation of agency, except for what it’s achieved by dint of association with us.
Echoing what Rob said above: these labels apply in very different ways. “Rationality” applies in a definition-like way, which is how it’s used on the page. Metacognition applies in an example-like way. You would never say “what we mean by metacognition is thinking in ways which systematically arrive at truth”, because even if the practice has that result, that’s not what the word means. (And I doubt the practice always does have that result.)
Eh, my reaction to this is something like, then they can have their metacognitive movement. And it can overlap with the rationality community, but they shouldn’t be the same thing.
I’m not very worried about people remaining attached to the old term because nobody was ever attached to the old term.
I remember Eliezer saying he doesn’t love the term and doesn’t remember there ever being a decision to adopt it; most real ones put “aspiring” on the front, recognizing problems mentioned; a lot of living rationalist-adjacent communities have started calling themselves “post-rationalists” in explicit rejection of the term; and objectively speaking, most of the people in the world who flatly identify as “rationalists” haven’t read the any yudkowsky at all (seems to be most popular among skeptics operating in very religious regions, increasingly rare in the west).
I feel very resistant to the name change, and am cringing at the thought process that even brought it up. This feels like it’s getting dangerously close to mindkilling/politics territory, the first step in a schism.
The Tribe is not the Movement is not the Goal is not the Brand is not the Way. Conflating these seems like a misstep. They are not the same. Which thing are we trying to name?
Branding is advertising is Dark Arts. When we resort to the practice of Dark Arts, we should do so with both eyes open: with a clear goal in mind and an awareness of the risks and costs.
Who gets to control the name? Who decides who is in or out? Names can be diluted or stolen. Remember what “Nanotechnology” used to mean? It used to be molecular assembers. Now it’s just fine powders. Remember what “the Singularity” used to mean? It used to be an AI FOOM, now it’s just Moore’s Law. I’m sure you could come up with more examples.
In another comment here, you said:
So is your discomfort with having a name for the community/movement/field/whatever? With a particular process by which it’s determined? With a particular choice of name? With a sense of identity attached to the information and practice?
I object to others calling me names. (That I don’t approve of, yes.) Ad hominem is a Dark Art. Even if the name isn’t exactly an insult, it can still put you in a box you maybe shouldn’t be in. It can bias how others see you, or how you see yourself, once internalized. I want to keep my identity small, because identity is one of those things that can bias thinking in unhealthy ways.
I object to your name-calling being the price of entry to a tribe that I already feel that I belong to. Names are boundaries. They can exclude as well as include. If part of the tribe objects to the name, but part accepts it, that’s a schism. Schisms aren’t categorically bad, but I think this one would make the tribe weaker.
I object to having the same name for the community/movement/field/whatever. They should each have different names. You seem to think they’re all the same thing. They’re not.
I think it’s an ok description of what we do in terms of epistemic rationality. I’m not so sure it captures the instrumental part. The biggest impact that joining this community had on my life was that I started really taking actions to further my goals.
On the plus side I guess this would mean CFAR would have to become CFAM. So I guess there’s a choice to make about whether you would rather see far or see your fam.
Metacognition is a fine name for a specific cognitive science, but, unlike “rationality” or “rationalist”, it has none of the call for action, just idle musings of how brains might work. “Metacognition is Systematized Winning” doesn’t pack any punch.
It seems to be inhererent to activist terms to lead to motte and bailey situations. “Rationalism is Systematized Winning” certainly leads to motte and bailey situations where people confuse the utility optimization which “thinking like the typical Lesswronger”.
Having a less activist term that’s less likely to lead to motte and bailey situations seem to me like it would be an improvement.
Right, that’s what I was getting at in “Crux: “Rationalism” Is An Activist Term.”
Through this lens, the reason people feel discomfort calling themselves a “rationalist” is because they’re afraid of making themselves big. If what we want is to be big, then maybe we should encourage each other to publicly and unapologetically identify as rationalists, and to project the notion that we are in fact dedicated to a practice and movement for rationality that does make us more rational than other people? Is that a stance we’re comfortable?
And if we’re not comfortable with that big stance, but do like the punch of the word “rationality,” then how do we reconcile that? Do we like being vicariously big through our champion, Big Yud?
I wouldn’t have gotten into rationality if it weren’t for the (implicit) promise that doing so would make me better.
I understand the core idea of it is this.
If we collect evidence about the world, and build a model using unbiased means, and predict the consequences of our possible actions: the only way to do better requires that someone else build a model, openly share how they did it, and show it has better predictions.
Paragraph 2 is being a rational agent and it also includes that 2 rational agents cannot agree to disagree if they share priors.
What this means for real world:
It means if there is a discussion about what to do, while we can never be certain, you cannot do better than the action predicted by a model without advancing a testably better model of your own. This applies to countless political questions.
It means that experts are often wrong, as it is irrelevant how many decades of experience or what position they hold. All that matters is what do they know and how do they know it.
I think the ‘metacognition’ part is a subset of it, it’s just a way to do better as a human. And unfortunately that has obviously diminishing returns. Our best bet as a human is to build tools—software that actually realizes some of the math implied above for instance—and then to listen to what our tools tell us, once we get them properly debugged...
I like it. By all means, as long as we’re thinking about thinking, let’s think about how we label ourselves.
I commit support.
If there is a difference between studying rationality and studying metacognition, our proclivity, if we’re honest, was always first inclined towards studying metacognition, towards truth, principled, correct functionality, then to instrumental concerns as labor. If metacognition had not turned out to have instrumental uses, I would still have this itch to do it.
Similarly, if there exists any good philosophy that isn’t just metacognition, I don’t really know about it, I wouldn’t know about it, I probably wouldn’t find any of it interesting, it’s not what I’m turning up for. All of the robust, high-return philosophy seems to be about thinking about thinking.
The philosophy channel of our ideology discord has been named “metacogning” for many months. Should a metacognition student be called a metacogner. Probably not. But it wouldn’t be terrible.
I am strongly in favor of this new name you propose. I especially like the alliteration. It is my belief that a good and nice-sounding name is a prerequisite to having something be accepted by Society™. I have long disliked the term “rationalist” to refer to this community for several reasons, but you lay them all out clearly here. Thank you for that.
I do like the term metacognitive movement. When we looked into describing the kind of people who go to our LessWrong meetup in Berlin “People who like to think about how to think” was one description we came up with matches well with the term.
I remember a conversation with someone from CFAR who said that CFAR tries to give people agency about how they think which goes into a similar direction.
Setting aside the name for the moment, I feel that “we” have never been particularly good at solving coordination problems. See Why Our Kind Can’t Cooperate.
LessWrong itself is not really about coordination. It’s more about sanity in a mad world. One of the prices we pay for that sanity is a cultural taboo on getting too deep into politics, because Politics is the Mind Killer. It’s why the default commenting guidelines here say “Aim to explain, not persuade”.
To the extent that “we” have succeeded in coordinating, it’s usually been by building institutions of sane(r) people outside of LessWrong proper, not by persuading lots of LessWrongers to coordinate on LessWrong. If you want to organize an outside institution around a goal, then you can give that institution an appropriate brand.
So I take it that you basically see LessWrong as a blogging forum that should not be a locus for community- or institution-building, or for activism? Just for individuals writing, reading, and commenting in an intimate way to refine their own personal thinking?
What I think it is now and what I think it should be are two different things. I would prefer that saner people have more control in the world, and if that means some kind of activism, so be it. That could be instrumentally rational.
LessWrong, the website, however, isn’t really set up for governance or mass coordination. We don’t have parliamentary procedures, arbitration procedures, good ways to punish defectors, ways to buy in like a Kickstarter, or even a philosopher-king. And our only voting system is karma, which is too easily gamed were it to become political. We sort of have ways to establish facts, but that could be improved. The only avenue I’m seeing is political persuasion, which is taboo for good reason, because it risks destroying the very thing that makes LessWrong valuable: some semblance of sanity in a mad world. We have not yet perfected the Art. We are not immune to human failings. Perhaps rather than “sane” we are just a little LessWrong :)
If you want to set up a platform for coordination and governance by, for, and of rationalists, then I’m for it. I would love to have trustworthy rationalist institutions to tell me what to invest in or donate to or who to vote for or what risks are reasonable or aren’t or how to optimize my health and well-being, but this should be external to LessWrong, so that when it fails (and it likely will—institution building is hard) then it won’t take LessWrong down with it. My priors include rumors of group houses that did poorly, MetaMed and Arbital, which haven’t worked out so well, and CFAR, which… is still there and doing stuff, I guess.
Discussing hypothetically how such institutions should work is totally on-topic for LessWrong though. Prediction markets? Approval voting? Science courts? Iterated trust kickstarters? Dominant assurance contracts? I don’t think we’ve figured that out yet. I don’t personally think that I have the skills to attempt this right now. But we should keep trying. How often do startups fail? 90% of the time? And those that do succeed often have the experience from failing before. The 10% that do succeed are worth it. We see postmortems from institution building attempts here from time to time. I think they’re valuable.
That’s an interesting perspective.
What people are doing here can look, both to participants and to outsiders, like some form of movement-building or ideology-spreading. There’s a demand both from within and without to name and describe what “we” do, what “we” think, “our” takes on the political and scientific questions of the day, “our” relationships with other people/ideologies/movements/ideas.
The way some people, including me here, respond this demand is by trying to supply an acceptable answer. One that is accurate, bold enough to be interesting, yet cautious enough to avoid provoking unnecessary ire.
An alternative way to respond to this demand is by rejecting it, to say “that question is unhelpful, so please stop asking.”
In that light, some basic principles of the forum might be:
Do not choose speech norms for the forum based on their impact on political movements.
For example, do not make arguments of the form “Movement X is bad. Speech norm Y is Xist, while speech norm Z is anti-X. Therefore, we should reject speech norm Y and use speech norm Z.”
Do not apply rewards and punishments in order to disrupt thinking and enforce conformity. Use them instead to reward cogency and critical thinking. Use scout mindset, not soldier mindset.
Actively resist attempts to tie the forum into a particular movement, ideology, or institution, or frame it as an enemy. This includes attempts to tie it to a movement that champions these principles in the wider world. Keep the forum independent, neutral, and small.
Actively resist attempts to use incentives (voting mechanisms, rhetoric, official site policy) to try and enforce a unified approach to the execution of forum independence and neutrality. Make reasoned arguments, and let people make their own decisions.
Overall agreed that metacognitive is better branding. I like rationality as in-community jargon for the thing that cognition should aim for.
How is that different then saying that cognition should strive for winning and why do you like it as in-community jargon instead of saying directly “cognition should strive for winning”?
Because rationality is not just winning! Its about winning systematically rather than by fluke, about narrowing the diff between reality and your model of it such that the forecasted wins you aim for correspond to actual wins.
The textbook definition from Jonathan Baron’s Thinking and deciding for example doesn’t include that sense. Eliezers definition about systematized winning also include that it has to be through modeling reality.
The CFAR goal of giving people agency about their own thinking is also not directly about reducing diffs between models and reality.
I find rationalist cringey for some reason and won’t describe myself like that. As you said, in seems to discount intuition, emotion and instinct. 99% of human behaviour is driven by irrational forces and that’s not necessarily a bad thing. The word rationalist to me feels like a denial of our true nature and a doomed attempt to be purely rational—rather than trying to be a bit more deliberate in action and beliefs
There once was a time when people who were obsessed with knowledge (and the appropriate action flowing therefrom) were called scientists. Now they are just adherents to scientism, and the rest of us have to pick up terms to describe our taking the mantle. Where the “rationalist” community seems to be is at the intersection of metacognition and “rational” (whatever that means :P ). Neither describes the movement entirely on its own, but with their powers combined...
Interesting post, thanks.
A love of knowledge? I have always had a terrible memory for all types of knowledge except metacognition. I only learned the age and scale of the universe in recent years for mostly spiritual reasons and I am still way way more interested in uncovering the meta-level of these things, the sorts of ages and scales that we can infer would be common for life-supporting laws-of-physics (the meta seems more directly relevant to anthropic problems).