Certainly it is possible to find success in some areas anonymously. No argument with you there!
I view LW-style rationality as a community of practice, a culture of people aggregating, transmitting, and extending knowledge about how to think rationally. As in “The Secret of Our Success,” we don’t accomplish this by independently inventing the techniques we need to do our work. We accomplish this primarily by sharing knowledge that already exists.
Another insight from TSOOS is that people use prestige as a guide for who they should imitate. So rationalists tend to respect people with a reputation for rationality.
But what if a reputation for rationality can be cultivated separately from tangible accomplishments?
In fact, prestige is already one step removed from the tangible accomplishments. But how do we know if somebody is prestigious?
Perhaps a reputation can be built not by gaining the respect of others through a track record of tangible accomplishments, but by persuading others that:
a) You are widely respected by other people whom they haven’t met, or by anonymous people they cannot identify, making them feel behind the times, out of the loop.
b) That the basis on which people allocate prestige conventionally is flawed, and that they should do it differently in a way that is favorable to you, making them feel conformist or conservative.
c) That other people’s track record of tangible accomplishments are in fact worthless, because they are not of the incredible value of the project that the reputation-builder is “working on,” or are suspect in terms of their actual utility. This makes people insecure.
d) Giving people an ability to participate in the incredible value you are generating by convincing them to evangelize your concept, and thereby to evangelize you. Or of course, just donating money. This makes people feel a sense of meaning and purpose.
I could think of other strategies for building hype. One is to participate in cooperative games, whereby you and other hype each other, create a culture of exclusivity. If enough people do this, it could perhaps trick our monkey brains into perceiving that they’re a socially dominant person in a much larger sphere than they really are.
Underlying this anxious argument is a conjunction that I want to make explicit, because it could lead to fallacy:
It rests on a hypothesis that prestige has historically been a useful proxy for success...
… and that imitation of prestigious people has been a good way to become successful...
… and that we’re hardwired to continue using it now...
… and that prestige can be cheap to cultivate or credit easy to steal in some domains, with rationality being one such domain; or that we can delude ourselves about somebody’s prestige more easily in a modern social and technological context...
… and that we’re likely enough to imitate a rationalist-by-reputation rather than a rationalist-in-fact that this is a danger worth speaking about...
… and perhaps that one such danger is that we pervert our sense of rationality to align with success in reputation-management rather than success in doing concrete good things.
You could argue against this anxiety by arguing against any of these six points, and perhaps others. It has many failure points.
One counterargument is something like this:
People are selfish creatures looking out for their own success. They have a strong incentive not to fall for hype unless it can benefit them. They are also incentivized to look for ideas and people who can actually help them be more successful in their endeavors. If part of the secret of our success is cultural transmission of knowledge, another part is probably the cultural destruction of hype. Perhaps we’re wired for skepticism of strangers and slow admission into the circle of people we trust.
Hype is excitement. Excitement is a handy emotion. It grabs your attention fleetingly. Anything you’re excited about has only a small probability of being as true and important as it seems at first. But occasionally, what glitters is gold. Likewise, being attracted to a magnetic, apparently prestigious figure is fine, even if the majority of the time they prove to be a bad role model, if we’re able to figure that out in time to distance ourselves and try again.
So the Secret of Our Success isn’t blind, instinctive imitation of prestigious people and popular ideas. Nor is it rank traditionalism.
Instead, it’s cultural and instinctive transmission of knowledge among people with some capacity for individual creativity and skeptical realism.
So as a rationalist, the approach this might suggest is to use popularity, hype, and prestige to decide which books to buy, which blogs to peruse, which arguments to read. But actually read and question these arguments with a critical mind. Ask whether they seem true and useful before you accept them. If you’re not sure, find a few people who you think might know better and solicit their opinion.
Gain some sophistication in interpreting why controversy among experts persists, even when they’re all considering the same questions and are looking at the same evidence. As you go examining arguments and ideas in building your own career and your own life, be mindful not only of what argument is being made, but of who’s making it. If you find them persuasive and helpful, look for other writings. See if you can form a relationship with them, or with others who find them respectable. Look for opportunities to put those ideas to the test. Make things.
I find this counter-argument more persuasive than the idea of being paranoid of people’s reputations. In most cases, there are too many checks on reputation for a faulty one to last for too long; there are too many reputations with a foundation in fact to make the few that are baseless be common confounders; we seem to have some level of instinctive skepticism that prevents us from giving ourselves over full to a superficially persuasive argument or to one person’s ill-considered dismissal; and even being “taken in” by a bad argument may often lead to a learning process that has long term value. Perhaps the vivid examples of durable delusions are artifacts of survivorship bias: most people have many dalliances with a large number of bad ideas, but end up having selected enough of the few true and useful ones to end up in a pretty good place in the end.
Ah, so you mean within the rationalist (and adjacent) community; how can we make sure that we instinctively copy our most rational members, as opposed to random or even least rational ones.
When I reflect on what I do by default… well, long ago I perceived “works at MIRI/CFAR” as the source of prestige, but recently it became “writes articles I find interesting”. Both heuristics have their advantages and disadvantages. The “MIRI/CFAR” heuristic allows me to outsource judgment to people who are smarter than me and have more data about their colleagues; but it ignores people outside Bay Area and those who already have another job. The “blogging” heuristic allows me to judge the thinking of authors; but it ignores people who are too busy doing something important or don’t wish to write publicly.
But what if a reputation for rationality can be cultivated separately from tangible accomplishments?
Here is how to exploit my heuristics:
Be charming, and convince people at MIRI/CFAR/GiveWell/etc. to give you some role in their organization; it could be a completely unimportant one. Make your association known.
Have good verbal skills, and deep knowledge of some topic. Write a blog about that topic and the rationalist community.
Looking at your list: Option a) if someone doesn’t live in Bay Area, it could be quite simple to add a few rationalist celebrities as friends on Facebook, and then pretend that you have some deeper interaction with them. People usually don’t verify this information, so if no one at your local meetup is in regular contact with them, the risk of exposure is low. Your prestige is then limited to the local meetup.
Options b) and c) would probably lead to a big debate. Arguably, “metarationality” is an example of “actually, all popular rationalists are doing it wrong, this is the true rationality” claim.
Option d) was tried by Intentional Insights, Logic Nation, and I have heard about people who try to extract free work from programmers at LW meetups. Your prestige is limited to the few people you manage to recruit.
Rationalist community has a few people in almost undefeatable positions (MIRI and CFAR, Scott Alexander), who have the power to ruin the reputation of any pretender, if they collectively choose so. Someone trying to get undeserved prestige needs to stay under their radar, or infiltrate them, because trying to replace them by a paralell structure would be too much work.
At this point, for someone trying to get into a position of high prestige, it would be much easier to simply start their own movement, built on different values. However, should the rationalist community become more powerful in the future, this equation may change.
Certainly it is possible to find success in some areas anonymously. No argument with you there!
I view LW-style rationality as a community of practice, a culture of people aggregating, transmitting, and extending knowledge about how to think rationally. As in “The Secret of Our Success,” we don’t accomplish this by independently inventing the techniques we need to do our work. We accomplish this primarily by sharing knowledge that already exists.
Another insight from TSOOS is that people use prestige as a guide for who they should imitate. So rationalists tend to respect people with a reputation for rationality.
But what if a reputation for rationality can be cultivated separately from tangible accomplishments?
In fact, prestige is already one step removed from the tangible accomplishments. But how do we know if somebody is prestigious?
Perhaps a reputation can be built not by gaining the respect of others through a track record of tangible accomplishments, but by persuading others that:
a) You are widely respected by other people whom they haven’t met, or by anonymous people they cannot identify, making them feel behind the times, out of the loop.
b) That the basis on which people allocate prestige conventionally is flawed, and that they should do it differently in a way that is favorable to you, making them feel conformist or conservative.
c) That other people’s track record of tangible accomplishments are in fact worthless, because they are not of the incredible value of the project that the reputation-builder is “working on,” or are suspect in terms of their actual utility. This makes people insecure.
d) Giving people an ability to participate in the incredible value you are generating by convincing them to evangelize your concept, and thereby to evangelize you. Or of course, just donating money. This makes people feel a sense of meaning and purpose.
I could think of other strategies for building hype. One is to participate in cooperative games, whereby you and other hype each other, create a culture of exclusivity. If enough people do this, it could perhaps trick our monkey brains into perceiving that they’re a socially dominant person in a much larger sphere than they really are.
Underlying this anxious argument is a conjunction that I want to make explicit, because it could lead to fallacy:
It rests on a hypothesis that prestige has historically been a useful proxy for success...
… and that imitation of prestigious people has been a good way to become successful...
… and that we’re hardwired to continue using it now...
… and that prestige can be cheap to cultivate or credit easy to steal in some domains, with rationality being one such domain; or that we can delude ourselves about somebody’s prestige more easily in a modern social and technological context...
… and that we’re likely enough to imitate a rationalist-by-reputation rather than a rationalist-in-fact that this is a danger worth speaking about...
… and perhaps that one such danger is that we pervert our sense of rationality to align with success in reputation-management rather than success in doing concrete good things.
You could argue against this anxiety by arguing against any of these six points, and perhaps others. It has many failure points.
One counterargument is something like this:
People are selfish creatures looking out for their own success. They have a strong incentive not to fall for hype unless it can benefit them. They are also incentivized to look for ideas and people who can actually help them be more successful in their endeavors. If part of the secret of our success is cultural transmission of knowledge, another part is probably the cultural destruction of hype. Perhaps we’re wired for skepticism of strangers and slow admission into the circle of people we trust.
Hype is excitement. Excitement is a handy emotion. It grabs your attention fleetingly. Anything you’re excited about has only a small probability of being as true and important as it seems at first. But occasionally, what glitters is gold. Likewise, being attracted to a magnetic, apparently prestigious figure is fine, even if the majority of the time they prove to be a bad role model, if we’re able to figure that out in time to distance ourselves and try again.
So the Secret of Our Success isn’t blind, instinctive imitation of prestigious people and popular ideas. Nor is it rank traditionalism.
Instead, it’s cultural and instinctive transmission of knowledge among people with some capacity for individual creativity and skeptical realism.
So as a rationalist, the approach this might suggest is to use popularity, hype, and prestige to decide which books to buy, which blogs to peruse, which arguments to read. But actually read and question these arguments with a critical mind. Ask whether they seem true and useful before you accept them. If you’re not sure, find a few people who you think might know better and solicit their opinion.
Gain some sophistication in interpreting why controversy among experts persists, even when they’re all considering the same questions and are looking at the same evidence. As you go examining arguments and ideas in building your own career and your own life, be mindful not only of what argument is being made, but of who’s making it. If you find them persuasive and helpful, look for other writings. See if you can form a relationship with them, or with others who find them respectable. Look for opportunities to put those ideas to the test. Make things.
I find this counter-argument more persuasive than the idea of being paranoid of people’s reputations. In most cases, there are too many checks on reputation for a faulty one to last for too long; there are too many reputations with a foundation in fact to make the few that are baseless be common confounders; we seem to have some level of instinctive skepticism that prevents us from giving ourselves over full to a superficially persuasive argument or to one person’s ill-considered dismissal; and even being “taken in” by a bad argument may often lead to a learning process that has long term value. Perhaps the vivid examples of durable delusions are artifacts of survivorship bias: most people have many dalliances with a large number of bad ideas, but end up having selected enough of the few true and useful ones to end up in a pretty good place in the end.
Ah, so you mean within the rationalist (and adjacent) community; how can we make sure that we instinctively copy our most rational members, as opposed to random or even least rational ones.
When I reflect on what I do by default… well, long ago I perceived “works at MIRI/CFAR” as the source of prestige, but recently it became “writes articles I find interesting”. Both heuristics have their advantages and disadvantages. The “MIRI/CFAR” heuristic allows me to outsource judgment to people who are smarter than me and have more data about their colleagues; but it ignores people outside Bay Area and those who already have another job. The “blogging” heuristic allows me to judge the thinking of authors; but it ignores people who are too busy doing something important or don’t wish to write publicly.
Here is how to exploit my heuristics:
Be charming, and convince people at MIRI/CFAR/GiveWell/etc. to give you some role in their organization; it could be a completely unimportant one. Make your association known.
Have good verbal skills, and deep knowledge of some topic. Write a blog about that topic and the rationalist community.
Looking at your list: Option a) if someone doesn’t live in Bay Area, it could be quite simple to add a few rationalist celebrities as friends on Facebook, and then pretend that you have some deeper interaction with them. People usually don’t verify this information, so if no one at your local meetup is in regular contact with them, the risk of exposure is low. Your prestige is then limited to the local meetup.
Options b) and c) would probably lead to a big debate. Arguably, “metarationality” is an example of “actually, all popular rationalists are doing it wrong, this is the true rationality” claim.
Option d) was tried by Intentional Insights, Logic Nation, and I have heard about people who try to extract free work from programmers at LW meetups. Your prestige is limited to the few people you manage to recruit.
Rationalist community has a few people in almost undefeatable positions (MIRI and CFAR, Scott Alexander), who have the power to ruin the reputation of any pretender, if they collectively choose so. Someone trying to get undeserved prestige needs to stay under their radar, or infiltrate them, because trying to replace them by a paralell structure would be too much work.
At this point, for someone trying to get into a position of high prestige, it would be much easier to simply start their own movement, built on different values. However, should the rationalist community become more powerful in the future, this equation may change.