I have some pretty complicated thoughts on this, and my heart isn’t really in responding to you because I think some things are helpful for some people, but a sketch of what I’m thinking:
First, a clarification. Some early claims—like the ones I was responding to in my 2009 essay—were that rationalists should be able to basically accomplish miracles, become billionaires with minimal work, unify physics with a couple of years of study, etc. I still occasionally hear claims along those lines. I am still against those, but I interpret you as making weaker claims, like that rationalists can be 10% better at things than nonrationalists, after putting in a decent amount of work. I’m less opposed to those claims, especially if “a decent amount of work” is interpreted as “the same amount of work you would need to get good at those things through other methods”. But I’m still a little bit concerned about them.
First: I’m interpreting “rationalist self-help” to mean rationalist ideas and practices that are helpful for getting common real-life goals like financial, social, and romantic success. I’m not including things like doing charity better, for reasons that I hope will become clear later.
These are the kinds of things most people want, which means two things. First, we should expect a lot of previous effort has gone into optimizing them. Second, we should expect that normal human psychology is designed to optimize them. If we’re trying to do differential equations, we’re outside our brain’s design specs; if we’re trying to gain status and power, we’re operating exactly as designed.
When the brain fails disastrously, it tends to be at things outside the design specs, that don’t matter for things we want. For example, you quoted me describing some disastrous failures in people understanding some philosophy around atheism, and I agree that sort of thing happens often. But this is because it’s outside of our common sense. I can absolutely imagine a normal person saying “Since I can’t prove God doesn’t exist, God must exist”, but it would take a much more screwed-up person to think “Since I can’t prove I can’t fly, I’m going to jump off this cliff.”
Another example: doctors fail miserably on the Bayes mammogram problem, but usually handle actual breast cancer diagnosis okay. And even diagnosing breast cancer is a little outside common sense and everyday life. Faced with the most chimpish possible version of the Bayes mammogram problem—maybe something like “This guy I met at a party claims he’s the king of a distant country, and admittedly he is wearing a crown, but what’s the chance he’s *really* a king?” my guess is people are already near-optimal.
If you have this amazing computer perfectly-tuned for finding strategies in a complex space, I think your best bet is just to throw lots and lots of training data at it, then try navigating the complex space.
I think it’s ironic that you use practicing basketball as your example here, because rationalist techniques very much are *not* practice. If you want to become a better salesman, practice is going out and trying to make lots of sales. I don’t think this is a “rationalist technique” and I think the kind of self-help you’re arguing for is very different (though it may involve better ways to practice). We both agree that practice is useful; I think our remaining disagreement is on whether there are things other than practice that are more useful to do, on the margin, than another unit of practice.
Why do I think this is unlikely?
1. Although rationalists have done pretty well for themselves, they don’t seem to have done too remarkably well. Even lots of leading rationalist organizations are led by people who haven’t put particular effort into anything you could call rationalist self-help! That’s really surprising!
2. Efficient markets. Rationalists developed rationalist self-help by thinking about it for a while. This implies that everyone else left a $100 bill on the ground for the past 4000 years. If there were techniques to improve your financial, social, and romantic success that you could develop just by thinking about them, the same people who figured out the manioc detoxification techniques, or oracle bone randomization for hunting, or all the other amazingly complex adaptations they somehow developed, would have come up with them. Even if they only work in modern society, one of the millions of modern people who wanted financial, social, and romantic success before you would have come up with them. Obviously this isn’t 100% true—someone has to be the first person to discover everything—but you should expect the fruits here to be very high up, high enough that a single community putting in a moderate amount of effort shouldn’t be able to get too many of them.
(some of this becomes less relevant if your idea of rationalist self-help is just collecting the best self-help from elsewhere and giving it a stamp of approval, but then some of the other considerations apply more.)
3. Rationalist self-help starts looking a lot like therapy. If we’re trying to make you a more successful computer programmer using something other than studying computer programming, it’s probably going to involve removing mental blocks or something. Therapy has been pretty well studied, and the most common conclusion is that it is mostly nonspecific factors and the techniques themselves don’t seem to have any special power. I am prepared to suspend this conclusion for occasional miracles when extremely charismatic therapists meet exactly the right patient and some sort of non-scaleable flash of lightning happens, but this also feels different from “the techniques do what they’re supposed to”. If rationalists are trying to do therapy, they are competing with a field of tens of thousands of PhD-level practitioners with all the resources of the academic and health systems who have worked on the problem for decades. This is not the kind of situation that encourages me we can make fast progress. See https://slatestarcodex.com/2019/11/20/book-review-all-therapy-books/ for more on this.
4. General skepticism of premature practical application. It took 300 years between Harvey discovering the circulatory system and anyone being very good at treating circulatory disease. It took 50 years between Pasteur discovering germ theory and anyone being very good at treating infections. It took 250 years between Newton discovering gravity and anyone being very good at flying. I have a lower prior than you on good science immediately translating into useful applications. And I am just not too impressed with the science here. Kahneman and Tversky discovered a grab bag of interesting facts, some of which in retrospect were false. I still don’t think we’re anywhere near the deep understanding of rationality that would make me feel happy here.
This doesn’t mean I think rationality is useless. I think there are lots of areas outside our brain’s normal design specs where rationality is really useful. And because these don’t involve getting sex or money, there’s been a lot less previous exploration of the space and the low hanging fruits haven’t been gobbled up. Or, when the space has been explored, people haven’t done a great job formalizing their insights, or they haven’t spread, or things like that. I am constantly shocked by how much really important knowledge there is sitting around that nobody knows about or thinks about because it doesn’t have an immediate payoff.
Along with all of this, I’m increasingly concerned that anything that has payoff in sex or money is an epistemic death zone. Because you can make so much money teaching it, it attracts too much charlatanry to navigate easily, and it subjects anyone who enters to extreme pressure to become charlatan-adjacent. Because it touches so closely on our emotions and sense of self-worth, it’s a mind-killer in the same way politics are. Because everybody is so different, there’s almost irresistible pressure to push the thing that saved your own life, without checking whether it will help anyone else. Because it’s such a noisy field and RCTs are so hard, I don’t trust us to be able to check our intuitions against reality. And finally, I think there are whole things lurking out there of the approximate size and concerningness of “people are homeostasis-preserving control systems which will expend their entire energy on undoing any change you do to them” that we just have no idea about even though they have the potential to make everything in this sphere useless if we don’t respond to them.
I actually want to expand on the politics analogy. If someone were to say rationality was great at figuring out whether liberalism or conservatism was better, I would agree that this is the sort of thing rationality should be able to do, in principle. But it’s such a horrible topic that has the potential to do so much damage to anyone trying to navigate it that I would be really nervous about it—about whether we were really up to the task, and about what it would do to our movement if we tried. These are some of my concerns around self-help too.
Thank you for the detailed reply. I’m not going to reply point by point because you made a lot of points, but also because I don’t disagree with a lot of it. I do want to offer a couple of intuitions that run counter to your pessimism.
While you’re right that we shouldn’t expect Rationalists to be 10x better at starting companies because of efficient markets, the same is not true of things that contribute to personal happiness. For example: how many people have a strong incentive in helping you build fulfilling romantic relationships? Not the government, not capitalism, not most of your family or friends, often not even your potential partners. Even dating apps make money when you *don’t* successfully seduce your soulmate. But Rationality can be a huge help: learning that your emotions are information, learning about biases and intuitions, learning about communication styles, learning to take 5-minute timers to make plans — all of those can 10x your romantic life.
Going back to efficient markets, I get the sense that a lot of things out there are designed by the 1% most intelligent and ruthless people to take advantage of the 95% and their psychological biases. Outrage media, predatory finance, conspicuous brand consumption and other expensive status ladders, etc. Rationality doesn’t help me design a better YouTube algorithm or finance scam, but at least it allows me to escape the 95% and keeps me away from outrage and in index funds.
Finally, I do believe that the world is getting weirder faster, and the thousands of years of human tradition are becoming obsolete at a faster pace. We are moving ever further from our “design specs”. In this weirding world, I already hit jackpot with Bitcoin and polyamory, two things that couldn’t really exist successfully 100 years ago. Rationality guided me to both. You hit jackpot with blogging— can you imagine your great grand uncle telling you that you’ll become a famous intellectual by writing about cactus people and armchair sociology for free? And we’re both still very young.
For any particular achievement like basketball or making your first million, there are more dedicated practices that help you to your goal faster than Rationality. But for taking advantage of unknown unknowns, the only two things I know that work are Rationality and making friends.
I think efficient market doesn’t just suggest we can’t do much better at starting companies. It also means we can’t do much better at providing self-help, which is a service that can make people lots of money and status if they do it well.
I’m not sure if you’re using index fund investing as an example of rationalist self-help, or just as a metaphor for it. If you’re using it an example, I worry that your standards are so low that almost any good advice could be rationalist self-help. I think if you’re from a community where you didn’t get a lot of good advice, being part of the rationalist community can be really helpful in exposing you to it (sort of like the theory where college makes you successful because it inducts you into the upper-middle class). I think I got most of my “invest in index funds” level good advice before entering the rationalist community, so I didn’t count that.
Being part of the rationalist community has definitely improved my life, partly through giving me better friends and partly through giving me access to good ideas of the “invest in index funds” level. I hadn’t counted that as part of our discussion, but if I do, then I agree it is great. My archetypal idea of “rationalist self-help” is sitting around at a CFAR workshop trying very hard to examine your mental blocks. I’m not sure if we agree on that or if I’m caricaturing your position.
I’m not up for any gigantic time commitment, but if you want to propose some kind of rationalist self-help exercise that I should try (of the order of 10 minutes/day for a few weeks) to see if I change my mind about it, I’m up for that, though I would also believe you if you said such a halfhearted commitment wouldn’t be a good test.
I have several friends in New York who are a match to my Rationalist friends in age, class, intelligence etc. and who:
Pick S&P 500 stocks based on CNBC and blogs because their intuition tells them they’ve beat the market (but they don’t check or track it, just remember the winners).
Stay in jobs they hate because they don’t have a robust decision process for making such a switch (I used goal factoring, Yoda timer job research, and decision matrices to decide where to work).
Go so back asswards about dating that it hurts to watch (because they can’t think about it systematically).
Retweet Trump with comment.
Throw the most boring parties.
Spend thousands of dollars on therapists but would never do a half-hour debugging session with a friend because “that would be weird”.
In general, live mostly within “social reality” where the only question is “is this weird/acceptable” and never “is this true/false”.
Now perhaps Rationalist self-improvement can’t help them, but if you’re reading LessWrong you may be someone who may snap out of social reality long enough for Rationality to change your life significantly.
> if you want to propose some kind of rationalist self-help exercise that I should try
Different strokes for different folks. You can go through alkjash’s Hammertime Sequence and pick one, although even there the one that he rates lowest (goal factoring) is the one that was the most influential in my own life. You must be friends with CFAR instructors/mentors who know your personality and pressing issues better than I do and can recommend and teach a useful exercise.
Agreed, I see a major problem with an argument that seems to imply that since advice exists elsewhere/wasn’t invented by rationality techniques, a meta-heuristic for aggregating trustworthy sources isn’t hugely valuable.
In general, live mostly within “social reality” where the only question is “is this weird/acceptable” and never “is this true/false”.
It seems to me like people who primarily think in terms of weird/acceptable never join the rationality in the first place. Or do you believe that our community has taught people who used to think in those terms to think otherwise?
As I said, someone who is 100% in thrall to social reality will probably not be reading this. But once you peek outside the bubble there is still a long way to enlightenment: first learning how signaling, social roles, tribal impulses etc. shape your behavior so you can avoid their worst effects, then learning to shape the rules of social reality to suit your own goals. Our community is very helpful for getting the first part right, it certainly has been for me. And hopefully we can continue fruitfully exploring the second part too.
It also means we can’t do much better at providing self-help, which is a service that can make people lots of money and status if they do it well.
Maybe the incentives are all wrong here, and the most profitable form of “self-help” is one that doesn’t provide long term improvement, so that customers return for more and more books and seminars.
In that case, we can easily do better—better for the “customers”, but less profitable for the “gurus”.
through giving me access to good ideas of the “invest in index funds” level
for me the point is about getting *consistently* good ideas, getting reliable ideas where applying scientific method is too hard. It is much less about self-improvement as it is about community improvement in the face of more and more connected (and thus weird) world. Rationality is epistemology for the internet era.
Even lots of leading rationalist organizations are led by people who haven’t put particular effort into anything you could call rationalist self-help! That’s really surprising!
Indeed, I’m surprised to read that, because for the leading Berkeley rationalist organizations (MIRI, CFAR, CEA) I can think of at least one person in the top part of their org chart whom I personally know has done a rationalist self-help push for at least a couple of months before taking said role. (In two of those cases it’s the top person.)
Can you say what organizations you’re thinking of?
Also, yes rationalists do more curating of good advice than invention of it, just as we do with philosophy. But there’s a huge value-add in sorting out the good advice in a domain from the bad advice, and this I think the community does in a more cross-domain way than I see elsewhere.
I’m not sure how much we disagree; it sounds like I disagree with you, but maybe most of that is that we’re using different framings / success thresholds.
Efficient markets. Rationalists developed rationalist self-help by thinking about it for a while. This implies that everyone else left a $100 bill on the ground for the past 4000 years. If there were techniques to improve your financial, social, and romantic success that you could develop just by thinking about them, the same people who figured out the manioc detoxification techniques, or oracle bone randomization for hunting, or all the other amazingly complex adaptations they somehow developed, would have come up with them.
If you teleported me 4000 years into the past and deleted all of modernity and rationalism’s object-level knowledge of facts from my head, but let me keep as many thinking heuristics and habits of thought as I wanted, I think those heuristics would have a pretty large positive effect on my ability to pursue mundane happiness and success (compared to someone with the same object-level knowledge but more normal-for-the-time heuristics).
The way you described things here feels to me like it would yield a large overestimate of how much deliberate quality-adjusted optimization (or even experimentation and random-cultural-drift-plus-selection-for-things-rationalists-happen-to-value) human individuals and communities probably put into discovering, using, and propagating “rationalist skills that work” throughout all of human history.
Example: implementation intentions / TAPs are an almost comically simple idea. AFAIK, it has a large effect size that hasn’t fallen victim to the replication crisis (yet!). Humanity crystallized this idea in 1999. A well-calibrated model of “how much optimization humanity has put into generating, using, and propagating rationality techniques” shouldn’t strongly predict that an idea this useful and simple will reach fixation in any culture or group throughout human history before the 1990s, since this in fact never happened. But your paragraph above seems to me like it would predict that many societies throughout history would have made heavy use of TAPs.
I’d similarly worry that the “manioc detoxification is the norm + human societies are as efficient at installing mental habits and group norms as they are at detoxifying manioc” model should predict that the useful heuristics underlying the ‘scientific method’ (e.g., ‘test literally everything’, using controls, trying to randomize) reach fixation in more societies earlier.
Plausibly science is more useful to the group than to the individual; but the same is true for manioc detoxification. There’s something about ideas like science that caused societies not to converge on them earlier. (And this should hold with even more force for any ideas that are hard to come up with, deploy, or detect-the-usefulness-of without science.)
Another thing that it sounds like your stated model predicts: “adopting prediction markets wouldn’t help organizations or societies make money, or they’d already have been widely adopted”. (Of course, what helps the group succeed might not be what helps the relevant decisionmakers in that organization succeed. But it didn’t sound like you expected rationalists to outperform common practice or common sense on “normal” problems, even at the group level.)
I’d similarly worry that the “manioc detoxification is the norm + human societies are as efficient at installing mental habits and group norms as they are at detoxifying manioc” model should predict that the useful heuristics underlying the ‘scientific method’ (e.g., ‘test literally everything’, using controls, trying to randomize) reach fixation in more societies earlier.
I’d disagree! Randomized controlled trials have many moving parts, removing any of which makes them worse than useless. Remove placebo control, and your trials are always positive and you do worse than intuition. Remove double-blinding, same. Remove power calculations, and your trials give random results and you do worse than intuition. Remove significance testing, same. Even in our own advanced civilization, if RCTs give a result different than common sense it’s a 50-50 chance which is right; a primitive civilization who replaced their intuitions with the results of proto-RCTs would be a disaster. This ends up like the creationist example where evolution can’t use half an eye so eyes don’t evolve; obviously this isn’t permanently true with either RCTs or eyes, but in both cases it took a long time for all the parts to evolve independently for other reasons.
Also, you might be underestimating inferential distance—tribes that count “one, two, many” are not going to be able to run trials effectively. Did you know that people didn’t consistently realize you could take an average of more than two numbers until the Middle Ages?
Also, what would these tribes use RCTs to figure out? Whether their traditional healing methods work? St. John’s Wort is a traditional healing method, there have now been about half a dozen high-quality RCTs investigating it, with thousands of patients, and everyone is *still* confused. I am pretty sure primitive civilizations would not really have benefited from this much.
I am less sure about trigger-action plans. I think a history of the idea of procrastination would be very interesting. I get the impression that ancient peoples had very confused beliefs around it. I don’t feel like there is some corpus of ancient anti-procrastination techniques from which TAPs are conspicuously missing, but why not? And premodern people seem weirdly productive compared to moderns in a lot of ways. Overall I notice I am confused here, but this could be an example where you’re right.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t. This seems different to me than the prediction market example, since (as Robin has discussed) decision-makers might self-interestedly prefer not to have prediction markets so they can keep having high status as decision-makers.
Randomized controlled trials have many moving parts, removing any of which makes them worse than useless.
I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t.
Yeah, I was wrong about manioc.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”. Something feels off to me when I visualize a world where all the stupidly-simple epistemic-methods-that-are-instrumentally-useful fruit got plucked 4000 years ago, but where Feynman can see big gains from mental habits like “look at the water” (which I do think happened).
Your other responses make sense. I’ll need to chew on your comments longer to see how much I end up updating overall toward your view.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”.
I’d propose that there’s a massive qualitative difference between black-box results (like RCTs) and gears-level model-building (like Archimedes, Newton, and Feynman). The latter are where basically all of the big gains are, and it does seem like society is under-invested in building gears-level models. One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
I would suspect it’s the other way around, that they have very high depreciation rates; we no longer have Feynman’s gears-level models, for example.
about whether we were really up to the task, and about what it would do to our movement if we tried
Does your outlook change at all if you try dropping the assumption that there is a “we” who cares about “our movement”? I’ve certainly found a lot of the skills I learned from reading the Sequences useful in my subsequent thinking about politics and social science. (Which thinking, obviously, mostly does not take the form of asking whether “liberalism” or “conservatism” is “better”; political spectra are dimensionality reductions.) I’m trying. I’m not sure how much I’ve succeeded. But I expect my inquiries to be more fruitful than those of people who make appeals to consequences to someone’s “movement” before trying to think about something.
You’re right in catching and calling out the appeal to consequences there, of course.
But aside from me really caring about the movement, I think part of my thought process is that “the movement” is also the source of these self-help techniques. If some people go into this space and then report later with what they think, I am worried that this information is less trustworthy than information that would have come from these same people before they started dealing with this question.
Even if they only work in modern society, one of the millions of modern people who wanted financial, social, and romantic success before you would have come up with them.
Nobody is claiming that everything around rationalist circles is completely new or invented by them. It’s often looked to me more like separating the more and less useful stuff with various combinations of bottom-up and top-down approaches.
Additionally, I’d like to also identify as someone who is definitely in a much much better place now because they discovered LW almost a decade ago even though I also struggle with akrasia and do less to improve myself than I’d like, I’m very sure that just going to therapy wouldn’t have improved my outcomes in so many areas, especially financially.
There are so many arguments trying to be had at once here that it’s making my head spin.
Here’s one. What do we mean by self-help?
I think by self-help Scott is thinking about becoming psychologically a well-adjusted person. But what I think Jacobian means by “rationalist self-help” is coming to a gears level understanding of how the world works to aid in becoming well-adapted. So while Scott is right that we shouldn’t expect rationalist self-help to be significantly better than other self-help techniques for becoming a well-adjusted person, Jacobian is right that rationalist self-help is an attempt to become both a well-adjusted person AND a person who participates in developing an understanding of how the world works.
So perhaps you want to learn how to navigate the space of relationships, but you also have this added constraint that you want the theory of how to navigate relationships to be part of a larger understanding of the universe, and not just some hanging chad of random methods without satisfactory explanations of how or why they work. That is to say, you are not willing to settle for unexamined common sense. If that is the case, then rationalist self-help is useful in a way that standard self-help is not.
A little addendum. This is not a new idea. Socrates thought of philosophy as way of life, and tried to found a philosophy which would not only help people discover more truths, but also make them better, braver, and more just people. Stoics and Epicureans continued the tradition of philosophy as a way of life. Since then, there have always been people who have made a way of life out of applying the methods of rationality to normal human endeavors, and human society since then has pretty much always been complicated enough to marginally reward them for the effort.
If someone were to say rationality was great at figuring out whether liberalism or conservatism was better, I would agree that this is the sort of thing rationality should be able to do, in principle.
I think the best answer would be that rationality 101 is tabooing terms and not having the discussion on the level of “is liberalism or conservatism better”.
OpenPhil does invest money into individual political decisions such as prison reform and I would count them to be part of our rationalist project.
If rationalists are trying to do therapy, they are competing with a field of tens of thousands of PhD-level practitioners with all the resources of the academic and health systems who have worked on the problem for decades.
Part of the “resources” is working inside a bureaucratic system where it’s necessary to spend a lot of time jumping through hoops. There’s another large chunk of time invested in intellectual analysis. Most of the people inside academia spend relatively little time in deliberate practice to build skills to do intervention. It’s not surprising to me to have outsiders outperform academia by a large margin.
I don’t know what the actual causal story is here, but it’s at any rate not obviously right that if doctors were good at it then there’d be no reason to increase the age, for a few reasons.
Changing the age doesn’t say anything about who’s how good at what, it says that something has changed.
What’s changed could be that doctors have got worse at breast cancer diagnosis, or that we’ve suddenly discovered that they’re bad. But it could also be, for instance:
That patients have become more anxious and therefore (1) more harmed directly by a false-positive result and (2) more likely to push their doctors for further procedures that would in expectation be bad for them.
That we’ve got better or worse, or discovered we’re better or worse than we thought, at treating certain kinds of cancers at certain stages, in a way that changes the cost/benefit analysis around finding things earlier.
E.g., I’ve heard it said (but I don’t remember by whom and it might be wrong, so this is not health advice) that the benefits of catching cancers early are smaller than they used to be thought to be, because actually the reason why earlier-caught cancers kill you less is that ones you catch when they’re smaller are more likely to be slower-growing ones that were less likely to kill you whenever you caught them; if that’s true and a recent discovery then it would suggest reducing the amount of screening you do.
That previous protocols were designed without sufficient attention to the downsides of testing.
You had a situation where the amount of people who died from cancer were roughly the same in the US and Europe.
At the same time the US started diagnosis earlier and had a higher rate of curing people who are diagnosed with cancer. This does suggest that women got diagnosed in the US with cancer while they wouldn’t have been in Europe with lower testing rates but where whether or not they are treated had in the end little effect on mortality due to breast cancer.
If someone is good at making treatment decisions then he should get better outcomes if he gets more testing data. The fact that this didn’t seem to happen suggests a problem with the decision making of the cancer doctors.
It’s not definite but at the same time I don’t see evidence that the doctors are actually good at making decisions.
I have some pretty complicated thoughts on this, and my heart isn’t really in responding to you because I think some things are helpful for some people, but a sketch of what I’m thinking:
First, a clarification. Some early claims—like the ones I was responding to in my 2009 essay—were that rationalists should be able to basically accomplish miracles, become billionaires with minimal work, unify physics with a couple of years of study, etc. I still occasionally hear claims along those lines. I am still against those, but I interpret you as making weaker claims, like that rationalists can be 10% better at things than nonrationalists, after putting in a decent amount of work. I’m less opposed to those claims, especially if “a decent amount of work” is interpreted as “the same amount of work you would need to get good at those things through other methods”. But I’m still a little bit concerned about them.
First: I’m interpreting “rationalist self-help” to mean rationalist ideas and practices that are helpful for getting common real-life goals like financial, social, and romantic success. I’m not including things like doing charity better, for reasons that I hope will become clear later.
These are the kinds of things most people want, which means two things. First, we should expect a lot of previous effort has gone into optimizing them. Second, we should expect that normal human psychology is designed to optimize them. If we’re trying to do differential equations, we’re outside our brain’s design specs; if we’re trying to gain status and power, we’re operating exactly as designed.
When the brain fails disastrously, it tends to be at things outside the design specs, that don’t matter for things we want. For example, you quoted me describing some disastrous failures in people understanding some philosophy around atheism, and I agree that sort of thing happens often. But this is because it’s outside of our common sense. I can absolutely imagine a normal person saying “Since I can’t prove God doesn’t exist, God must exist”, but it would take a much more screwed-up person to think “Since I can’t prove I can’t fly, I’m going to jump off this cliff.”
Another example: doctors fail miserably on the Bayes mammogram problem, but usually handle actual breast cancer diagnosis okay. And even diagnosing breast cancer is a little outside common sense and everyday life. Faced with the most chimpish possible version of the Bayes mammogram problem—maybe something like “This guy I met at a party claims he’s the king of a distant country, and admittedly he is wearing a crown, but what’s the chance he’s *really* a king?” my guess is people are already near-optimal.
If you have this amazing computer perfectly-tuned for finding strategies in a complex space, I think your best bet is just to throw lots and lots of training data at it, then try navigating the complex space.
I think it’s ironic that you use practicing basketball as your example here, because rationalist techniques very much are *not* practice. If you want to become a better salesman, practice is going out and trying to make lots of sales. I don’t think this is a “rationalist technique” and I think the kind of self-help you’re arguing for is very different (though it may involve better ways to practice). We both agree that practice is useful; I think our remaining disagreement is on whether there are things other than practice that are more useful to do, on the margin, than another unit of practice.
Why do I think this is unlikely?
1. Although rationalists have done pretty well for themselves, they don’t seem to have done too remarkably well. Even lots of leading rationalist organizations are led by people who haven’t put particular effort into anything you could call rationalist self-help! That’s really surprising!
2. Efficient markets. Rationalists developed rationalist self-help by thinking about it for a while. This implies that everyone else left a $100 bill on the ground for the past 4000 years. If there were techniques to improve your financial, social, and romantic success that you could develop just by thinking about them, the same people who figured out the manioc detoxification techniques, or oracle bone randomization for hunting, or all the other amazingly complex adaptations they somehow developed, would have come up with them. Even if they only work in modern society, one of the millions of modern people who wanted financial, social, and romantic success before you would have come up with them. Obviously this isn’t 100% true—someone has to be the first person to discover everything—but you should expect the fruits here to be very high up, high enough that a single community putting in a moderate amount of effort shouldn’t be able to get too many of them.
(some of this becomes less relevant if your idea of rationalist self-help is just collecting the best self-help from elsewhere and giving it a stamp of approval, but then some of the other considerations apply more.)
3. Rationalist self-help starts looking a lot like therapy. If we’re trying to make you a more successful computer programmer using something other than studying computer programming, it’s probably going to involve removing mental blocks or something. Therapy has been pretty well studied, and the most common conclusion is that it is mostly nonspecific factors and the techniques themselves don’t seem to have any special power. I am prepared to suspend this conclusion for occasional miracles when extremely charismatic therapists meet exactly the right patient and some sort of non-scaleable flash of lightning happens, but this also feels different from “the techniques do what they’re supposed to”. If rationalists are trying to do therapy, they are competing with a field of tens of thousands of PhD-level practitioners with all the resources of the academic and health systems who have worked on the problem for decades. This is not the kind of situation that encourages me we can make fast progress. See https://slatestarcodex.com/2019/11/20/book-review-all-therapy-books/ for more on this.
4. General skepticism of premature practical application. It took 300 years between Harvey discovering the circulatory system and anyone being very good at treating circulatory disease. It took 50 years between Pasteur discovering germ theory and anyone being very good at treating infections. It took 250 years between Newton discovering gravity and anyone being very good at flying. I have a lower prior than you on good science immediately translating into useful applications. And I am just not too impressed with the science here. Kahneman and Tversky discovered a grab bag of interesting facts, some of which in retrospect were false. I still don’t think we’re anywhere near the deep understanding of rationality that would make me feel happy here.
This doesn’t mean I think rationality is useless. I think there are lots of areas outside our brain’s normal design specs where rationality is really useful. And because these don’t involve getting sex or money, there’s been a lot less previous exploration of the space and the low hanging fruits haven’t been gobbled up. Or, when the space has been explored, people haven’t done a great job formalizing their insights, or they haven’t spread, or things like that. I am constantly shocked by how much really important knowledge there is sitting around that nobody knows about or thinks about because it doesn’t have an immediate payoff.
Along with all of this, I’m increasingly concerned that anything that has payoff in sex or money is an epistemic death zone. Because you can make so much money teaching it, it attracts too much charlatanry to navigate easily, and it subjects anyone who enters to extreme pressure to become charlatan-adjacent. Because it touches so closely on our emotions and sense of self-worth, it’s a mind-killer in the same way politics are. Because everybody is so different, there’s almost irresistible pressure to push the thing that saved your own life, without checking whether it will help anyone else. Because it’s such a noisy field and RCTs are so hard, I don’t trust us to be able to check our intuitions against reality. And finally, I think there are whole things lurking out there of the approximate size and concerningness of “people are homeostasis-preserving control systems which will expend their entire energy on undoing any change you do to them” that we just have no idea about even though they have the potential to make everything in this sphere useless if we don’t respond to them.
I actually want to expand on the politics analogy. If someone were to say rationality was great at figuring out whether liberalism or conservatism was better, I would agree that this is the sort of thing rationality should be able to do, in principle. But it’s such a horrible topic that has the potential to do so much damage to anyone trying to navigate it that I would be really nervous about it—about whether we were really up to the task, and about what it would do to our movement if we tried. These are some of my concerns around self-help too.
Thank you for the detailed reply. I’m not going to reply point by point because you made a lot of points, but also because I don’t disagree with a lot of it. I do want to offer a couple of intuitions that run counter to your pessimism.
While you’re right that we shouldn’t expect Rationalists to be 10x better at starting companies because of efficient markets, the same is not true of things that contribute to personal happiness. For example: how many people have a strong incentive in helping you build fulfilling romantic relationships? Not the government, not capitalism, not most of your family or friends, often not even your potential partners. Even dating apps make money when you *don’t* successfully seduce your soulmate. But Rationality can be a huge help: learning that your emotions are information, learning about biases and intuitions, learning about communication styles, learning to take 5-minute timers to make plans — all of those can 10x your romantic life.
Going back to efficient markets, I get the sense that a lot of things out there are designed by the 1% most intelligent and ruthless people to take advantage of the 95% and their psychological biases. Outrage media, predatory finance, conspicuous brand consumption and other expensive status ladders, etc. Rationality doesn’t help me design a better YouTube algorithm or finance scam, but at least it allows me to escape the 95% and keeps me away from outrage and in index funds.
Finally, I do believe that the world is getting weirder faster, and the thousands of years of human tradition are becoming obsolete at a faster pace. We are moving ever further from our “design specs”. In this weirding world, I already hit jackpot with Bitcoin and polyamory, two things that couldn’t really exist successfully 100 years ago. Rationality guided me to both. You hit jackpot with blogging— can you imagine your great grand uncle telling you that you’ll become a famous intellectual by writing about cactus people and armchair sociology for free? And we’re both still very young.
For any particular achievement like basketball or making your first million, there are more dedicated practices that help you to your goal faster than Rationality. But for taking advantage of unknown unknowns, the only two things I know that work are Rationality and making friends.
Thanks, all good points.
I think efficient market doesn’t just suggest we can’t do much better at starting companies. It also means we can’t do much better at providing self-help, which is a service that can make people lots of money and status if they do it well.
I’m not sure if you’re using index fund investing as an example of rationalist self-help, or just as a metaphor for it. If you’re using it an example, I worry that your standards are so low that almost any good advice could be rationalist self-help. I think if you’re from a community where you didn’t get a lot of good advice, being part of the rationalist community can be really helpful in exposing you to it (sort of like the theory where college makes you successful because it inducts you into the upper-middle class). I think I got most of my “invest in index funds” level good advice before entering the rationalist community, so I didn’t count that.
Being part of the rationalist community has definitely improved my life, partly through giving me better friends and partly through giving me access to good ideas of the “invest in index funds” level. I hadn’t counted that as part of our discussion, but if I do, then I agree it is great. My archetypal idea of “rationalist self-help” is sitting around at a CFAR workshop trying very hard to examine your mental blocks. I’m not sure if we agree on that or if I’m caricaturing your position.
I’m not up for any gigantic time commitment, but if you want to propose some kind of rationalist self-help exercise that I should try (of the order of 10 minutes/day for a few weeks) to see if I change my mind about it, I’m up for that, though I would also believe you if you said such a halfhearted commitment wouldn’t be a good test.
I have several friends in New York who are a match to my Rationalist friends in age, class, intelligence etc. and who:
Pick S&P 500 stocks based on CNBC and blogs because their intuition tells them they’ve beat the market (but they don’t check or track it, just remember the winners).
Stay in jobs they hate because they don’t have a robust decision process for making such a switch (I used goal factoring, Yoda timer job research, and decision matrices to decide where to work).
Go so back asswards about dating that it hurts to watch (because they can’t think about it systematically).
Retweet Trump with comment.
Throw the most boring parties.
Spend thousands of dollars on therapists but would never do a half-hour debugging session with a friend because “that would be weird”.
In general, live mostly within “social reality” where the only question is “is this weird/acceptable” and never “is this true/false”.
Now perhaps Rationalist self-improvement can’t help them, but if you’re reading LessWrong you may be someone who may snap out of social reality long enough for Rationality to change your life significantly.
> if you want to propose some kind of rationalist self-help exercise that I should try
Different strokes for different folks. You can go through alkjash’s Hammertime Sequence and pick one, although even there the one that he rates lowest (goal factoring) is the one that was the most influential in my own life. You must be friends with CFAR instructors/mentors who know your personality and pressing issues better than I do and can recommend and teach a useful exercise.
Agreed, I see a major problem with an argument that seems to imply that since advice exists elsewhere/wasn’t invented by rationality techniques, a meta-heuristic for aggregating trustworthy sources isn’t hugely valuable.
It seems to me like people who primarily think in terms of weird/acceptable never join the rationality in the first place. Or do you believe that our community has taught people who used to think in those terms to think otherwise?
As I said, someone who is 100% in thrall to social reality will probably not be reading this. But once you peek outside the bubble there is still a long way to enlightenment: first learning how signaling, social roles, tribal impulses etc. shape your behavior so you can avoid their worst effects, then learning to shape the rules of social reality to suit your own goals. Our community is very helpful for getting the first part right, it certainly has been for me. And hopefully we can continue fruitfully exploring the second part too.
What is the error that you’re implying here?
Could be a don’t feed the troll error.
Maybe the incentives are all wrong here, and the most profitable form of “self-help” is one that doesn’t provide long term improvement, so that customers return for more and more books and seminars.
In that case, we can easily do better—better for the “customers”, but less profitable for the “gurus”.
for me the point is about getting *consistently* good ideas, getting reliable ideas where applying scientific method is too hard. It is much less about self-improvement as it is about community improvement in the face of more and more connected (and thus weird) world. Rationality is epistemology for the internet era.
Indeed, I’m surprised to read that, because for the leading Berkeley rationalist organizations (MIRI, CFAR, CEA) I can think of at least one person in the top part of their org chart whom I personally know has done a rationalist self-help push for at least a couple of months before taking said role. (In two of those cases it’s the top person.)
Can you say what organizations you’re thinking of?
Also, yes rationalists do more curating of good advice than invention of it, just as we do with philosophy. But there’s a huge value-add in sorting out the good advice in a domain from the bad advice, and this I think the community does in a more cross-domain way than I see elsewhere.
I’m not sure how much we disagree; it sounds like I disagree with you, but maybe most of that is that we’re using different framings / success thresholds.
If you teleported me 4000 years into the past and deleted all of modernity and rationalism’s object-level knowledge of facts from my head, but let me keep as many thinking heuristics and habits of thought as I wanted, I think those heuristics would have a pretty large positive effect on my ability to pursue mundane happiness and success (compared to someone with the same object-level knowledge but more normal-for-the-time heuristics).
The way you described things here feels to me like it would yield a large overestimate of how much deliberate quality-adjusted optimization (or even experimentation and random-cultural-drift-plus-selection-for-things-rationalists-happen-to-value) human individuals and communities probably put into discovering, using, and propagating “rationalist skills that work” throughout all of human history.
Example: implementation intentions / TAPs are an almost comically simple idea. AFAIK, it has a large effect size that hasn’t fallen victim to the replication crisis (yet!). Humanity crystallized this idea in 1999. A well-calibrated model of “how much optimization humanity has put into generating, using, and propagating rationality techniques” shouldn’t strongly predict that an idea this useful and simple will reach fixation in any culture or group throughout human history before the 1990s, since this in fact never happened. But your paragraph above seems to me like it would predict that many societies throughout history would have made heavy use of TAPs.
I’d similarly worry that the “manioc detoxification is the norm + human societies are as efficient at installing mental habits and group norms as they are at detoxifying manioc” model should predict that the useful heuristics underlying the ‘scientific method’ (e.g., ‘test literally everything’, using controls, trying to randomize) reach fixation in more societies earlier.
Plausibly science is more useful to the group than to the individual; but the same is true for manioc detoxification. There’s something about ideas like science that caused societies not to converge on them earlier. (And this should hold with even more force for any ideas that are hard to come up with, deploy, or detect-the-usefulness-of without science.)
Another thing that it sounds like your stated model predicts: “adopting prediction markets wouldn’t help organizations or societies make money, or they’d already have been widely adopted”. (Of course, what helps the group succeed might not be what helps the relevant decisionmakers in that organization succeed. But it didn’t sound like you expected rationalists to outperform common practice or common sense on “normal” problems, even at the group level.)
I’d disagree! Randomized controlled trials have many moving parts, removing any of which makes them worse than useless. Remove placebo control, and your trials are always positive and you do worse than intuition. Remove double-blinding, same. Remove power calculations, and your trials give random results and you do worse than intuition. Remove significance testing, same. Even in our own advanced civilization, if RCTs give a result different than common sense it’s a 50-50 chance which is right; a primitive civilization who replaced their intuitions with the results of proto-RCTs would be a disaster. This ends up like the creationist example where evolution can’t use half an eye so eyes don’t evolve; obviously this isn’t permanently true with either RCTs or eyes, but in both cases it took a long time for all the parts to evolve independently for other reasons.
Also, you might be underestimating inferential distance—tribes that count “one, two, many” are not going to be able to run trials effectively. Did you know that people didn’t consistently realize you could take an average of more than two numbers until the Middle Ages?
Also, what would these tribes use RCTs to figure out? Whether their traditional healing methods work? St. John’s Wort is a traditional healing method, there have now been about half a dozen high-quality RCTs investigating it, with thousands of patients, and everyone is *still* confused. I am pretty sure primitive civilizations would not really have benefited from this much.
I am less sure about trigger-action plans. I think a history of the idea of procrastination would be very interesting. I get the impression that ancient peoples had very confused beliefs around it. I don’t feel like there is some corpus of ancient anti-procrastination techniques from which TAPs are conspicuously missing, but why not? And premodern people seem weirdly productive compared to moderns in a lot of ways. Overall I notice I am confused here, but this could be an example where you’re right.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t. This seems different to me than the prediction market example, since (as Robin has discussed) decision-makers might self-interestedly prefer not to have prediction markets so they can keep having high status as decision-makers.
I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.
I am curious. Could you expand on this?
Yeah, I was wrong about manioc.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”. Something feels off to me when I visualize a world where all the stupidly-simple epistemic-methods-that-are-instrumentally-useful fruit got plucked 4000 years ago, but where Feynman can see big gains from mental habits like “look at the water” (which I do think happened).
Your other responses make sense. I’ll need to chew on your comments longer to see how much I end up updating overall toward your view.
I’d propose that there’s a massive qualitative difference between black-box results (like RCTs) and gears-level model-building (like Archimedes, Newton, and Feynman). The latter are where basically all of the big gains are, and it does seem like society is under-invested in building gears-level models. One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
I would suspect it’s the other way around, that they have very high depreciation rates; we no longer have Feynman’s gears-level models, for example.
Does your outlook change at all if you try dropping the assumption that there is a “we” who cares about “our movement”? I’ve certainly found a lot of the skills I learned from reading the Sequences useful in my subsequent thinking about politics and social science. (Which thinking, obviously, mostly does not take the form of asking whether “liberalism” or “conservatism” is “better”; political spectra are dimensionality reductions.) I’m trying. I’m not sure how much I’ve succeeded. But I expect my inquiries to be more fruitful than those of people who make appeals to consequences to someone’s “movement” before trying to think about something.
You’re right in catching and calling out the appeal to consequences there, of course.
But aside from me really caring about the movement, I think part of my thought process is that “the movement” is also the source of these self-help techniques. If some people go into this space and then report later with what they think, I am worried that this information is less trustworthy than information that would have come from these same people before they started dealing with this question.
Nobody is claiming that everything around rationalist circles is completely new or invented by them. It’s often looked to me more like separating the more and less useful stuff with various combinations of bottom-up and top-down approaches.
Additionally, I’d like to also identify as someone who is definitely in a much much better place now because they discovered LW almost a decade ago even though I also struggle with akrasia and do less to improve myself than I’d like, I’m very sure that just going to therapy wouldn’t have improved my outcomes in so many areas, especially financially.
There are so many arguments trying to be had at once here that it’s making my head spin.
Here’s one. What do we mean by self-help?
I think by self-help Scott is thinking about becoming psychologically a well-adjusted person. But what I think Jacobian means by “rationalist self-help” is coming to a gears level understanding of how the world works to aid in becoming well-adapted. So while Scott is right that we shouldn’t expect rationalist self-help to be significantly better than other self-help techniques for becoming a well-adjusted person, Jacobian is right that rationalist self-help is an attempt to become both a well-adjusted person AND a person who participates in developing an understanding of how the world works.
So perhaps you want to learn how to navigate the space of relationships, but you also have this added constraint that you want the theory of how to navigate relationships to be part of a larger understanding of the universe, and not just some hanging chad of random methods without satisfactory explanations of how or why they work. That is to say, you are not willing to settle for unexamined common sense. If that is the case, then rationalist self-help is useful in a way that standard self-help is not.
A little addendum. This is not a new idea. Socrates thought of philosophy as way of life, and tried to found a philosophy which would not only help people discover more truths, but also make them better, braver, and more just people. Stoics and Epicureans continued the tradition of philosophy as a way of life. Since then, there have always been people who have made a way of life out of applying the methods of rationality to normal human endeavors, and human society since then has pretty much always been complicated enough to marginally reward them for the effort.
I think the best answer would be that rationality 101 is tabooing terms and not having the discussion on the level of “is liberalism or conservatism better”.
OpenPhil does invest money into individual political decisions such as prison reform and I would count them to be part of our rationalist project.
Part of the “resources” is working inside a bureaucratic system where it’s necessary to spend a lot of time jumping through hoops. There’s another large chunk of time invested in intellectual analysis. Most of the people inside academia spend relatively little time in deliberate practice to build skills to do intervention. It’s not surprising to me to have outsiders outperform academia by a large margin.
If doctors would be good at breast cancer diagnosis, the US wouldn’t have upped the age where testing is done under the Obama administration.
We live in a world where doctors created so many unnecessary operations after they did their diagnosis, that we decided we should do less testing.
I don’t know what the actual causal story is here, but it’s at any rate not obviously right that if doctors were good at it then there’d be no reason to increase the age, for a few reasons.
Changing the age doesn’t say anything about who’s how good at what, it says that something has changed.
What’s changed could be that doctors have got worse at breast cancer diagnosis, or that we’ve suddenly discovered that they’re bad. But it could also be, for instance:
That patients have become more anxious and therefore (1) more harmed directly by a false-positive result and (2) more likely to push their doctors for further procedures that would in expectation be bad for them.
That we’ve got better or worse, or discovered we’re better or worse than we thought, at treating certain kinds of cancers at certain stages, in a way that changes the cost/benefit analysis around finding things earlier.
E.g., I’ve heard it said (but I don’t remember by whom and it might be wrong, so this is not health advice) that the benefits of catching cancers early are smaller than they used to be thought to be, because actually the reason why earlier-caught cancers kill you less is that ones you catch when they’re smaller are more likely to be slower-growing ones that were less likely to kill you whenever you caught them; if that’s true and a recent discovery then it would suggest reducing the amount of screening you do.
That previous protocols were designed without sufficient attention to the downsides of testing.
You had a situation where the amount of people who died from cancer were roughly the same in the US and Europe.
At the same time the US started diagnosis earlier and had a higher rate of curing people who are diagnosed with cancer. This does suggest that women got diagnosed in the US with cancer while they wouldn’t have been in Europe with lower testing rates but where whether or not they are treated had in the end little effect on mortality due to breast cancer.
If someone is good at making treatment decisions then he should get better outcomes if he gets more testing data. The fact that this didn’t seem to happen suggests a problem with the decision making of the cancer doctors.
It’s not definite but at the same time I don’t see evidence that the doctors are actually good at making decisions.