I’m not sure how much we disagree; it sounds like I disagree with you, but maybe most of that is that we’re using different framings / success thresholds.
Efficient markets. Rationalists developed rationalist self-help by thinking about it for a while. This implies that everyone else left a $100 bill on the ground for the past 4000 years. If there were techniques to improve your financial, social, and romantic success that you could develop just by thinking about them, the same people who figured out the manioc detoxification techniques, or oracle bone randomization for hunting, or all the other amazingly complex adaptations they somehow developed, would have come up with them.
If you teleported me 4000 years into the past and deleted all of modernity and rationalism’s object-level knowledge of facts from my head, but let me keep as many thinking heuristics and habits of thought as I wanted, I think those heuristics would have a pretty large positive effect on my ability to pursue mundane happiness and success (compared to someone with the same object-level knowledge but more normal-for-the-time heuristics).
The way you described things here feels to me like it would yield a large overestimate of how much deliberate quality-adjusted optimization (or even experimentation and random-cultural-drift-plus-selection-for-things-rationalists-happen-to-value) human individuals and communities probably put into discovering, using, and propagating “rationalist skills that work” throughout all of human history.
Example: implementation intentions / TAPs are an almost comically simple idea. AFAIK, it has a large effect size that hasn’t fallen victim to the replication crisis (yet!). Humanity crystallized this idea in 1999. A well-calibrated model of “how much optimization humanity has put into generating, using, and propagating rationality techniques” shouldn’t strongly predict that an idea this useful and simple will reach fixation in any culture or group throughout human history before the 1990s, since this in fact never happened. But your paragraph above seems to me like it would predict that many societies throughout history would have made heavy use of TAPs.
I’d similarly worry that the “manioc detoxification is the norm + human societies are as efficient at installing mental habits and group norms as they are at detoxifying manioc” model should predict that the useful heuristics underlying the ‘scientific method’ (e.g., ‘test literally everything’, using controls, trying to randomize) reach fixation in more societies earlier.
Plausibly science is more useful to the group than to the individual; but the same is true for manioc detoxification. There’s something about ideas like science that caused societies not to converge on them earlier. (And this should hold with even more force for any ideas that are hard to come up with, deploy, or detect-the-usefulness-of without science.)
Another thing that it sounds like your stated model predicts: “adopting prediction markets wouldn’t help organizations or societies make money, or they’d already have been widely adopted”. (Of course, what helps the group succeed might not be what helps the relevant decisionmakers in that organization succeed. But it didn’t sound like you expected rationalists to outperform common practice or common sense on “normal” problems, even at the group level.)
I’d similarly worry that the “manioc detoxification is the norm + human societies are as efficient at installing mental habits and group norms as they are at detoxifying manioc” model should predict that the useful heuristics underlying the ‘scientific method’ (e.g., ‘test literally everything’, using controls, trying to randomize) reach fixation in more societies earlier.
I’d disagree! Randomized controlled trials have many moving parts, removing any of which makes them worse than useless. Remove placebo control, and your trials are always positive and you do worse than intuition. Remove double-blinding, same. Remove power calculations, and your trials give random results and you do worse than intuition. Remove significance testing, same. Even in our own advanced civilization, if RCTs give a result different than common sense it’s a 50-50 chance which is right; a primitive civilization who replaced their intuitions with the results of proto-RCTs would be a disaster. This ends up like the creationist example where evolution can’t use half an eye so eyes don’t evolve; obviously this isn’t permanently true with either RCTs or eyes, but in both cases it took a long time for all the parts to evolve independently for other reasons.
Also, you might be underestimating inferential distance—tribes that count “one, two, many” are not going to be able to run trials effectively. Did you know that people didn’t consistently realize you could take an average of more than two numbers until the Middle Ages?
Also, what would these tribes use RCTs to figure out? Whether their traditional healing methods work? St. John’s Wort is a traditional healing method, there have now been about half a dozen high-quality RCTs investigating it, with thousands of patients, and everyone is *still* confused. I am pretty sure primitive civilizations would not really have benefited from this much.
I am less sure about trigger-action plans. I think a history of the idea of procrastination would be very interesting. I get the impression that ancient peoples had very confused beliefs around it. I don’t feel like there is some corpus of ancient anti-procrastination techniques from which TAPs are conspicuously missing, but why not? And premodern people seem weirdly productive compared to moderns in a lot of ways. Overall I notice I am confused here, but this could be an example where you’re right.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t. This seems different to me than the prediction market example, since (as Robin has discussed) decision-makers might self-interestedly prefer not to have prediction markets so they can keep having high status as decision-makers.
Randomized controlled trials have many moving parts, removing any of which makes them worse than useless.
I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t.
Yeah, I was wrong about manioc.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”. Something feels off to me when I visualize a world where all the stupidly-simple epistemic-methods-that-are-instrumentally-useful fruit got plucked 4000 years ago, but where Feynman can see big gains from mental habits like “look at the water” (which I do think happened).
Your other responses make sense. I’ll need to chew on your comments longer to see how much I end up updating overall toward your view.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”.
I’d propose that there’s a massive qualitative difference between black-box results (like RCTs) and gears-level model-building (like Archimedes, Newton, and Feynman). The latter are where basically all of the big gains are, and it does seem like society is under-invested in building gears-level models. One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
I would suspect it’s the other way around, that they have very high depreciation rates; we no longer have Feynman’s gears-level models, for example.
I’m not sure how much we disagree; it sounds like I disagree with you, but maybe most of that is that we’re using different framings / success thresholds.
If you teleported me 4000 years into the past and deleted all of modernity and rationalism’s object-level knowledge of facts from my head, but let me keep as many thinking heuristics and habits of thought as I wanted, I think those heuristics would have a pretty large positive effect on my ability to pursue mundane happiness and success (compared to someone with the same object-level knowledge but more normal-for-the-time heuristics).
The way you described things here feels to me like it would yield a large overestimate of how much deliberate quality-adjusted optimization (or even experimentation and random-cultural-drift-plus-selection-for-things-rationalists-happen-to-value) human individuals and communities probably put into discovering, using, and propagating “rationalist skills that work” throughout all of human history.
Example: implementation intentions / TAPs are an almost comically simple idea. AFAIK, it has a large effect size that hasn’t fallen victim to the replication crisis (yet!). Humanity crystallized this idea in 1999. A well-calibrated model of “how much optimization humanity has put into generating, using, and propagating rationality techniques” shouldn’t strongly predict that an idea this useful and simple will reach fixation in any culture or group throughout human history before the 1990s, since this in fact never happened. But your paragraph above seems to me like it would predict that many societies throughout history would have made heavy use of TAPs.
I’d similarly worry that the “manioc detoxification is the norm + human societies are as efficient at installing mental habits and group norms as they are at detoxifying manioc” model should predict that the useful heuristics underlying the ‘scientific method’ (e.g., ‘test literally everything’, using controls, trying to randomize) reach fixation in more societies earlier.
Plausibly science is more useful to the group than to the individual; but the same is true for manioc detoxification. There’s something about ideas like science that caused societies not to converge on them earlier. (And this should hold with even more force for any ideas that are hard to come up with, deploy, or detect-the-usefulness-of without science.)
Another thing that it sounds like your stated model predicts: “adopting prediction markets wouldn’t help organizations or societies make money, or they’d already have been widely adopted”. (Of course, what helps the group succeed might not be what helps the relevant decisionmakers in that organization succeed. But it didn’t sound like you expected rationalists to outperform common practice or common sense on “normal” problems, even at the group level.)
I’d disagree! Randomized controlled trials have many moving parts, removing any of which makes them worse than useless. Remove placebo control, and your trials are always positive and you do worse than intuition. Remove double-blinding, same. Remove power calculations, and your trials give random results and you do worse than intuition. Remove significance testing, same. Even in our own advanced civilization, if RCTs give a result different than common sense it’s a 50-50 chance which is right; a primitive civilization who replaced their intuitions with the results of proto-RCTs would be a disaster. This ends up like the creationist example where evolution can’t use half an eye so eyes don’t evolve; obviously this isn’t permanently true with either RCTs or eyes, but in both cases it took a long time for all the parts to evolve independently for other reasons.
Also, you might be underestimating inferential distance—tribes that count “one, two, many” are not going to be able to run trials effectively. Did you know that people didn’t consistently realize you could take an average of more than two numbers until the Middle Ages?
Also, what would these tribes use RCTs to figure out? Whether their traditional healing methods work? St. John’s Wort is a traditional healing method, there have now been about half a dozen high-quality RCTs investigating it, with thousands of patients, and everyone is *still* confused. I am pretty sure primitive civilizations would not really have benefited from this much.
I am less sure about trigger-action plans. I think a history of the idea of procrastination would be very interesting. I get the impression that ancient peoples had very confused beliefs around it. I don’t feel like there is some corpus of ancient anti-procrastination techniques from which TAPs are conspicuously missing, but why not? And premodern people seem weirdly productive compared to moderns in a lot of ways. Overall I notice I am confused here, but this could be an example where you’re right.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t. This seems different to me than the prediction market example, since (as Robin has discussed) decision-makers might self-interestedly prefer not to have prediction markets so they can keep having high status as decision-makers.
I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.
I am curious. Could you expand on this?
Yeah, I was wrong about manioc.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”. Something feels off to me when I visualize a world where all the stupidly-simple epistemic-methods-that-are-instrumentally-useful fruit got plucked 4000 years ago, but where Feynman can see big gains from mental habits like “look at the water” (which I do think happened).
Your other responses make sense. I’ll need to chew on your comments longer to see how much I end up updating overall toward your view.
I’d propose that there’s a massive qualitative difference between black-box results (like RCTs) and gears-level model-building (like Archimedes, Newton, and Feynman). The latter are where basically all of the big gains are, and it does seem like society is under-invested in building gears-level models. One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
I would suspect it’s the other way around, that they have very high depreciation rates; we no longer have Feynman’s gears-level models, for example.