I’d similarly worry that the “manioc detoxification is the norm + human societies are as efficient at installing mental habits and group norms as they are at detoxifying manioc” model should predict that the useful heuristics underlying the ‘scientific method’ (e.g., ‘test literally everything’, using controls, trying to randomize) reach fixation in more societies earlier.
I’d disagree! Randomized controlled trials have many moving parts, removing any of which makes them worse than useless. Remove placebo control, and your trials are always positive and you do worse than intuition. Remove double-blinding, same. Remove power calculations, and your trials give random results and you do worse than intuition. Remove significance testing, same. Even in our own advanced civilization, if RCTs give a result different than common sense it’s a 50-50 chance which is right; a primitive civilization who replaced their intuitions with the results of proto-RCTs would be a disaster. This ends up like the creationist example where evolution can’t use half an eye so eyes don’t evolve; obviously this isn’t permanently true with either RCTs or eyes, but in both cases it took a long time for all the parts to evolve independently for other reasons.
Also, you might be underestimating inferential distance—tribes that count “one, two, many” are not going to be able to run trials effectively. Did you know that people didn’t consistently realize you could take an average of more than two numbers until the Middle Ages?
Also, what would these tribes use RCTs to figure out? Whether their traditional healing methods work? St. John’s Wort is a traditional healing method, there have now been about half a dozen high-quality RCTs investigating it, with thousands of patients, and everyone is *still* confused. I am pretty sure primitive civilizations would not really have benefited from this much.
I am less sure about trigger-action plans. I think a history of the idea of procrastination would be very interesting. I get the impression that ancient peoples had very confused beliefs around it. I don’t feel like there is some corpus of ancient anti-procrastination techniques from which TAPs are conspicuously missing, but why not? And premodern people seem weirdly productive compared to moderns in a lot of ways. Overall I notice I am confused here, but this could be an example where you’re right.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t. This seems different to me than the prediction market example, since (as Robin has discussed) decision-makers might self-interestedly prefer not to have prediction markets so they can keep having high status as decision-makers.
Randomized controlled trials have many moving parts, removing any of which makes them worse than useless.
I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t.
Yeah, I was wrong about manioc.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”. Something feels off to me when I visualize a world where all the stupidly-simple epistemic-methods-that-are-instrumentally-useful fruit got plucked 4000 years ago, but where Feynman can see big gains from mental habits like “look at the water” (which I do think happened).
Your other responses make sense. I’ll need to chew on your comments longer to see how much I end up updating overall toward your view.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”.
I’d propose that there’s a massive qualitative difference between black-box results (like RCTs) and gears-level model-building (like Archimedes, Newton, and Feynman). The latter are where basically all of the big gains are, and it does seem like society is under-invested in building gears-level models. One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
I would suspect it’s the other way around, that they have very high depreciation rates; we no longer have Feynman’s gears-level models, for example.
I’d disagree! Randomized controlled trials have many moving parts, removing any of which makes them worse than useless. Remove placebo control, and your trials are always positive and you do worse than intuition. Remove double-blinding, same. Remove power calculations, and your trials give random results and you do worse than intuition. Remove significance testing, same. Even in our own advanced civilization, if RCTs give a result different than common sense it’s a 50-50 chance which is right; a primitive civilization who replaced their intuitions with the results of proto-RCTs would be a disaster. This ends up like the creationist example where evolution can’t use half an eye so eyes don’t evolve; obviously this isn’t permanently true with either RCTs or eyes, but in both cases it took a long time for all the parts to evolve independently for other reasons.
Also, you might be underestimating inferential distance—tribes that count “one, two, many” are not going to be able to run trials effectively. Did you know that people didn’t consistently realize you could take an average of more than two numbers until the Middle Ages?
Also, what would these tribes use RCTs to figure out? Whether their traditional healing methods work? St. John’s Wort is a traditional healing method, there have now been about half a dozen high-quality RCTs investigating it, with thousands of patients, and everyone is *still* confused. I am pretty sure primitive civilizations would not really have benefited from this much.
I am less sure about trigger-action plans. I think a history of the idea of procrastination would be very interesting. I get the impression that ancient peoples had very confused beliefs around it. I don’t feel like there is some corpus of ancient anti-procrastination techniques from which TAPs are conspicuously missing, but why not? And premodern people seem weirdly productive compared to moderns in a lot of ways. Overall I notice I am confused here, but this could be an example where you’re right.
I’m confused about how manioc detox is more useful to the group than the individual—each individual self-interestedly would prefer to detox manioc, since they will die (eventually) if they don’t. This seems different to me than the prediction market example, since (as Robin has discussed) decision-makers might self-interestedly prefer not to have prediction markets so they can keep having high status as decision-makers.
I disagree with this- for one thing, they caught on before those patches were known, and still helped make progress. The patches help you discern smaller effects, with less bias, and better understanding of whether the result is a fluke; but the basic version of a randomized trial between two interventions is still vastly superior to human intuition when it comes to something with a large but not blindingly obvious effect size.
I am curious. Could you expand on this?
Yeah, I was wrong about manioc.
Something about the “science is fragile” argument feels off to me. Perhaps it’s that I’m not really thinking about RCTs; I’m looking at Archimedes, Newton, and Feynman, and going “surely there’s something small that could have been tweaked about culture beforehand to make some of this low-hanging scientific fruit get grabbed earlier by a bunch of decent thinkers, rather than everything needing to wait for lone geniuses”. Something feels off to me when I visualize a world where all the stupidly-simple epistemic-methods-that-are-instrumentally-useful fruit got plucked 4000 years ago, but where Feynman can see big gains from mental habits like “look at the water” (which I do think happened).
Your other responses make sense. I’ll need to chew on your comments longer to see how much I end up updating overall toward your view.
I’d propose that there’s a massive qualitative difference between black-box results (like RCTs) and gears-level model-building (like Archimedes, Newton, and Feynman). The latter are where basically all of the big gains are, and it does seem like society is under-invested in building gears-level models. One possible economic reason for the under-investment is that gears-level models have very low depreciation rates, so they pay off over a very long timescale.
I would suspect it’s the other way around, that they have very high depreciation rates; we no longer have Feynman’s gears-level models, for example.