There are indeed a couple of different ways I do mean it, but my best specific examples come from athletics. About eight or nine years ago I started getting seriously interested in long distance trail running. Like most enthusiastic autodidacts I started reading lots of material about shoes, clothing, hydration, nutrition, electrolytes, training, and so on. As I’m sure you’ve seen, a lot of people on the Internet can get paralyzed by analysis in the face of vast easily available information. In particular, they have a lot of trouble sorting out conflicting information gained from other knowledgeable people.
Frequently, further research will help you arrive at less-wrong conclusions. However, in some endeavors there really is a great deal of individual variation, and you just have to engage in lengthy, often-frustrating self-experimentation to figure out what techniques or training methods work best for you. This base of experience can’t really be replaced by secondary research. Where research skill comes in, though, is in figuring out where to focus that secondary research (and this in itself is a skill that is honed by experience). As a friend of mine likes to put it: the best practitioners of [insert skill here] in the world perform almost all components of their skill the same way. They all have weird idiosyncrasies too. The place to focus your research is in the areas they have in common.
Anyway, this is a longer response than I had intended, and undoubtedly this is not new to you; it’s just variation on standard cognitive bias. However, I think that deferral of experience and self-experimentation in favor of secondary research (aka, analysis paralysis) is a common bias blind-spot among rationality enthusiasts.
I agree it’s a common failure mode, and that the areas in which I’ve done cheap self-experimentation and kept notes showed remarkably quick improvement. There are some LW posts expounding the meme of actuallytrying things, but it’s less prominent than it ought to be.
There are a couple of things you could mean by this. Can you give an example?
There are indeed a couple of different ways I do mean it, but my best specific examples come from athletics. About eight or nine years ago I started getting seriously interested in long distance trail running. Like most enthusiastic autodidacts I started reading lots of material about shoes, clothing, hydration, nutrition, electrolytes, training, and so on. As I’m sure you’ve seen, a lot of people on the Internet can get paralyzed by analysis in the face of vast easily available information. In particular, they have a lot of trouble sorting out conflicting information gained from other knowledgeable people.
Frequently, further research will help you arrive at less-wrong conclusions. However, in some endeavors there really is a great deal of individual variation, and you just have to engage in lengthy, often-frustrating self-experimentation to figure out what techniques or training methods work best for you. This base of experience can’t really be replaced by secondary research. Where research skill comes in, though, is in figuring out where to focus that secondary research (and this in itself is a skill that is honed by experience). As a friend of mine likes to put it: the best practitioners of [insert skill here] in the world perform almost all components of their skill the same way. They all have weird idiosyncrasies too. The place to focus your research is in the areas they have in common.
Anyway, this is a longer response than I had intended, and undoubtedly this is not new to you; it’s just variation on standard cognitive bias. However, I think that deferral of experience and self-experimentation in favor of secondary research (aka, analysis paralysis) is a common bias blind-spot among rationality enthusiasts.
I agree it’s a common failure mode, and that the areas in which I’ve done cheap self-experimentation and kept notes showed remarkably quick improvement. There are some LW posts expounding the meme of actually trying things, but it’s less prominent than it ought to be.