Alief is sort of an easy one. I found for example instrumental versus terminal to be a hard one to track down the source for. I think it would be a mistake to underestimate the difficulty of tracking down a hard citation, especially if your prior is that there’s nothing to track down past a certain point. For example let’s say you believe that Scott Alexander is the originator of Motte/Bailey, thankfully if you go back to his ‘original source’ for information you’ll see that he’s clearly referencing someone elses idea and be led back. But if you believed that and he hadn’t cited his source, well this is the top search results for Motte-Bailey:
I could easily see our intrepid researcher going “oh, ratwiki is just a tertiary source no need to look at that, Scott’s post is at the top so it’s probably the canonical reference...junk underneath yeah I think we’re good”. Confirmation bias causes us to assume our best known etymology is the etymology.
For example, mathematical history is littered with examples where important concepts are named, not after the originator, but after the person who first really made good use of the concept and/or really popularized it (I particularly have in mind motte and bailey as I say this). Most mathematicians probably couldn’t tell you the originator of most of the concepts they use. This does not seem to have done much damage at all.
This isn’t an issue with using the concepts, this is an issue with being able to tap into the original research or sources of received knowledge. Previous performance predicts future performance, and places where one good idea originated probably have others you could use. It’s possible for someone to popularize one of a dozen really great ideas, but never get around to the other eleven. If we want to make full use of intellectual work it’s a good idea not to set ourselves up for that to silently happen all the time.
Confirmation bias causes us to assume our best known etymology is the etymology.
But what is the actual bad thing that happens if this happens?
Previous performance predicts future performance, and places where one good idea originated probably have others you could use.
This is not as true as it sounds if you’re selecting on previous performance as opposed to observing it; said another way, you aren’t accounting for regression to the mean.
In a model where people have some hidden “propensity to have good ideas” stat and your probability of generating a good idea at any time is some function of this stat + noise, for many plausible distributions of this stat and the noise, most good ideas will have been had by a person who has one or two good ideas, simply because there are many more people who are okay at having good ideas + get lucky than there are people who are extremely good at having good ideas.
In any case, searching for people with good ideas is only one of many things that people might want to do and I don’t see the hurry in wanting everyone to have gone through the first step or two of this process if they have other things to do.
Alief is sort of an easy one. I found for example instrumental versus terminal to be a hard one to track down the source for. I think it would be a mistake to underestimate the difficulty of tracking down a hard citation, especially if your prior is that there’s nothing to track down past a certain point. For example let’s say you believe that Scott Alexander is the originator of Motte/Bailey, thankfully if you go back to his ‘original source’ for information you’ll see that he’s clearly referencing someone elses idea and be led back. But if you believed that and he hadn’t cited his source, well this is the top search results for Motte-Bailey:
I could easily see our intrepid researcher going “oh, ratwiki is just a tertiary source no need to look at that, Scott’s post is at the top so it’s probably the canonical reference...junk underneath yeah I think we’re good”. Confirmation bias causes us to assume our best known etymology is the etymology.
This isn’t an issue with using the concepts, this is an issue with being able to tap into the original research or sources of received knowledge. Previous performance predicts future performance, and places where one good idea originated probably have others you could use. It’s possible for someone to popularize one of a dozen really great ideas, but never get around to the other eleven. If we want to make full use of intellectual work it’s a good idea not to set ourselves up for that to silently happen all the time.
But what is the actual bad thing that happens if this happens?
This is not as true as it sounds if you’re selecting on previous performance as opposed to observing it; said another way, you aren’t accounting for regression to the mean.
In a model where people have some hidden “propensity to have good ideas” stat and your probability of generating a good idea at any time is some function of this stat + noise, for many plausible distributions of this stat and the noise, most good ideas will have been had by a person who has one or two good ideas, simply because there are many more people who are okay at having good ideas + get lucky than there are people who are extremely good at having good ideas.
In any case, searching for people with good ideas is only one of many things that people might want to do and I don’t see the hurry in wanting everyone to have gone through the first step or two of this process if they have other things to do.