To say that most academic research is anything, you’re going to have to pick a measure over research. Uniform measure is not going to be exciting – you’re going to get almost entirely undergraduate assignments and Third World paper mills. If your weighted sampler is “papers linked in articles about how academia is woke” you’re going to find a high %fake. If your weighed measure is “papers read during work hours by employees at F500 companies” you’ll find a lower, nonzero %fake.
Handwringing over public, vitriolic retractions spats is going to fuck your epistemology via sampling bias. There is no replication crisis in underwater basket weaving
Regardless, the point of the essay is that the overall academic enterprise is not designed to seek the truth. Ideological bias, perverse incentives, social circularity, naive/fake empiricism, and misleading statistics (e.g. p-hacking) compromise the production of truthful research. The sequel essay elaborates on all these ongoing issues. I could expand it even further, but it would probably take me a week to do so, when I have more important priorities.
Hastings is responding to the claim in the title, so if he’s missed the point of the essay, you’ve mistitled it.
I think this would’ve done much better with a more modest title something like “academia is mostly not truthseeking” or something similar. LW is highly suspicious of clickbait titles and inflated claims—the goal to “inform not persuade” is almost the opposite of essay writing elsewhere.
I think you come off as condescending and defensive in the above response, and it probably earned the post an extra downvote or two. I upvoted it, because I think it’s an important topic, and I agree with the spirit if not the literal claim in the title. I think many LWers would roughly agree, they just wouldn’t state it this hyperbolically.
Having been a professional academic for 23 years and considering the epistemology pretty closely, I think the title isn’t far off. As Hastings said, you’d have to quantify what’s academic research, but most of the top-journal stuff I read wouldn’t deserve the commonsense meaning of “fake”. But it is so low-quality as a result of conflicting incentives that calling most of it “confused” or “wrong” wouldn’t be a stretch.
I didn’t come up with the title for the essay, but I re-titled this LW post, so thank you for your suggestion. In hindsight, I’ll agree that my comment came off as condescending to some extent, so I edited that as well. I just haven’t been in the best mood when I post on this site since I’ve gotten used to people giving me downvotes, disagreeing with my comments, and sometimes sending condescending comments into my inbox, though that doesn’t justify me being condescending to others. Regardless of the essay’s title, the essay’s contents raise serious questions about whether academia is intellectually honest.
I’ve thought about expanding my sequel essay even further to more precisely quantify and evaluate the research in each academic field, but I ended up not doing this since it would probably take me a week or longer to further detail everything. Another problem was that even if I finished it, people could always say that I failed to evaluate this or that, since there are tens of thousands of papers out there. Another issue is that not everybody agrees on what counts as “fake”, as I mentioned in the sequel essay. So even if someone quantified all academic research as best as they can, it’s not possible for them to make an overall assessment that a majority of people would agree with.
For these reasons, I don’t think it’s productive to quantify whether most academic research is true or false or high-quality or low-quality, which would explain why the author didn’t do so. I think it’s more productive to analyze how academia and the academic research process work and what kind of output such a system is likely to produce. From everything that I’ve seen across a multitude of fields, my overall impression is that most academic research tends to be low-quality. Blithering Genius’s analysis and my own analysis both conclude that that’s probably the case for most academic research.
Anyway, I appreciate your comment and reading your thoughts.
To say that most academic research is anything, you’re going to have to pick a measure over research. Uniform measure is not going to be exciting – you’re going to get almost entirely undergraduate assignments and Third World paper mills. If your weighted sampler is “papers linked in articles about how academia is woke” you’re going to find a high %fake. If your weighed measure is “papers read during work hours by employees at F500 companies” you’ll find a lower, nonzero %fake.
Handwringing over public, vitriolic retractions spats is going to fuck your epistemology via sampling bias. There is no replication crisis in underwater basket weaving
The bigger issue is that not everybody agrees on what’s true or false. I did my best to address these considerations in greater depth in my sequel essay: https://zerocontradictions.net/epistemology/academia-critique
Regardless, the point of the essay is that the overall academic enterprise is not designed to seek the truth. Ideological bias, perverse incentives, social circularity, naive/fake empiricism, and misleading statistics (e.g. p-hacking) compromise the production of truthful research. The sequel essay elaborates on all these ongoing issues. I could expand it even further, but it would probably take me a week to do so, when I have more important priorities.
Hastings is responding to the claim in the title, so if he’s missed the point of the essay, you’ve mistitled it.
I think this would’ve done much better with a more modest title something like “academia is mostly not truthseeking” or something similar. LW is highly suspicious of clickbait titles and inflated claims—the goal to “inform not persuade” is almost the opposite of essay writing elsewhere.
I think you come off as condescending and defensive in the above response, and it probably earned the post an extra downvote or two. I upvoted it, because I think it’s an important topic, and I agree with the spirit if not the literal claim in the title. I think many LWers would roughly agree, they just wouldn’t state it this hyperbolically.
Having been a professional academic for 23 years and considering the epistemology pretty closely, I think the title isn’t far off. As Hastings said, you’d have to quantify what’s academic research, but most of the top-journal stuff I read wouldn’t deserve the commonsense meaning of “fake”. But it is so low-quality as a result of conflicting incentives that calling most of it “confused” or “wrong” wouldn’t be a stretch.
I didn’t come up with the title for the essay, but I re-titled this LW post, so thank you for your suggestion. In hindsight, I’ll agree that my comment came off as condescending to some extent, so I edited that as well. I just haven’t been in the best mood when I post on this site since I’ve gotten used to people giving me downvotes, disagreeing with my comments, and sometimes sending condescending comments into my inbox, though that doesn’t justify me being condescending to others. Regardless of the essay’s title, the essay’s contents raise serious questions about whether academia is intellectually honest.
I’ve thought about expanding my sequel essay even further to more precisely quantify and evaluate the research in each academic field, but I ended up not doing this since it would probably take me a week or longer to further detail everything. Another problem was that even if I finished it, people could always say that I failed to evaluate this or that, since there are tens of thousands of papers out there. Another issue is that not everybody agrees on what counts as “fake”, as I mentioned in the sequel essay. So even if someone quantified all academic research as best as they can, it’s not possible for them to make an overall assessment that a majority of people would agree with.
For these reasons, I don’t think it’s productive to quantify whether most academic research is true or false or high-quality or low-quality, which would explain why the author didn’t do so. I think it’s more productive to analyze how academia and the academic research process work and what kind of output such a system is likely to produce. From everything that I’ve seen across a multitude of fields, my overall impression is that most academic research tends to be low-quality. Blithering Genius’s analysis and my own analysis both conclude that that’s probably the case for most academic research.
Anyway, I appreciate your comment and reading your thoughts.