I don’t think people should feel obligated to read all that’s been written on a topic before posting their thoughts on that topic to Less Wrong, especially if that writing is not supported by randomized controlled trials (unsure if this is true for the writings you cite).
Fair point about the bar for posting, but this doesn’t read like “posting their [tentative] thoughts,” it reads like conclusions based on extensive review. As a matter of good epistemics, the difference should be made clearer. Similarly, if you dismiss large parts of the literature, it would be good to at least let people know what you think should be ignored, so they don’t waste their time, and even better, why, so they can decide if they agree.
As a side point, I think that considering RCTs as a source of evidence in this domain is a strange bar. There’s lots of case study and other quantitative observational evidence that supports these other approaches, and specifying what evidence counts is, as the phrase goes, logically rude—how would you even design an RCT to test these theories?
If it’s hard to conduct RCTs in a domain, it’s hard to have reliable knowledge about it period. Who’s to say whether your anecdotal observations & conclusions beat mine or someone else’s? One way is to check whether someone’s job is high status enough that their writings on the topic can be considered part of “the literature”. But this is a weak heuristic IMO.
I strongly disagree. There are many domains where we have knowledge with little or no ability to conduct RCTs—geology, evolutionary theory, astronomy, etc. The models work because we have strong Bayesian evidence for them—as I understood it, this was the point of a large section of the sequences, so I’m not going to try to re-litigate that debate here.
Agreed—one is not expected to read all that’s been written on a topic. However, one should acknowledge alternate models when they’re pointed out, and your summaries and conclusions will carry a lot more weight if you can explain why you prefer those over the other ideas.
I do also think that Samo has engaged with a large part of the literature, and it just doesn’t come through in this post. (My model is that he has dismissed a large part of the existing literature for pretty decent reasons, but am not confident)
I don’t think people should feel obligated to read all that’s been written on a topic before posting their thoughts on that topic to Less Wrong, especially if that writing is not supported by randomized controlled trials (unsure if this is true for the writings you cite).
Fair point about the bar for posting, but this doesn’t read like “posting their [tentative] thoughts,” it reads like conclusions based on extensive review. As a matter of good epistemics, the difference should be made clearer. Similarly, if you dismiss large parts of the literature, it would be good to at least let people know what you think should be ignored, so they don’t waste their time, and even better, why, so they can decide if they agree.
As a side point, I think that considering RCTs as a source of evidence in this domain is a strange bar. There’s lots of case study and other quantitative observational evidence that supports these other approaches, and specifying what evidence counts is, as the phrase goes, logically rude—how would you even design an RCT to test these theories?
If it’s hard to conduct RCTs in a domain, it’s hard to have reliable knowledge about it period. Who’s to say whether your anecdotal observations & conclusions beat mine or someone else’s? One way is to check whether someone’s job is high status enough that their writings on the topic can be considered part of “the literature”. But this is a weak heuristic IMO.
I strongly disagree. There are many domains where we have knowledge with little or no ability to conduct RCTs—geology, evolutionary theory, astronomy, etc. The models work because we have strong Bayesian evidence for them—as I understood it, this was the point of a large section of the sequences, so I’m not going to try to re-litigate that debate here.
Agreed—one is not expected to read all that’s been written on a topic. However, one should acknowledge alternate models when they’re pointed out, and your summaries and conclusions will carry a lot more weight if you can explain why you prefer those over the other ideas.
I do also think that Samo has engaged with a large part of the literature, and it just doesn’t come through in this post. (My model is that he has dismissed a large part of the existing literature for pretty decent reasons, but am not confident)