It’s not clear to me what the disagreement is here. Which heuristic are you defending again?
If it’s not published, it’s not science
Response: Can we skip the pointless categorizations and evaluate whether material is valid or useful on a case by case basis? Clearly there is some material that has not been published that is useful (see: This website).
If it’s not published in a peer-reviewed journal, there’s no reason to treat it any differently than the ramblings of the Time Cube guy.
Response: Ahh yes, anything not peer-reviewed clearly contains Time Cube-levels of crazy.
Or none of the above? I’m not sure we actually disagree on anything here.
The problem of publication bias is another reason to be wary of the publication heuristic recommended a few comments above. If you follow that heuristic rigorously, you will necessarily expose yourself to the systematic distortions arising from publication bias.
This is not to say that you should therefore believe the first unpublished paper you come across. It’s only to point out that the publication heuristic has certain problems, and while not ignored, it should be supplemented. You ignore unpublished research at your peril. In an ideal world, peer review filters the good from the bad and nothing else. We do not live in an ideal world, so caveat lector.
The process of journal publication is also extremely slow, so that refusal to read unpublished research threatens to retard your progress. This link gives time to publication for several journals—the average appears to be well over a year and approaching two years. What’s two years in Internet Time? Pretty long.
It’s not clear to me what the disagreement is here. Which heuristic are you defending again?
Response: Can we skip the pointless categorizations and evaluate whether material is valid or useful on a case by case basis? Clearly there is some material that has not been published that is useful (see: This website).
Response: Ahh yes, anything not peer-reviewed clearly contains Time Cube-levels of crazy.
Or none of the above? I’m not sure we actually disagree on anything here.
The problem of publication bias is another reason to be wary of the publication heuristic recommended a few comments above. If you follow that heuristic rigorously, you will necessarily expose yourself to the systematic distortions arising from publication bias.
This is not to say that you should therefore believe the first unpublished paper you come across. It’s only to point out that the publication heuristic has certain problems, and while not ignored, it should be supplemented. You ignore unpublished research at your peril. In an ideal world, peer review filters the good from the bad and nothing else. We do not live in an ideal world, so caveat lector.
The process of journal publication is also extremely slow, so that refusal to read unpublished research threatens to retard your progress. This link gives time to publication for several journals—the average appears to be well over a year and approaching two years. What’s two years in Internet Time? Pretty long.