A spot check is supposed to take a number of random sources and check them, not pick the one claim you find most suspicious (that isn’t even about co-ops) and use that to dismiss the entire literature on co-ops.
A spot check is supposed to take a number of random sources and check them, not pick the one claim you find most suspicious (that isn’t even about co-ops)
There seem to be three parts to this objections:
I did not spot-check sufficiently many claims
I filtered the claim to spot-check based on being the most suspicious
I did not filter the claims based on being about co-ops
With regards to point 1, I agree that I cannot know your accuracy very precisely without doing more checks, but the problem is that each check takes time and there are a lot of posts on the internet to read, so I have to limit how much I search.
With regards to point 2, it’s not that I spot-checked the most suspicious one, rather it’s that I spot-checked the first suspicious one. This is still a filter on suspiciousness, but a much weaker one. I think some filter on suspiciousness is appropriate since suspicious claims are also the ones I can learn the most from if they turn out to be true, as claims become suspicious through a combination of being unlikely and having big implications.
With regards to point 3, if you put much more effort into verifying the accuracy of your claims about co-ops than your claims about other stuff, then your accuracy on co-ops might not be that correlated with your accuracy on other stuff, and I ought to do a spot-check specifically on your claims about co-ops. I don’t know if that is true. If it is true, it might also be helpful to mention it as a disclaimer in the post so people know what claims to mostly focus on.
and use that to dismiss the entire literature on co-ops.
So it’s not so much that I’m dismissing the entire literature on co-ops. (Or well, I would generally dismiss any social science that I haven’t done some surface checks of. But that’s different from my comments here.) It’s more that I’m dismissing your literature review of the literature.
A spot check is supposed to take a number of random sources and check them, not pick the one claim you find most suspicious (that isn’t even about co-ops) and use that to dismiss the entire literature on co-ops.
There seem to be three parts to this objections:
I did not spot-check sufficiently many claims
I filtered the claim to spot-check based on being the most suspicious
I did not filter the claims based on being about co-ops
With regards to point 1, I agree that I cannot know your accuracy very precisely without doing more checks, but the problem is that each check takes time and there are a lot of posts on the internet to read, so I have to limit how much I search.
With regards to point 2, it’s not that I spot-checked the most suspicious one, rather it’s that I spot-checked the first suspicious one. This is still a filter on suspiciousness, but a much weaker one. I think some filter on suspiciousness is appropriate since suspicious claims are also the ones I can learn the most from if they turn out to be true, as claims become suspicious through a combination of being unlikely and having big implications.
With regards to point 3, if you put much more effort into verifying the accuracy of your claims about co-ops than your claims about other stuff, then your accuracy on co-ops might not be that correlated with your accuracy on other stuff, and I ought to do a spot-check specifically on your claims about co-ops. I don’t know if that is true. If it is true, it might also be helpful to mention it as a disclaimer in the post so people know what claims to mostly focus on.
So it’s not so much that I’m dismissing the entire literature on co-ops. (Or well, I would generally dismiss any social science that I haven’t done some surface checks of. But that’s different from my comments here.) It’s more that I’m dismissing your literature review of the literature.