Bruce K. Britton,
Let’s start with simpler things, like having people make their data and calculations available. (Or to be really simple, journals with such rules should enforce them!) Without this, you can just hide the data-mining in poorly specified protocols, not to mention fraud.
Data-mining is not that bad because it has systematic effects that an outsider can predict and account for; at least you can hope that it will wash out in the meta-analyses. This reminds me of this Robin Hanson post on how to extract experiments from the medical literature you don’t trust.
Perry E. Metzger makes similar recommendations to BKB and RH replies that it’s not going to happen. Actually, the medical community is moving towards things like registering studies. I worry that actions taken with a definite sense of who is the bad guy (drug companies) may make us worse off than the status quo, though I don’t see any downsides to anything that is actually going forward.
Bruce K. Britton, Let’s start with simpler things, like having people make their data and calculations available. (Or to be really simple, journals with such rules should enforce them!) Without this, you can just hide the data-mining in poorly specified protocols, not to mention fraud.
Data-mining is not that bad because it has systematic effects that an outsider can predict and account for; at least you can hope that it will wash out in the meta-analyses. This reminds me of this Robin Hanson post on how to extract experiments from the medical literature you don’t trust.
Perry E. Metzger makes similar recommendations to BKB and RH replies that it’s not going to happen. Actually, the medical community is moving towards things like registering studies. I worry that actions taken with a definite sense of who is the bad guy (drug companies) may make us worse off than the status quo, though I don’t see any downsides to anything that is actually going forward.
Yvain, read that post.