The goal of the endeveour is to move academic resources from publishing out-there-research to verifiable research. The best way to do that is to reduce the incentives for out-there-research and increase those for verification.
Sometimes it might make sense to cite every study on a given topic. The default should be to cite the first replication or a meta-analysis when available.
If I’m a reader of an academic paper I profit from reading the paper that provides the evidence that the claim is actually true.
There just no good reason to give out prestige to the first person who finds an effect.
In short term, we might want less research and more verification. But in long term, we don’t want to discourage the research completely.
Also, people will always try to game the system. How will you get points for the first verification of the original idea, when the original ideas become scarce? Split the work with your friend: one of you will write the original article, another will write the verification article; then publish both at the same time. Would that be a big improvement? I would prefer the verification to be done by someone who is not a friend, and especially does not owe a favor for getting such opportunity.
Note: In ice hockey, players get points for goals and assists. Sure, it’s not completely the same situation, but it is a way to encourage two behaviors that are both needed for a success. In science we need to encourage both research and verification.
Yes, two people getting a P<0.05 and both setting up the experiment are better than one.
How will you get points for the first verification of the original idea, when the original ideas become scarce?
I don’t believe that original ideas become scarce. Researching original ideas is fun. There will be still grant giving agencies that give out grands to persue original ideas.
Yes, two people getting a P<0.05 and both setting up the experiment are better than one.
How many false-positives get published as opposed to negatives (or rarely even false-negatives)? If the ratio is too high then you’ll need more than just two positive studies. If, as claimed in the quoted article, 70% of papers in a field are not reproducible that implies finding papers at random would require about nine positive studies to reach a true P<0.05, and that’s only if each paper is statistically independent from the others.
If there is financial incentive to reproduce existing studies when there is a ready-made template to copy and paste into a grant-funding paper I think the overall quality of published research could decline. At least in the current model there’s a financial incentive to invent novel ideas and then test them, versus just publishing a false reproduction.
The goal of the endeveour is to move academic resources from publishing out-there-research to verifiable research. The best way to do that is to reduce the incentives for out-there-research and increase those for verification.
Sometimes it might make sense to cite every study on a given topic. The default should be to cite the first replication or a meta-analysis when available. If I’m a reader of an academic paper I profit from reading the paper that provides the evidence that the claim is actually true.
There just no good reason to give out prestige to the first person who finds an effect.
In short term, we might want less research and more verification. But in long term, we don’t want to discourage the research completely.
Also, people will always try to game the system. How will you get points for the first verification of the original idea, when the original ideas become scarce? Split the work with your friend: one of you will write the original article, another will write the verification article; then publish both at the same time. Would that be a big improvement? I would prefer the verification to be done by someone who is not a friend, and especially does not owe a favor for getting such opportunity.
Note: In ice hockey, players get points for goals and assists. Sure, it’s not completely the same situation, but it is a way to encourage two behaviors that are both needed for a success. In science we need to encourage both research and verification.
Yes, two people getting a P<0.05 and both setting up the experiment are better than one.
I don’t believe that original ideas become scarce. Researching original ideas is fun. There will be still grant giving agencies that give out grands to persue original ideas.
How many false-positives get published as opposed to negatives (or rarely even false-negatives)? If the ratio is too high then you’ll need more than just two positive studies. If, as claimed in the quoted article, 70% of papers in a field are not reproducible that implies finding papers at random would require about nine positive studies to reach a true P<0.05, and that’s only if each paper is statistically independent from the others.
If there is financial incentive to reproduce existing studies when there is a ready-made template to copy and paste into a grant-funding paper I think the overall quality of published research could decline. At least in the current model there’s a financial incentive to invent novel ideas and then test them, versus just publishing a false reproduction.