If that’s true, why are replication rates so poor?
They may be the best way to determine what the researcher originally intended to measure, but they’re not the only way to know if that was the only thing a researcher measured. I’ve found asking questions often works.
You can ask questions but how do you know whether the answers you are getting are right? It’s quite easy for people who fit a linear model to play a bit around with the parameters and not even remember all parameters they tested.
Then two or more other people with PhDs who weren’t involved in the research (editor and peer reviewer(s)) have to review what the researcher did
More often they don’t review what the researcher did but what the researchers claimed they did.
If that’s true, why are replication rates so poor?
There is no feedback post publication. Researchers are expected to individually decide on the quality of a published study, or occasionally ask the colleagues in their department.
I don’t get the impression that low replication rates are due to malice generally. I think it’s a training and incentive problem most of the time. In that case just asking should often work.
Science has very little feedback and lots of filtering at present. Preregistration is just more filtering. Science needs more feedback.
If that’s true, why are replication rates so poor?
You can ask questions but how do you know whether the answers you are getting are right? It’s quite easy for people who fit a linear model to play a bit around with the parameters and not even remember all parameters they tested.
More often they don’t review what the researcher did but what the researchers claimed they did.
There is no feedback post publication. Researchers are expected to individually decide on the quality of a published study, or occasionally ask the colleagues in their department.
I don’t get the impression that low replication rates are due to malice generally. I think it’s a training and incentive problem most of the time. In that case just asking should often work.
Science has very little feedback and lots of filtering at present. Preregistration is just more filtering. Science needs more feedback.
What kind of feedback would you want to exist?