Nice article. Much of psychology suffers from the failure to replicate experiments, for various reasons like funding, time pressure, and difficulties in obtaining the population required. I’ve worked in sensorimotor control for several years and recently some researchers have come up with the idea of putting together a database of studies on perturbations during reaching (which is a very widely used paradigm) because they can so often be divergent due to tiny changes in the experiments.
I’d love to see more of this kind of thing in psychology in general, just as I’d like to see registration of medical trials from pharmaceutical companies (with both negative and positive results published) to avoid the all-too pervasive publication bias.
When I heard a lecture on this subject (there is pretty damning statistical evidence that drug trials are always slanted towards the company paying for the trials) the only viable proposal I heard discussed was to have the testing completely performed and controlled by an unbiased third party. (probably the government)
It’s not necessary to have them completely performed and controlled by a third party—but the idea is if you want to do a drug trial, you sign up with an independent register saying which drug you’re testing and what your methodology is. Then when the trial is done, you must report your results publicly.
That stops companies hiding negative trials and only publishing positive ones. It doesn’t stop the data being manipulated, but that’s another problem.
Nice article. Much of psychology suffers from the failure to replicate experiments, for various reasons like funding, time pressure, and difficulties in obtaining the population required. I’ve worked in sensorimotor control for several years and recently some researchers have come up with the idea of putting together a database of studies on perturbations during reaching (which is a very widely used paradigm) because they can so often be divergent due to tiny changes in the experiments.
I’d love to see more of this kind of thing in psychology in general, just as I’d like to see registration of medical trials from pharmaceutical companies (with both negative and positive results published) to avoid the all-too pervasive publication bias.
How would registry of the trials work?
When I heard a lecture on this subject (there is pretty damning statistical evidence that drug trials are always slanted towards the company paying for the trials) the only viable proposal I heard discussed was to have the testing completely performed and controlled by an unbiased third party. (probably the government)
It’s not necessary to have them completely performed and controlled by a third party—but the idea is if you want to do a drug trial, you sign up with an independent register saying which drug you’re testing and what your methodology is. Then when the trial is done, you must report your results publicly.
That stops companies hiding negative trials and only publishing positive ones. It doesn’t stop the data being manipulated, but that’s another problem.