One particular way this issue could be ameliorated is by encouraging people to write up null results/negative results, and one part of your model here is that a null result doesn’t get reported and thus other people don’t hear about failure, while people do hear about success stories, meaning that there is a selection effect to work on successful programs, and no one hears about the failures to tackle the problem, which is bad for research culture, and negative results not being shown is a universal problem across fields.
Lack of publicly reporting null results was a subtle but huge problem in cognitive neuroscience. It took a while to figure out just how much effort was being wasted running studies that others had already tried and not reported because results were null.
Alignment doesn’t have the same journal gatekeeping system that filters out null results, but there’s probably a pretty strong tendency to report less on lack of progress than actual progress.
So post about it if you worked hard at something and got nowhere. This is valuable information when others choose their problems and approaches.
I do see people doing this; it would probably be valuable if we did it more.
One particular way this issue could be ameliorated is by encouraging people to write up null results/negative results, and one part of your model here is that a null result doesn’t get reported and thus other people don’t hear about failure, while people do hear about success stories, meaning that there is a selection effect to work on successful programs, and no one hears about the failures to tackle the problem, which is bad for research culture, and negative results not being shown is a universal problem across fields.
Definitely.
Lack of publicly reporting null results was a subtle but huge problem in cognitive neuroscience. It took a while to figure out just how much effort was being wasted running studies that others had already tried and not reported because results were null.
Alignment doesn’t have the same journal gatekeeping system that filters out null results, but there’s probably a pretty strong tendency to report less on lack of progress than actual progress.
So post about it if you worked hard at something and got nowhere. This is valuable information when others choose their problems and approaches.
I do see people doing this; it would probably be valuable if we did it more.