Good question. I’d say: writing a paper proving your peers wrong is great fun, but requires a paper. You are expected to make a strong, detailed case, even when the work is pretty obviously flawed. You can’t just ignore a bad model in a background section or have a one-sentence “X found Y, but they’re blatantly p-hacking”—those moves risk a reviewer complaining. And even after writing the prove-them-wrong paper, you still can’t just ignore the bad work in background sections of future papers without risking reviewers’ ire.
Comment on “paper x” to my mind is the usual vehicle for complaining about faulty methods and poor statistical analysis. Since journals that accept comments tend to give a right of reply, review can be pretty light.
I would agree though that commenting on flaws like this is not as satisfying (mostly) as proper paper where an alternative hypothesis is promoted and opponents flaws lightly commented on. It is still a lot of work to comment and not a lot of point unless driving new science other than ego-tripping.
However, my original point remains—I don’t think researchers are remotely shy about criticizing the work of their peers.
Good question. I’d say: writing a paper proving your peers wrong is great fun, but requires a paper. You are expected to make a strong, detailed case, even when the work is pretty obviously flawed. You can’t just ignore a bad model in a background section or have a one-sentence “X found Y, but they’re blatantly p-hacking”—those moves risk a reviewer complaining. And even after writing the prove-them-wrong paper, you still can’t just ignore the bad work in background sections of future papers without risking reviewers’ ire.
Does that fit your experience?
Comment on “paper x” to my mind is the usual vehicle for complaining about faulty methods and poor statistical analysis. Since journals that accept comments tend to give a right of reply, review can be pretty light.
I would agree though that commenting on flaws like this is not as satisfying (mostly) as proper paper where an alternative hypothesis is promoted and opponents flaws lightly commented on. It is still a lot of work to comment and not a lot of point unless driving new science other than ego-tripping.
However, my original point remains—I don’t think researchers are remotely shy about criticizing the work of their peers.