This problem seems to me to have the flavor of Moloch and/or inadequate equilibria. Your criticisms have two parts, the pre-edit part based on your personal experience, in which you state why the personal actions they recommend are actually not possible because of the inadequate equilibria (i.e. because of academic incentives), and the criticism of the author’s proposed non-personal actions, which you say is just based on intuition.
I think the author would be unsurprised that the personal actions are not reasonable. They have already said this problem requires government intervention, basically to resolve the incentive problem. But maybe at the margin you can take some of the actions that the author refers to in the personal actions. If a paper is on the cusp of “needing to be cited” but you think it won’t replicate, take that into account! Or if reviewing a paper, at least take into account the probability of replication in your decision.
I think you are maybe reading the author’s claim to “stop assuming good faith” too literally. In the subsequent sentence they are basically refining that to the idea that most people are acting in good faith, but are not competent enough for good faith to be a useful assumption, which seems reasonable to me.
If a paper is on the cusp of “needing to be cited” but you think it won’t replicate, take that into account! Or if reviewing a paper, at least take into account the probability of replication in your decision.
Why do you think people don’t already do this?
In general, if you want to make a recommendation on the margin, you have to talk about what the current margin is.
I think you are maybe reading the author’s claim to “stop assuming good faith” too literally. In the subsequent sentence they are basically refining that to the idea that most people are acting in good faith, but are not competent enough for good faith to be a useful assumption
Huh? The sentence I see is
I’m not saying every academic interaction should be hostile and adversarial, but the good guys are behaving like dodos right now and the predators are running wild.
“the predators are running wild” does not mean “most people are acting in good faith, but are not competent enough for good faith to be a useful assumption”.
They have to do it to some extent, otherwise replicability would be literally uncorrelated with publishability, which probably isn’t the case. But because of the outcomes, we can see that people aren’t doing it enough at the margin, so encouraging people to move as far in that direction as they can seems like a useful reminder.
There are two models here, one is that everyone is a homo economicus when citing papers, so no amount of persuasion is going to adjust people’s citations. They are already making the optimal tradeoff based on their utility function of their personal interests vs. society’s interests. The other is that people are subject to biases and blind spots, or just haven’t even really considered whether they have the OPTION of not citing something that is questionable, in which case reminding them is a useful affordance.
I’m trying to be charitable to the author here, to recover useful advice. They didn’t say things in the way I’m saying them. But they may have been pointing in a useful direction, and I’m trying to steelman that.
“the predators are running wild” does not mean “most people are acting in good faith, but are not competent enough for good faith to be a useful assumption”.
Even upon careful rereading of that sentence, I disagree. But to parse this out based on this little sentence is too pointless for me. Like I said, I’m trying to focus on finding useful substance, not nitpicking the author, or you!
This problem seems to me to have the flavor of Moloch and/or inadequate equilibria. Your criticisms have two parts, the pre-edit part based on your personal experience, in which you state why the personal actions they recommend are actually not possible because of the inadequate equilibria (i.e. because of academic incentives), and the criticism of the author’s proposed non-personal actions, which you say is just based on intuition.
I think the author would be unsurprised that the personal actions are not reasonable. They have already said this problem requires government intervention, basically to resolve the incentive problem. But maybe at the margin you can take some of the actions that the author refers to in the personal actions. If a paper is on the cusp of “needing to be cited” but you think it won’t replicate, take that into account! Or if reviewing a paper, at least take into account the probability of replication in your decision.
I think you are maybe reading the author’s claim to “stop assuming good faith” too literally. In the subsequent sentence they are basically refining that to the idea that most people are acting in good faith, but are not competent enough for good faith to be a useful assumption, which seems reasonable to me.
Why do you think people don’t already do this?
In general, if you want to make a recommendation on the margin, you have to talk about what the current margin is.
Huh? The sentence I see is
“the predators are running wild” does not mean “most people are acting in good faith, but are not competent enough for good faith to be a useful assumption”.
They have to do it to some extent, otherwise replicability would be literally uncorrelated with publishability, which probably isn’t the case. But because of the outcomes, we can see that people aren’t doing it enough at the margin, so encouraging people to move as far in that direction as they can seems like a useful reminder.
There are two models here, one is that everyone is a homo economicus when citing papers, so no amount of persuasion is going to adjust people’s citations. They are already making the optimal tradeoff based on their utility function of their personal interests vs. society’s interests. The other is that people are subject to biases and blind spots, or just haven’t even really considered whether they have the OPTION of not citing something that is questionable, in which case reminding them is a useful affordance.
I’m trying to be charitable to the author here, to recover useful advice. They didn’t say things in the way I’m saying them. But they may have been pointing in a useful direction, and I’m trying to steelman that.
Even upon careful rereading of that sentence, I disagree. But to parse this out based on this little sentence is too pointless for me. Like I said, I’m trying to focus on finding useful substance, not nitpicking the author, or you!