It’s far easier for me to figure out how much to update on evidence when someone else has looked at the details and highlighted ways in which the evidence is stronger or weaker than a reader might naively take away from the paper. (At least, assuming the reviewer did a good job.)
This doesn’t apply to big-picture reviews because such reviews are typically a rehash of old arguments I already know.
This is similar to the general idea in AI safety via debate—when you have access to a review you are more like a judge; without a review you are more like the debate opponent.
Having someone else explain the paper from their perspective can surface other ways of thinking about the paper that can help with understanding it.
This sometimes does happen with big-picture reviews, though I think it’s less common.
Tbc, I’m not necessarily saying it is worth the opportunity cost of the reviewer’s time; I haven’t thought much about it.
A couple of reasons:
It’s far easier for me to figure out how much to update on evidence when someone else has looked at the details and highlighted ways in which the evidence is stronger or weaker than a reader might naively take away from the paper. (At least, assuming the reviewer did a good job.)
This doesn’t apply to big-picture reviews because such reviews are typically a rehash of old arguments I already know.
This is similar to the general idea in AI safety via debate—when you have access to a review you are more like a judge; without a review you are more like the debate opponent.
Having someone else explain the paper from their perspective can surface other ways of thinking about the paper that can help with understanding it.
This sometimes does happen with big-picture reviews, though I think it’s less common.
Tbc, I’m not necessarily saying it is worth the opportunity cost of the reviewer’s time; I haven’t thought much about it.