How would you feel about a review process that had two sections?
Section One: How important do you find this work & to what extent do you think the research is worth doing? (Ex: Does it strike at what you see as core alignment problems?)
Section Two: What do you think of the details of the research? (Ex: Do you see any methodological flaws, do you have any ideas for further work, etc).
My impression is that academic peer-reviewers generally do both of these. Compared to academic peer-review, LW/AF discussions tend to have a lot of Section One and not much Section Two.
My (low-resilience) guess is that the field would benefit from more “Section Two Engagement” from people with different worldviews.
(On the other hand, perhaps people with different worldviews and research priorities won’t have a comparative advantage in offering specific, detailed critiques. Ex: An agent foundations researcher might not be very good at providing detailed critiques of an interpretability paper.)
(But to counter that point, maybe there are certain specific/detailed critiques that are easier for “outsiders” to catch.)
Disclaimer: I’m not particularly confident in any of my views here or in my original comment. I mostly commented on the post because the post implied that I supported the idea of a review process whereas my actual opinion is mixed (not uniformly negative). If I hadn’t been named explicitly I wouldn’t have said anything; I don’t want to stop people from trying this out if they think it would be good; it’s quite plausible that someone thinking about this full time would have a vision for it that would be good that I haven’t even considered yet (given how little I’ve thought about it).
I think how excited I’d be would depend a lot more on the details (e.g. who are the reviewers, how much time do they spend, how are they incentivized, what happens after the reviews are completed). But if we just imagine the LessWrong Review extended to the Alignment Forum, I’m not that excited, because I predict (not confidently) that the reviews just wouldn’t be good at engaging with the details. (Mostly because LW / AF comments don’t seem very good at engaging with details on existing LW / AF posts, and because typical LW / AF commenters don’t seem familiar enough with ML to judge details in ML papers.)
My impression is that academic peer-reviewers generally do both of these.
Academic peer review does do both in principle, but I’d say that typically most of the emphasis is on Section Two. Generally the Section One style review is just “yup, this is in fact trying to make progress on a problem academia has previously deemed important, and is not just regurgitating things that people previously said” (i.e. it is significant and novel).
(It is common for bad reviews to just say “this is not significant / not novel” and then ignore Section One entirely, but this is pretty commonly thought of as explicitly “this was a bad review”, unless they actually justified “not significant / not novel” well enough that most others would agree with them.)
How would you feel about a review process that had two sections?
Section One: How important do you find this work & to what extent do you think the research is worth doing? (Ex: Does it strike at what you see as core alignment problems?)
Section Two: What do you think of the details of the research? (Ex: Do you see any methodological flaws, do you have any ideas for further work, etc).
My impression is that academic peer-reviewers generally do both of these. Compared to academic peer-review, LW/AF discussions tend to have a lot of Section One and not much Section Two.
My (low-resilience) guess is that the field would benefit from more “Section Two Engagement” from people with different worldviews.
(On the other hand, perhaps people with different worldviews and research priorities won’t have a comparative advantage in offering specific, detailed critiques. Ex: An agent foundations researcher might not be very good at providing detailed critiques of an interpretability paper.)
(But to counter that point, maybe there are certain specific/detailed critiques that are easier for “outsiders” to catch.)
Disclaimer: I’m not particularly confident in any of my views here or in my original comment. I mostly commented on the post because the post implied that I supported the idea of a review process whereas my actual opinion is mixed (not uniformly negative). If I hadn’t been named explicitly I wouldn’t have said anything; I don’t want to stop people from trying this out if they think it would be good; it’s quite plausible that someone thinking about this full time would have a vision for it that would be good that I haven’t even considered yet (given how little I’ve thought about it).
I think how excited I’d be would depend a lot more on the details (e.g. who are the reviewers, how much time do they spend, how are they incentivized, what happens after the reviews are completed). But if we just imagine the LessWrong Review extended to the Alignment Forum, I’m not that excited, because I predict (not confidently) that the reviews just wouldn’t be good at engaging with the details. (Mostly because LW / AF comments don’t seem very good at engaging with details on existing LW / AF posts, and because typical LW / AF commenters don’t seem familiar enough with ML to judge details in ML papers.)
Academic peer review does do both in principle, but I’d say that typically most of the emphasis is on Section Two. Generally the Section One style review is just “yup, this is in fact trying to make progress on a problem academia has previously deemed important, and is not just regurgitating things that people previously said” (i.e. it is significant and novel).
(It is common for bad reviews to just say “this is not significant / not novel” and then ignore Section One entirely, but this is pretty commonly thought of as explicitly “this was a bad review”, unless they actually justified “not significant / not novel” well enough that most others would agree with them.)