I’d be curious to learn the alternative ways you favor, or more detail on why this approach is flawed. Standard academic peer review has its issues, but seemingly a community should have a way it reviews material and determines what’s great, what needs work, and what is plain wrong.
Well part of rationality is being able to assess and integrate this information yourself, rather than trusting in the accuracy of curators (which reinforces bad habits IMHO, hence the concern). Things that are useful get referenced, build citations, and are therefore more visible and likely to be found.
Sorry if I wasn’t clear: I don’t think it’s a useful thing to do, full stop.
I don’t mean to rain on anyone’s parade. I was really just replying to the top-level comment which started with:
The shortage of reviews is both puzzling and concerning...
I was just pointing out that some people aren’t participating because they don’t find the project worth doing in the first place. To me it’s just noise. I’m not going to get in the way of anyone else, if they want to contribute, but if you’re wondering why there is a shortage of reviews.. well I gave my reasons for not contributing.
That makes sense. As I’m won’t to say, there often risks/benefits/costs in each direction.
Ways in which I think communal and collaborative review are imperative:
Public reviews help establish the standards or reasoning expected in the community.
By reading other people’s evaluations, you can better learn how to perform your own.
It’s completely time prohibitive for me to thoroughly review every post that I might reference, instead I trust in the author. Dangerously, many people might do this and a post becomes highly cited despite flaws that would be exposed if a person or two spent several hours evaluating it*
I might be competent to understand and reference a paper, but lack the domain expertise to review it myself. The review of another domain expert can help me understanding the shortcoming’s of a post.
And as I think has been posted about, having a coordinated “review festival” is ideally an opportunity for people with different opinions about controversial topics to get together and hash it out. In an ideal world, review is the time when the community gets together to resolve what debates it can.
*An example is the work I began auditing the paper Eternity in Six Hours which is tied to the Astronomical Waste argument. Many people reference that argument, but as far as I know, few people have spent much time attempting to systematically evaluate its claims. (I do hope to finish that work and publish more on it sometime.)
I’d be curious to learn the alternative ways you favor, or more detail on why this approach is flawed. Standard academic peer review has its issues, but seemingly a community should have a way it reviews material and determines what’s great, what needs work, and what is plain wrong.
Well part of rationality is being able to assess and integrate this information yourself, rather than trusting in the accuracy of curators (which reinforces bad habits IMHO, hence the concern). Things that are useful get referenced, build citations, and are therefore more visible and likely to be found.
Do you think there are any ways the 2018 Review as we’ve been doing it could be modified to be better along the dimensions you’re concerned about?
Sorry if I wasn’t clear: I don’t think it’s a useful thing to do, full stop.
I don’t mean to rain on anyone’s parade. I was really just replying to the top-level comment which started with:
I was just pointing out that some people aren’t participating because they don’t find the project worth doing in the first place. To me it’s just noise. I’m not going to get in the way of anyone else, if they want to contribute, but if you’re wondering why there is a shortage of reviews.. well I gave my reasons for not contributing.
Yeah, true, that seems like a fair reason to point out for why there wouldn’t be more reviews. Thanks for sharing your personal reasons.
That makes sense. As I’m won’t to say, there often risks/benefits/costs in each direction.
Ways in which I think communal and collaborative review are imperative:
Public reviews help establish the standards or reasoning expected in the community.
By reading other people’s evaluations, you can better learn how to perform your own.
It’s completely time prohibitive for me to thoroughly review every post that I might reference, instead I trust in the author. Dangerously, many people might do this and a post becomes highly cited despite flaws that would be exposed if a person or two spent several hours evaluating it*
I might be competent to understand and reference a paper, but lack the domain expertise to review it myself. The review of another domain expert can help me understanding the shortcoming’s of a post.
And as I think has been posted about, having a coordinated “review festival” is ideally an opportunity for people with different opinions about controversial topics to get together and hash it out. In an ideal world, review is the time when the community gets together to resolve what debates it can.
*An example is the work I began auditing the paper Eternity in Six Hours which is tied to the Astronomical Waste argument. Many people reference that argument, but as far as I know, few people have spent much time attempting to systematically evaluate its claims. (I do hope to finish that work and publish more on it sometime.)