I’m someone who both prefers and practises the ‘status quo’.
My impression is the key feature of this is limited (and author controlled) sharing. (There are other nifty features for things like gdocs—e.g. commenting ‘on a line’ - but this practice predates gdocs). The key benefits for ‘me as author’ are these:
1. I can target the best critics: I usually have a good idea of who is likely to help make my work better. If I broadcast, the mean quality of feedback almost certainly goes down.
2. I can leverage existing relationships: The implicit promise if I send out a draft to someone for feedback is I will engage with their criticism seriously (in contrast, there’s no obligation that I ‘should’ respond to every critical comment on a post I write). This both encourages them to do so, and may help further foster a collegial relationship going forward.
3. I can mess up privately: If what I write makes a critical (or embarrassing) mistake, or could be construed to say something objectionable, I’d prefer this be caught in private rather than my failing being on the public record for as long as there’s an internet archive (or someone inclined to take screen shots). (This community is no stranger to people—insiders or outsiders—publishing mordant criticisms of remarks made ‘off the cuff’ to infer serious faults in the speaker).
I also think the current status quo is a pretty good one from an ecosystem wide perspective too: I think there’s a useful division of labour between ‘early stage’ writings to be refined by a smaller network with lower stakes, and ‘final publications’ which the author implicitly offers an assurance (backed by their reputation) that the work is a valuable contribution to the epistemic commons.
For most work there is a ‘refining’ stage, which is better done by smaller pre-selected networks rather than of authors and critics mutually ‘shouting into the void’ (from the author’s side, there will likely be a fair amount of annoying/irrelevant/rubbish criticism; from a critic’s side, a fair risk your careful remarks will be ignored or brushed off).
Publication seems to be better for polished or refined work, as at this stage a) it hopefully it has fewer mistakes and so generally more valuable to the non-critical reader, b) if there is a key mistake/objection neglected (e.g. because the pre-selected network resulted in an echo chamber) disagreement between (‘steel-manned’) positions registered publicly and hashed seems a useful exercise. (I’m generally a fan of more ‘adversarial’ - or at least ‘adversarial-tolerant’ norms for public discussion for this reason.)
This isn’t perfect, although I don’t see the ‘comments going to waste’ issue as the greatest challenge (one can adapt one’s private comments to a public one to post, although I appreciate this is a costlier route than initially writing the public comment—ultimately, if one finds ones private feedback is repeatedly neglected, one can decline to provide it in the first place).
The biggest one I see is the risk of people who can benefit from a ‘selective high-quality feedback network’ (either contributing useful early stage criticism, having good early stage posts, or both) not being able to enter one. Yet so long as members of existing ones still ‘keep an eye out’ for posts and comments from ‘outsiders’, this does provide a means for such people to build up a reputation to be included in future (i.e. if Alice sees Bob make good remarks etc., she’s more interested in ‘running a draft by him’ next time, or to respond positively if Bob asks her to look something over).
if Alice sees Bob make good remarks etc., she’s more interested in ‘running a draft by him’ next time, or to respond positively if Bob asks her to look something over
This dynamic contributes to anxiety for me to comment in Google Docs, and makes it less fun than public commenting (apparently the opposite of many other people). I feel like if I fail to make a good contribution, or worse, make a dumb comment, I won’t be invited to future drafts and will end up missing a lot of good arguments, or entire articles because many drafts don’t get published until much later or ever. This is not a theoretical fear because I’ve “gotten in” to a couple of important and still unpublished drafts only by accidentally finding out about them and requesting invites.
Another thing that contributes to the anxiety is feeling a need to make a good first impression to people who are in the discussion who I’ve never talked to before because they don’t participate on public forums.
FYI, this was a significant update for me. I just wanted to note that this is a bigger part of my model now as opposed to an edge case tacked on.
I hadn’t actually been invited much to google docs where this dynamic would come up, but it makes sense that this experience would be common. (I’ve only actually talked to one other person who shared your experience, so still up for updating backwards again, but I don’t expect to)
Which should either cause me to downgrade the importance of the “feeling safer in a private space”, or have it apply a bit differently than I was expecting it to have applied. (I think it still applies to the author of the original paper, maybe less so to commenters. Although I do think think having a sense that your fellow commenters are being filtered for some kind of “on-the-same-page-ness” still improves the conversation in other ways)
Ah, yeah I think I’d feel the same if I were invited to docs that were either beyond my current reputation/social standing, or just, with a new group of people I don’t know as well. Especially if I considered it high stakes to keep getting into future similar discussions.
I think it’s a dramatically different experience when I invite or get invited to docs where I already know the people well, and already feel confident that I know how to contribute.
Another thing that contributes to the anxiety is feeling a need to make a good first impression to people who are in the discussion who I’ve never talked to before because they don’t participate on public forums.
It would be neat if Google Docs had settings for their comments to only allow the doc’s authors do see your comments, or to make yourself anonymous from other commenters on the doc. I think these kinds of features could help reduce this kind of anxiety when reviewing people’s docs.
I don’t see ‘comments going to waste’ issue as the greatest challenge
I think this underestimates the challenge. Empirically, people don’t crosspost those comments. Periodically saying “hey it’d be good if you crossposted those private comments” won’t change the underlying incentive structure.
(Similarly, the fact that one ‘could’ keep an eye out for posts and comments from outsiders won’t change the fact that people generally don’t)
I’m someone who both prefers and practises the ‘status quo’.
My impression is the key feature of this is limited (and author controlled) sharing. (There are other nifty features for things like gdocs—e.g. commenting ‘on a line’ - but this practice predates gdocs). The key benefits for ‘me as author’ are these:
1. I can target the best critics: I usually have a good idea of who is likely to help make my work better. If I broadcast, the mean quality of feedback almost certainly goes down.
2. I can leverage existing relationships: The implicit promise if I send out a draft to someone for feedback is I will engage with their criticism seriously (in contrast, there’s no obligation that I ‘should’ respond to every critical comment on a post I write). This both encourages them to do so, and may help further foster a collegial relationship going forward.
3. I can mess up privately: If what I write makes a critical (or embarrassing) mistake, or could be construed to say something objectionable, I’d prefer this be caught in private rather than my failing being on the public record for as long as there’s an internet archive (or someone inclined to take screen shots). (This community is no stranger to people—insiders or outsiders—publishing mordant criticisms of remarks made ‘off the cuff’ to infer serious faults in the speaker).
I also think the current status quo is a pretty good one from an ecosystem wide perspective too: I think there’s a useful division of labour between ‘early stage’ writings to be refined by a smaller network with lower stakes, and ‘final publications’ which the author implicitly offers an assurance (backed by their reputation) that the work is a valuable contribution to the epistemic commons.
For most work there is a ‘refining’ stage, which is better done by smaller pre-selected networks rather than of authors and critics mutually ‘shouting into the void’ (from the author’s side, there will likely be a fair amount of annoying/irrelevant/rubbish criticism; from a critic’s side, a fair risk your careful remarks will be ignored or brushed off).
Publication seems to be better for polished or refined work, as at this stage a) it hopefully it has fewer mistakes and so generally more valuable to the non-critical reader, b) if there is a key mistake/objection neglected (e.g. because the pre-selected network resulted in an echo chamber) disagreement between (‘steel-manned’) positions registered publicly and hashed seems a useful exercise. (I’m generally a fan of more ‘adversarial’ - or at least ‘adversarial-tolerant’ norms for public discussion for this reason.)
This isn’t perfect, although I don’t see the ‘comments going to waste’ issue as the greatest challenge (one can adapt one’s private comments to a public one to post, although I appreciate this is a costlier route than initially writing the public comment—ultimately, if one finds ones private feedback is repeatedly neglected, one can decline to provide it in the first place).
The biggest one I see is the risk of people who can benefit from a ‘selective high-quality feedback network’ (either contributing useful early stage criticism, having good early stage posts, or both) not being able to enter one. Yet so long as members of existing ones still ‘keep an eye out’ for posts and comments from ‘outsiders’, this does provide a means for such people to build up a reputation to be included in future (i.e. if Alice sees Bob make good remarks etc., she’s more interested in ‘running a draft by him’ next time, or to respond positively if Bob asks her to look something over).
This dynamic contributes to anxiety for me to comment in Google Docs, and makes it less fun than public commenting (apparently the opposite of many other people). I feel like if I fail to make a good contribution, or worse, make a dumb comment, I won’t be invited to future drafts and will end up missing a lot of good arguments, or entire articles because many drafts don’t get published until much later or ever. This is not a theoretical fear because I’ve “gotten in” to a couple of important and still unpublished drafts only by accidentally finding out about them and requesting invites.
Another thing that contributes to the anxiety is feeling a need to make a good first impression to people who are in the discussion who I’ve never talked to before because they don’t participate on public forums.
FYI, this was a significant update for me. I just wanted to note that this is a bigger part of my model now as opposed to an edge case tacked on.
I hadn’t actually been invited much to google docs where this dynamic would come up, but it makes sense that this experience would be common. (I’ve only actually talked to one other person who shared your experience, so still up for updating backwards again, but I don’t expect to)
Which should either cause me to downgrade the importance of the “feeling safer in a private space”, or have it apply a bit differently than I was expecting it to have applied. (I think it still applies to the author of the original paper, maybe less so to commenters. Although I do think think having a sense that your fellow commenters are being filtered for some kind of “on-the-same-page-ness” still improves the conversation in other ways)
FYI I’ve had this experience as well, though it’s not particularly strong or common.
Ah, yeah I think I’d feel the same if I were invited to docs that were either beyond my current reputation/social standing, or just, with a new group of people I don’t know as well. Especially if I considered it high stakes to keep getting into future similar discussions.
I think it’s a dramatically different experience when I invite or get invited to docs where I already know the people well, and already feel confident that I know how to contribute.
It would be neat if Google Docs had settings for their comments to only allow the doc’s authors do see your comments, or to make yourself anonymous from other commenters on the doc. I think these kinds of features could help reduce this kind of anxiety when reviewing people’s docs.
I think this underestimates the challenge. Empirically, people don’t crosspost those comments. Periodically saying “hey it’d be good if you crossposted those private comments” won’t change the underlying incentive structure.
(Similarly, the fact that one ‘could’ keep an eye out for posts and comments from outsiders won’t change the fact that people generally don’t)