This, however, assumes that “formative evaluations” must be complete works by single contributors, rather than collaborative efforts contributed to by multiple commenters. That is an unrealistic and unproductive assumption, and will lead to less evaluative work being done overall, not more.
I am curious as to your assessment of the degree of work done by a naked “this seems unclear, please explain”?
My own assessment would place the value of this (and nothing else) at fairly close to zero—unless, of course, you are implicitly taking credit for some of the discussion that follows (with the reasoning that, had the initiating comment been absent, the resulting discussion would not counterfactually exist). If so, I find this reasoning unconvincing, but I remain open to hearing reasons you might disagree with me about this—if in fact you do disagree. (And if you don’t disagree, then from my perspective that sounds awfully like conceding the point; but perhaps you disagree with that, and if so, I would also like to hear why.)
I am curious as to your assessment of the degree of work done by a naked “this seems unclear, please explain”?
By “degree of work” do you mean “amount of effort invested” or “magnitude of effect achieved”?
If the former, then the answer, of course, is “that is irrelevant”. But it seems like you mean the latter—yes? In which case, the answer, empirically, is “often substantial”.
… unless, of course, you are implicitly taking credit for some of the discussion that follows (with the reasoning that, had the initiating comment been absent, the resulting discussion would not counterfactually exist)
Essentially, yes. And we do not need to imagine counterfactuals, either; we can see this happen, often enough (i.e., some post will be written, and nobody asks for examples, and none are given, and no discussion of particulars ensues). Individual cases differ in details, of course, but the pattern is clear.
Although I wouldn’t phrase it quite in terms of “taking credit” for the ensuing discussion. That’s not the point. The point is that the effect be achieved, and that actions which lead to the effect being achieved, be encouraged. If I write a comment like this one, and someone (as an aside, note, that in this case it was not the OP!) responds with comments like this one and this one, then of course it would be silly of me to say “I deserve the credit for those replies!”—no, the author of those replies deserves the credit for those replies. But insofar as they wouldn’t’ve have existed if I hadn’t posted my comment, then I deserve credit for having posted my comment. You are welcome to say “but you deserve less credit, maybe even almost no credit”; that’s fine. (Although, as I’ve noted before, the degree to which such prompts are appreciated and rewarded ought to scale with the likelihood of their counterfactual absence, i.e., if I hadn’t written that comment, would someone else have? But that’s a secondary point.) It’s fine if you want to assign me only epsilon credit.
What’s not fine is if, instead, you debit me for that comment. That would be completely backwards, and fundamentally confused about what sorts of contributions are valuable, and indeed about what the point of this website even is.
I mostly don’t agree that “the pattern is clear”—which is to say, I do take issue with saying “we do not need to imagine counterfactuals”. Here is (to my mind) a salient example of a top-level comment which provides an example illustrating the point of the OP, without the need for prompting.
I think this is mostly what happens, in the absence of such prompting: if someone thinks of a useful example, they can provide it in the comments (and accrue social credit/karma for their contribution, if indeed other users found said contribution useful). Conversely, if no examples come to mind, then a mere request from some other user (“Examples?”) generally will not cause sudden examples to spring into mind (and to the extent that it does, the examples in question are likely to be ad hoc, generated in a somewhat defensive frame of mind, and accordingly less useful).
And, of course, the crucial observation here is that in neither case was the request for examples useful; in the former case, the request was unnecessary, as the examples would have been provided in any case, and in the latter case, the request was useless, as it failed to elicit anything of value.
Here, I anticipate a two-pronged objection from you—one prong for each branch I have described. The first prong I anticipate is that, empirically, we do observe people providing examples when asked, and not otherwise. My response to this is that (again) this does not serve as evidence for your thesis, since we cannot observe the counterfactual worlds in which this request was/wasn’t made, respectively. (I also observe that we have some evidence to the contrary, in our actual world, wherein sometimes an exhortation to provide examples is simply ignored; moreover, this occurs more often in cases where the asker appears to have put in little effort to generate examples of their own before asking.)
The second prong is that, in the case where no useful examples are elicited, this fact in itself conveys information—specifically, it conveys that the post’s thesis is (apparently) difficult to substantiate, which should cause us to question its very substance. I am more sympathetic to this objection than I am to the previous—but still not very sympathetic, as there are quite often other reasons, unrelated to the defensibility of one’s thesis, one might not wish to invest effort in producing such a response. In fact, I read Duncan’s complaint as concerned with just this effect: not that being asked to provide examples is bad, but that the accompanying (implicit) interpretation wherein a failure to respond is interpreted as lack of ability to defend one’s thesis creates an asymmetric (and undue) burden on him, the author.
That last bit in bold is, in my mind, the operative point here. Without that, even accepting everything else I said as valid and correct, you would still be able to respond, after all, that
What’s not fine is if, instead, you debit me for that comment. That would be completely backwards, and fundamentally confused about what sorts of contributions are valuable, and indeed about what the point of this website even is.
After all, even if such a comment is not particularly valuable in and of itself, it is not a net negative for discussion—and at least (arguably) sometimes positive. But with the inclusion of the bolded point, the cost-benefit analysis changes: asking for examples (without accompanying interpretive effort, much of whose use is in signaling to the author that you, the commenter, are interested in reducing the cost to them of responding) is, in this culture, not merely a “formative evaluation” or even a start to such, but a challenge to them to respond—and a timed challenge, at that. And it is not hard at all for me to see why we ought to increase the cost (“debit”, as you put it) for writing minimally useful comments that (often get construed as) issuing unilateral challenges to others!
I mostly don’t agree that “the pattern is clear”—which is to say, I do take issue with saying “we do not need to imagine counterfactuals”. Here is (to my mind) a salient example of a top-level comment which provides an example illustrating the point of the OP, without the need for prompting.
Yep, indeed, that is an example, and a good one.
Conversely, if no examples come to mind, then a mere request from some other user (“Examples?”) generally will not cause sudden examples to spring into mind (and to the extent that it does, the examples in question are likely to be ad hoc, generated in a somewhat defensive frame of mind, and accordingly less useful).
But I linked a case of exactly the thing you just said won’t happen! I linked it in the comment you just responded to!
The first prong I anticipate is that, empirically, we do observe people providing examples when asked, and not otherwise. My response to this is that (again) this does not serve as evidence for your thesis, since we cannot observe the counterfactual worlds in which this request was/wasn’t made, respectively.
This is a weak response given that I am pointing to a pattern.
I am more sympathetic to this objection than I am to the previous—but still not very sympathetic, as there are quite often other reasons, unrelated to the defensibility of one’s thesis, one might not wish to invest effort in producing such a response.
A very suspicious reply, in the general case. Not always false, of course! But suspicious. If such a condition obtains, it ought to be pointed out explicitly, and defended. It is quite improper, and lacking in intellectual integrity, to simply rely on social censure against requests for examples to shield you from having to explain why in this case it so happens that you don’t need to point to any extensions for your proffered intensions.
In fact, I read Duncan’s complaint as concerned with just this effect: not that being asked to provide examples is bad, but that the accompanying (implicit) interpretation wherein a failure to respond is interpreted as lack of ability to defend one’s thesis creates an asymmetric (and undue) burden on him, the author.
I agree that Duncan’s complaint includes this. I just think that he’s wrong about this. (And wrong in such a way that he should know that he’s wrong.) The burden is (a) not just on the author, but also on the reader (including the one who requested the examples!), and (b) not undue, but in fact quite the opposite.
But with the inclusion of the bolded point, the cost-benefit analysis changes: asking for examples (without accompanying interpretive effort, much of whose use is in signaling to the author that you, the commenter, are interested in reducing the cost to them of responding) is, in this culture, not merely a “formative evaluation” or even a start to such, but a challenge to them to respond—and a timed challenge, at that. And it is not hard at all for me to see why we ought to increase the cost (“debit”, as you put it) for writing minimally useful comments that (often get construed as) issuing unilateral challenges to others!
First, on the subject of “accompanying interpretive effort”: I think that such effort not only doesn’t reduce the cost to authors of responding, it can easily increase the cost. (See my previous commentary on the subject of “interpretive effort” for much expansion of this point.)
Second, on the subject of “cost to the author of responding”: that cost should not be very high, since the author should, ideally, already have examples in mind.
(As an aside, I wonder at the fact that you, and others here, seem so consistently to ignore this point: if an author makes a strong claim, and has no examples ready, and can’t easily come up with such, and also has no good case for why examples are inapplicable / unhelpful / irrelevant / whatever, that is a bad sign. There is a good chance that the author should not have written the post at all, in such a case!)
Third, on the subject of “challenge to the author”: see above re: “cost to the author”, but also note that the “challenge”, such as it is (I’d call it a “question” or a “prompt”; as I say elsewhere, it’s not adversarial by default!) can be met by others, as well.
I am curious as to your assessment of the degree of work done by a naked “this seems unclear, please explain”?
My own assessment would place the value of this (and nothing else) at fairly close to zero—unless, of course, you are implicitly taking credit for some of the discussion that follows (with the reasoning that, had the initiating comment been absent, the resulting discussion would not counterfactually exist). If so, I find this reasoning unconvincing, but I remain open to hearing reasons you might disagree with me about this—if in fact you do disagree. (And if you don’t disagree, then from my perspective that sounds awfully like conceding the point; but perhaps you disagree with that, and if so, I would also like to hear why.)
By “degree of work” do you mean “amount of effort invested” or “magnitude of effect achieved”?
If the former, then the answer, of course, is “that is irrelevant”. But it seems like you mean the latter—yes? In which case, the answer, empirically, is “often substantial”.
Essentially, yes. And we do not need to imagine counterfactuals, either; we can see this happen, often enough (i.e., some post will be written, and nobody asks for examples, and none are given, and no discussion of particulars ensues). Individual cases differ in details, of course, but the pattern is clear.
Although I wouldn’t phrase it quite in terms of “taking credit” for the ensuing discussion. That’s not the point. The point is that the effect be achieved, and that actions which lead to the effect being achieved, be encouraged. If I write a comment like this one, and someone (as an aside, note, that in this case it was not the OP!) responds with comments like this one and this one, then of course it would be silly of me to say “I deserve the credit for those replies!”—no, the author of those replies deserves the credit for those replies. But insofar as they wouldn’t’ve have existed if I hadn’t posted my comment, then I deserve credit for having posted my comment. You are welcome to say “but you deserve less credit, maybe even almost no credit”; that’s fine. (Although, as I’ve noted before, the degree to which such prompts are appreciated and rewarded ought to scale with the likelihood of their counterfactual absence, i.e., if I hadn’t written that comment, would someone else have? But that’s a secondary point.) It’s fine if you want to assign me only epsilon credit.
What’s not fine is if, instead, you debit me for that comment. That would be completely backwards, and fundamentally confused about what sorts of contributions are valuable, and indeed about what the point of this website even is.
Why?
I mostly don’t agree that “the pattern is clear”—which is to say, I do take issue with saying “we do not need to imagine counterfactuals”. Here is (to my mind) a salient example of a top-level comment which provides an example illustrating the point of the OP, without the need for prompting.
I think this is mostly what happens, in the absence of such prompting: if someone thinks of a useful example, they can provide it in the comments (and accrue social credit/karma for their contribution, if indeed other users found said contribution useful). Conversely, if no examples come to mind, then a mere request from some other user (“Examples?”) generally will not cause sudden examples to spring into mind (and to the extent that it does, the examples in question are likely to be ad hoc, generated in a somewhat defensive frame of mind, and accordingly less useful).
And, of course, the crucial observation here is that in neither case was the request for examples useful; in the former case, the request was unnecessary, as the examples would have been provided in any case, and in the latter case, the request was useless, as it failed to elicit anything of value.
Here, I anticipate a two-pronged objection from you—one prong for each branch I have described. The first prong I anticipate is that, empirically, we do observe people providing examples when asked, and not otherwise. My response to this is that (again) this does not serve as evidence for your thesis, since we cannot observe the counterfactual worlds in which this request was/wasn’t made, respectively. (I also observe that we have some evidence to the contrary, in our actual world, wherein sometimes an exhortation to provide examples is simply ignored; moreover, this occurs more often in cases where the asker appears to have put in little effort to generate examples of their own before asking.)
The second prong is that, in the case where no useful examples are elicited, this fact in itself conveys information—specifically, it conveys that the post’s thesis is (apparently) difficult to substantiate, which should cause us to question its very substance. I am more sympathetic to this objection than I am to the previous—but still not very sympathetic, as there are quite often other reasons, unrelated to the defensibility of one’s thesis, one might not wish to invest effort in producing such a response. In fact, I read Duncan’s complaint as concerned with just this effect: not that being asked to provide examples is bad, but that the accompanying (implicit) interpretation wherein a failure to respond is interpreted as lack of ability to defend one’s thesis creates an asymmetric (and undue) burden on him, the author.
That last bit in bold is, in my mind, the operative point here. Without that, even accepting everything else I said as valid and correct, you would still be able to respond, after all, that
After all, even if such a comment is not particularly valuable in and of itself, it is not a net negative for discussion—and at least (arguably) sometimes positive. But with the inclusion of the bolded point, the cost-benefit analysis changes: asking for examples (without accompanying interpretive effort, much of whose use is in signaling to the author that you, the commenter, are interested in reducing the cost to them of responding) is, in this culture, not merely a “formative evaluation” or even a start to such, but a challenge to them to respond—and a timed challenge, at that. And it is not hard at all for me to see why we ought to increase the cost (“debit”, as you put it) for writing minimally useful comments that (often get construed as) issuing unilateral challenges to others!
Yep, indeed, that is an example, and a good one.
But I linked a case of exactly the thing you just said won’t happen! I linked it in the comment you just responded to!
Here is another example.
Here are more examples: one two three (and a bonus particularly interesting sort-of-example)
This is a weak response given that I am pointing to a pattern.
A very suspicious reply, in the general case. Not always false, of course! But suspicious. If such a condition obtains, it ought to be pointed out explicitly, and defended. It is quite improper, and lacking in intellectual integrity, to simply rely on social censure against requests for examples to shield you from having to explain why in this case it so happens that you don’t need to point to any extensions for your proffered intensions.
I agree that Duncan’s complaint includes this. I just think that he’s wrong about this. (And wrong in such a way that he should know that he’s wrong.) The burden is (a) not just on the author, but also on the reader (including the one who requested the examples!), and (b) not undue, but in fact quite the opposite.
First, on the subject of “accompanying interpretive effort”: I think that such effort not only doesn’t reduce the cost to authors of responding, it can easily increase the cost. (See my previous commentary on the subject of “interpretive effort” for much expansion of this point.)
Second, on the subject of “cost to the author of responding”: that cost should not be very high, since the author should, ideally, already have examples in mind.
(As an aside, I wonder at the fact that you, and others here, seem so consistently to ignore this point: if an author makes a strong claim, and has no examples ready, and can’t easily come up with such, and also has no good case for why examples are inapplicable / unhelpful / irrelevant / whatever, that is a bad sign. There is a good chance that the author should not have written the post at all, in such a case!)
Third, on the subject of “challenge to the author”: see above re: “cost to the author”, but also note that the “challenge”, such as it is (I’d call it a “question” or a “prompt”; as I say elsewhere, it’s not adversarial by default!) can be met by others, as well.