Your criticism of steelmanning is apt. Of “interpretive effort” I have already spoken elsewhere. As for “addressing core points”, however…
The problem with this criterion is simply that an author and a reader may disagree on what the core points are.
Note that I am not simply talking about cases where a reader misunderstands what the author wrote, and mistakes something for an intended “core point” that is no such thing! Such cases are, in a sense, trivial, insofar as clearing up the misunderstanding results in the original piece standing unmodified (excepting, perhaps, any clarifying modifications that are aimed at preventing exactly such misunderstanding from reoccurring in other readers).
Rather, I’m talking about cases where the reader understands what the author is saying, but considers the author’s claimed “core point” to actually be of peripheral significance, and considers a different point (one which the author either did not mention at all, or devoted only scant attention to) to be central. In this case, no misunderstanding, per se, is occurring (unless you call a deep conceptual or empirical error a “misunderstanding”, which I do not—I reserve the latter term for miscommunication).
This sort of thing can come in many forms. I will describe just one of them here.
It sometimes happens that someone—call him Dave—will write a post describing some (purported) phenomenon P—some pattern, some dynamic, some causal structure, etc.—and, to illustrate P, will provide an example E.
Carol, a reader, then comes along and says: “Actually, Dave, I don’t think E is an example of P! Consider the following…” Dave perhaps defends his example, perhaps not, but agrees, in the end, that E is not really a good example of P after all. “But,” says Dave, “that was just an example! Quit nitpicking, Carol—address my core points!”
What is Carol to say to this? Does Dave accuse her fairly? Is it mere nitpicking to attack a mere example?
But if E is the only example of P that Dave had provided, then—E having now been disqualified from that role—Dave is left with no examples of P. And if Dave cannot provide any other (or, I should simply say, any) examples of P, then perhaps P is simply not real? I can hardly thinking of a more “core” point than the question of whether the thing we’re talking about is even a real thing!
As I say, the “attacking an example” form is just one of the ways in which the “disagreement about what the core points are” dynamic can manifest. But it alone has, in my view, been responsible for a great deal of epistemic trouble among rationalists. As I’ve said elsewhere, we—the types of people who frequent such sites as Less Wrong—are very good at inventing abstract patterns, “crystallizing concepts”, constructing systems of classification, and so on. We are so good at it, in fact, that this talent of ours often gets away from us. It is absolutely critical to keep it reined in. (I do not choose the expression lightly! Reins, after all, are used, not to prevent a horse from moving, but to make it move in the direction you want to go—instead of uselessly running wild.) And those reins are made of real-world examples, they are made of extensions (as vs. intensions), they are made of practice. Without it, deceiving ourselves that we’ve gained knowledge, when in fact we’re building sky castles of abstract nonsense, is all too easy.
Perhaps you have other examples of dynamics where what the ‘core points’ are is in dispute, but the Carol and Dave case seems like one where there’s just a sort of miscommunication: Dave thinks ‘whether E is an example of P’ is not a core point, Carol thinks (I presume) ‘whether there exist any examples of P’ is a core point, and it seems likely to me that both of these can be true and agreed upon by both parties. I’d imagine that if Carol’s initial comment were ‘I don’t think E is an example of P, because … Also, I doubt that there are any examples of P at all—could you give another one, or address my misgivings about E?’ or instead ‘Despite thinking that there are many examples of P, I don’t think E is one, because …’ then there wouldn’t be a dispute about whether core points were being addressed.
Dave thinks ‘whether E is an example of P’ is not a core point, Carol thinks (I presume) ‘whether there exist any examples of P’ is a core point, and it seems likely to me that both of these can be true and agreed upon by both parties.
These can both be true. But they also may not both be true—for instance, in cases where E is representative of a class of pseudo-examples (i.e., scenarios that have the property of seeming to be examples of P but not actually being examples of P). Similarly, ‘whether E is an example of P’ is often indeed a core point in virtue of ‘if E is not an example of P, why did Dave think that it is?’ being a core point; the latter question often goes to the heart of Dave’s view, and his reasons for holding it!
It also happens to empirically be the case that many (perhaps, most?) real-life analogues of Dave do not consider ‘whether there exist any examples of P’ to be a core point of their claims (or, at least, that is the strong impression one gets from the way in which they respond to inquiries about examples).
Finally, ‘does Dave have any actual examples of P’ is a very strong indicator—strong Bayesian evidence, if you want to view it that way—of whether we ought to believe P, or how seriously we ought to take Dave’s claims. (No, “just evaluate Dave’s argument, aside from examples” is not an acceptable response to this!)
… if Carol’s initial comment were ‘I don’t think E is an example of P, because … Also, I doubt that there are any examples of P at all—could you give another one, or address my misgivings about E?’ …
Doubting that there are any examples of P is not, so to speak, Carol’s job. The claim is that E is an example of P. The only reason Carol has for thinking that there are examples of P (excepting cases where P is something well-known, of which there are obviously many examples) is that Dave has described E to the reader. Once E is disqualified, Carol is back to having no particular reason to believe that there are any examples of P.
Once E is disqualified, it is (or it ought to be!) implied that supplying other examples of P is now incumbent upon Dave. Carol bears no obligation (either epistemic or rhetorical) to commit to any position on the question of “are there any examples of P”, in order for Dave to be faced with the need to provide replacement examples.
In short, I think that “having made a claim, has Dave in fact provided any actual examples of the claimed thing” is (barring edge cases) always a core point.
Doubting that there are any examples of P is not, so to speak, Carol’s job. The claim is that E is an example of P. The only reason Carol has for thinking that there are examples of P (excepting cases where P is something well-known, of which there are obviously many examples) is that Dave has described E to the reader. Once E is disqualified, Carol is back to having no particular reason to believe that there are any examples of P.
It seems to me that there’s likely to be enough cases where there are differences in opinion about whether P is well-known enough that examples aren’t needed, or whether P isn’t well-known but whether the reader upon hearing a definition could think of examples themselves, that it’s useful to have norms whereby we clarify whether or not we doubt that there examples of P.
All of the cases I am thinking of are those where P is a new concept, which the author is defining / describing / “crystallizing” for the first time. As such, it seems unlikely that this sort of edge case would apply.
I do agree that working examples are quite important (and that this is something authors should be encouraged to provide).
The issue I expect to be relevant to your past experience is different takes on which examples are valid.
(My impression is that it is often the case, esp. with discussions relating to internal mental states, that the author provides something that makes total sense to them, and me, but doesn’t make sense to you, and then you continuously ask for better/different examples when it seems like the underlying issue is that for whatever reason, the particular mental phenomena their referencing isn’t relevant to you.
How to resolve this seems like a different question that the rest of this thread is focusing on. But I think by this point a lot of people not-providing-you-in-particular-with-examples is because they don’t expect your criticism of their examples to be that useful.)
Oh, certainly this is a fair point. No argument there! But we can agree, I think, that “you say E is not an example of P, but I maintain that it is” is not at all the same thing as “you’re not addressing the core point”—yes?
Your criticism of steelmanning is apt. Of “interpretive effort” I have already spoken elsewhere. As for “addressing core points”, however…
The problem with this criterion is simply that an author and a reader may disagree on what the core points are.
Note that I am not simply talking about cases where a reader misunderstands what the author wrote, and mistakes something for an intended “core point” that is no such thing! Such cases are, in a sense, trivial, insofar as clearing up the misunderstanding results in the original piece standing unmodified (excepting, perhaps, any clarifying modifications that are aimed at preventing exactly such misunderstanding from reoccurring in other readers).
Rather, I’m talking about cases where the reader understands what the author is saying, but considers the author’s claimed “core point” to actually be of peripheral significance, and considers a different point (one which the author either did not mention at all, or devoted only scant attention to) to be central. In this case, no misunderstanding, per se, is occurring (unless you call a deep conceptual or empirical error a “misunderstanding”, which I do not—I reserve the latter term for miscommunication).
This sort of thing can come in many forms. I will describe just one of them here.
It sometimes happens that someone—call him Dave—will write a post describing some (purported) phenomenon P—some pattern, some dynamic, some causal structure, etc.—and, to illustrate P, will provide an example E.
Carol, a reader, then comes along and says: “Actually, Dave, I don’t think E is an example of P! Consider the following…” Dave perhaps defends his example, perhaps not, but agrees, in the end, that E is not really a good example of P after all. “But,” says Dave, “that was just an example! Quit nitpicking, Carol—address my core points!”
What is Carol to say to this? Does Dave accuse her fairly? Is it mere nitpicking to attack a mere example?
But if E is the only example of P that Dave had provided, then—E having now been disqualified from that role—Dave is left with no examples of P. And if Dave cannot provide any other (or, I should simply say, any) examples of P, then perhaps P is simply not real? I can hardly thinking of a more “core” point than the question of whether the thing we’re talking about is even a real thing!
As I say, the “attacking an example” form is just one of the ways in which the “disagreement about what the core points are” dynamic can manifest. But it alone has, in my view, been responsible for a great deal of epistemic trouble among rationalists. As I’ve said elsewhere, we—the types of people who frequent such sites as Less Wrong—are very good at inventing abstract patterns, “crystallizing concepts”, constructing systems of classification, and so on. We are so good at it, in fact, that this talent of ours often gets away from us. It is absolutely critical to keep it reined in. (I do not choose the expression lightly! Reins, after all, are used, not to prevent a horse from moving, but to make it move in the direction you want to go—instead of uselessly running wild.) And those reins are made of real-world examples, they are made of extensions (as vs. intensions), they are made of practice. Without it, deceiving ourselves that we’ve gained knowledge, when in fact we’re building sky castles of abstract nonsense, is all too easy.
Perhaps you have other examples of dynamics where what the ‘core points’ are is in dispute, but the Carol and Dave case seems like one where there’s just a sort of miscommunication: Dave thinks ‘whether E is an example of P’ is not a core point, Carol thinks (I presume) ‘whether there exist any examples of P’ is a core point, and it seems likely to me that both of these can be true and agreed upon by both parties. I’d imagine that if Carol’s initial comment were ‘I don’t think E is an example of P, because … Also, I doubt that there are any examples of P at all—could you give another one, or address my misgivings about E?’ or instead ‘Despite thinking that there are many examples of P, I don’t think E is one, because …’ then there wouldn’t be a dispute about whether core points were being addressed.
These can both be true. But they also may not both be true—for instance, in cases where E is representative of a class of pseudo-examples (i.e., scenarios that have the property of seeming to be examples of P but not actually being examples of P). Similarly, ‘whether E is an example of P’ is often indeed a core point in virtue of ‘if E is not an example of P, why did Dave think that it is?’ being a core point; the latter question often goes to the heart of Dave’s view, and his reasons for holding it!
It also happens to empirically be the case that many (perhaps, most?) real-life analogues of Dave do not consider ‘whether there exist any examples of P’ to be a core point of their claims (or, at least, that is the strong impression one gets from the way in which they respond to inquiries about examples).
Finally, ‘does Dave have any actual examples of P’ is a very strong indicator—strong Bayesian evidence, if you want to view it that way—of whether we ought to believe P, or how seriously we ought to take Dave’s claims. (No, “just evaluate Dave’s argument, aside from examples” is not an acceptable response to this!)
Doubting that there are any examples of P is not, so to speak, Carol’s job. The claim is that E is an example of P. The only reason Carol has for thinking that there are examples of P (excepting cases where P is something well-known, of which there are obviously many examples) is that Dave has described E to the reader. Once E is disqualified, Carol is back to having no particular reason to believe that there are any examples of P.
Once E is disqualified, it is (or it ought to be!) implied that supplying other examples of P is now incumbent upon Dave. Carol bears no obligation (either epistemic or rhetorical) to commit to any position on the question of “are there any examples of P”, in order for Dave to be faced with the need to provide replacement examples.
In short, I think that “having made a claim, has Dave in fact provided any actual examples of the claimed thing” is (barring edge cases) always a core point.
It seems to me that there’s likely to be enough cases where there are differences in opinion about whether P is well-known enough that examples aren’t needed, or whether P isn’t well-known but whether the reader upon hearing a definition could think of examples themselves, that it’s useful to have norms whereby we clarify whether or not we doubt that there examples of P.
All of the cases I am thinking of are those where P is a new concept, which the author is defining / describing / “crystallizing” for the first time. As such, it seems unlikely that this sort of edge case would apply.
I do agree that working examples are quite important (and that this is something authors should be encouraged to provide).
The issue I expect to be relevant to your past experience is different takes on which examples are valid.
(My impression is that it is often the case, esp. with discussions relating to internal mental states, that the author provides something that makes total sense to them, and me, but doesn’t make sense to you, and then you continuously ask for better/different examples when it seems like the underlying issue is that for whatever reason, the particular mental phenomena their referencing isn’t relevant to you.
How to resolve this seems like a different question that the rest of this thread is focusing on. But I think by this point a lot of people not-providing-you-in-particular-with-examples is because they don’t expect your criticism of their examples to be that useful.)
Oh, certainly this is a fair point. No argument there! But we can agree, I think, that “you say E is not an example of P, but I maintain that it is” is not at all the same thing as “you’re not addressing the core point”—yes?