Context
Imagine a conversation:
[Bob posts a problem]
Alice: You should use technique T₁. It especially suites this kind of problem
Bob: In my understanding this technique is only strong in condition C. If C doesn’t apply then using technique T₂ gives better result?
Alice: Not really. T₂ cannot do A. If you have good input and execution then T₁ is still better
Bob: But the nature of A is still C. As C doesn’t apply then we don’t need to care about not achieving A
Alice: You should try it first. You are reasoning too much
Reasoning
On one hand, I understand the rationale of Alice:
It’s hard for Bob to imagine the full pros and cons of the technique without having direct experience with it
It’s hard for Alice to verbalize implicit or tacit knowledge
It’s time and effort consuming for Alice
But still, isn’t that Alice doesn’t focus on the important topic: giving reasons to use T₁ in this problem? She derails on the experience of Bob, which is not relevant to the argument. It’s possible that Bob has already tried T₁ but fails to see its strengths on the problem, or consulted other experts and documentations to see that his assumptions haven’t been challenged successfully. Regardless of the experience state, it doesn’t contribute to the reasons to use T₁. I think a good explainer should be able to convince a no-experience person to abandon their initial assumptions. Of course Alice is not obligated to continue the conversation, but if she decides to continue, then she should stick to the real topic.
Questions
Therefore, I think these questions would help me validate the assumptions in the reasoning part above:
Is it correct that when Alice says “You should try it first. You are reasoning too much”, she is no longer giving reasons on why T₁ is better?
Is it correct that the experience state of Bob isn’t relevant to the reasons why T₁ is better?
Is it correct that it’s possible for one to get closer to the truth via reasoning and conversing with people with direct experience, while not having direct experience themself? After all there are theoretical scientists and experimental scientists. Isn’t that theoretical scientists “think too much and try nothing”, and their contributions are still valuable?
On what basis can Alice assume that Bob hasn’t tried T₁, when Bob says that T₁ is only strong in C, the nature of A is C, etc?
Is it correct that saying “You should try it first. You are reasoning too much” is a way to shut down the conversation?
Is it correct that shutting down the conversation is nonconstructive? Is it correct that using a irrelevant argument that looks like a relevant one is a fallacy? Is it correct that talking about Bob’s experience a kind of ad hominem?
Your belief that “a good explainer should be able to convince a no-experience person to abandon their initial assumptions” is quite false in practice and you should abandon this initial assumption. Good communication is hard, communication against an already formed opinion is much harder still, and doing so in 3 short lines of conversation is verging on impossible.
It may or may not be true that Alice is providing a good argument for T1, and Bob may or may not have a valid point about A but there’s no way to know since all the relevant information about that has been stripped from the scenario. It’s certainly not true that in general Alice has made a poor argument.
If this sort of exchange happened in a real conversation (and I have seen some similar cases in practice), I would interpret the last line to mean that Alice is writing off helping Bob as a waste of time since any further communication is likely to result in ongoing argument and no useful progress (see first paragraph).
It is very unlikely that Bob has tried T1, since he gave a very weak theoretical argument in favour of using T2 instead, rather than a much stronger and practical argument “I tried T1, and here’s why it didn’t help”. Likewise he claims to know about T2 and believes it to be better than T1 but fairly clearly hasn’t tried that one either. Why not?
Thanks for your reply. It helps me refine my thinking. I have come up these questions, hope you can help me answering them:
Is it correct that when Alice says “You should try it first. You are reasoning too much”, she is no longer giving reasons on why T₁ is better?
Is it correct that the experience state of Bob isn’t relevant to the reasons why T₁ is better?
Is it correct that it’s possible for one to get closer to the truth via reasoning and conversing with people with direct experience, while not having direct experience themself? After all there are theoretical scientists and experimental scientists. Isn’t that theoretical scientists “think too much and try nothing”, and their contributions are still valuable?
On what basis can Alice assume that Bob hasn’t tried T₁, when Bob says that T₁ is only strong in C, the nature of A is C, etc?
The implication is that once Bob actually tries it, Bob will find that it’s better. That is, Alice is showing how Bob can get evidence that it’s better in reality, rather than playing around with models that may or may not predict which is better. Evidence is better than arguments.
Bob’s experience state isn’t relevant to whether T1 is better than T2 in this situation, but is relevant to whether Bob should take notice of any particular reasons for one or the other.
Of course theory should inform experiment. Theory cannot replace experiment, however.
Alice does not need to merely assume it, Alice has evidence of it.
I guess it all comes down to this question: when Bob says T₁ is only strong in C, the nature of A is C, or C doesn’t apply to the problem, why that is the evidence for him to haven’t tried it, but not the evidence that he has actually tried it? You said that:
But I don’t see how saying “T₁ is only strong in C, the nature of A is C, C doesn’t apply to the problem” is to argue how better T₂ is, but only about how T₁ doesn’t help. Yes, the exact wording doesn’t fit, but I interpret the meaning isn’t much different. How is that different to “I tried T₁, and I realize that it only yields better results under condition C. But the problem doesn’t have C to begin with, so it didn’t help here”?
The meaning is very different.
To begin with, Bob actually starting this sentence with “In my understanding, …”. That is, Bob says that he has a mental model of how T1 works in relation to C (and A), and is specifically qualifying that this is purely Bob’s mental model. Most people who have actual practical experience demonstrating that something doesn’t work are more likely to say so instead of qualifying some theoretical statement with “in my understanding” and expectations under “if … then …” clauses.
It is still possible that Bob has actually tried T1 and that he’s just very bad at communicating, of course.
It’s also possible that Bob’s stated mental model is actually correct, and T1 isn’t strong when C doesn’t apply. That still isn’t a refutation of using technique T1, since there may be no better technique to use. So the only moderately strong argument against using T1 is Bob’s second sentence, which is what I was referring to in my sentence quoted.
Edit: This will be my last comment on the topic. It’s especially frustrating having to talk about a communication scenario in such vague generalities due to omission of almost all of the communication-relevant information from the original post.
Here is the full dialog, in case you are still interested.
[Bob posts a problem about data classification]
Alice: You should use LLM. It especially suites your problem
Bob: In my understanding LLM is only strong where the context is large. If the context is small then using regex gives better result? Also, regex has advantages of high accuracy, fast, understandable and debuggable?
Alice: Not really. Also, regex cannot work with synonyms and it must be in form. LLM is trained on multiple data, so if you make good prompt then it’s much better
Bob: But the nature of catching synonyms is still depending on context. As the context is small then there is not much synonyms at the beginning. If even human cannot get them then how can machine recognize them?
Alice: You should try it first. You are reasoning too much
There are some notes:
By “regex” Bob actually means rule-based approach. He thought in the context of NLP people generally understand regex and rule-based approach as one
He mistakes synonym with homonym. Had he been aware of that he might have not said “the nature of catching synonyms is still depending on context”
These info are only revealed later on.
Not actually assume, but that’s certainly Bayesian evidence (should Bob have tried T1, he would likely respond in another way).
Also, :smile: your own comment is a fairly large bit of evidence that you haven’t yet read the Sequences (by the way, I recommend doing that). For instance, you can consider different ways of thinking, answer the questions 1-4 from their perspectives, and that would be evidence on which way is better—though, reality is still the ultimate judge how each situation turns out.
Why doesn’t saying “T₁ is only strong in C, the nature of A is C” indicate that he has tried it?
Yes, I haven’t read the Sequences yet. Just the new user’s guide. To make it quick, can you give me the section that’s relevant to the question?
By “their perspectives”, whose? If Alice’s, then I think she would say all of them are incorrect. Because that’s her position at the beginning. And that’s useless information, since I don’t know why she thinks so. If I know then I wouldn’t ask this question at the beginning.