The implication is that once Bob actually tries it, Bob will find that it’s better. That is, Alice is showing how Bob can get evidence that it’s better in reality, rather than playing around with models that may or may not predict which is better. Evidence is better than arguments.
Bob’s experience state isn’t relevant to whether T1 is better than T2 in this situation, but is relevant to whether Bob should take notice of any particular reasons for one or the other.
Of course theory should inform experiment. Theory cannot replace experiment, however.
Alice does not need to merely assume it, Alice has evidence of it.
I guess it all comes down to this question: when Bob says T₁ is only strong in C, the nature of A is C, or C doesn’t apply to the problem, why that is the evidence for him to haven’t tried it, but not the evidence that he has actually tried it? You said that:
It is very unlikely that Bob has tried T1, since he gave a very weak theoretical argument in favour of using T2 instead, rather than a much stronger and practical argument “I tried T1, and here’s why it didn’t help”.
But I don’t see how saying “T₁ is only strong in C, the nature of A is C, C doesn’t apply to the problem” is to argue how better T₂ is, but only about how T₁ doesn’t help. Yes, the exact wording doesn’t fit, but I interpret the meaning isn’t much different. How is that different to “I tried T₁, and I realize that it only yields better results under condition C. But the problem doesn’t have C to begin with, so it didn’t help here”?
To begin with, Bob actually starting this sentence with “In my understanding, …”. That is, Bob says that he has a mental model of how T1 works in relation to C (and A), and is specifically qualifying that this is purely Bob’s mental model. Most people who have actual practical experience demonstrating that something doesn’t work are more likely to say so instead of qualifying some theoretical statement with “in my understanding” and expectations under “if … then …” clauses.
It is still possible that Bob has actually tried T1 and that he’s just very bad at communicating, of course.
It’s also possible that Bob’s stated mental model is actually correct, and T1 isn’t strong when C doesn’t apply. That still isn’t a refutation of using technique T1, since there may be no better technique to use. So the only moderately strong argument against using T1 is Bob’s second sentence, which is what I was referring to in my sentence quoted.
Edit: This will be my last comment on the topic. It’s especially frustrating having to talk about a communication scenario in such vague generalities due to omission of almost all of the communication-relevant information from the original post.
Here is the full dialog, in case you are still interested.
[Bob posts a problem about data classification]
Alice: You should use LLM. It especially suites your problem
Bob: In my understanding LLM is only strong where the context is large. If the context is small then using regex gives better result? Also, regex has advantages of high accuracy, fast, understandable and debuggable?
Alice: Not really. Also, regex cannot work with synonyms and it must be in form. LLM is trained on multiple data, so if you make good prompt then it’s much better
Bob: But the nature of catching synonyms is still depending on context. As the context is small then there is not much synonyms at the beginning. If even human cannot get them then how can machine recognize them?
Alice: You should try it first. You are reasoning too much
There are some notes:
By “regex” Bob actually means rule-based approach. He thought in the context of NLP people generally understand regex and rule-based approach as one
He mistakes synonym with homonym. Had he been aware of that he might have not said “the nature of catching synonyms is still depending on context”
The implication is that once Bob actually tries it, Bob will find that it’s better. That is, Alice is showing how Bob can get evidence that it’s better in reality, rather than playing around with models that may or may not predict which is better. Evidence is better than arguments.
Bob’s experience state isn’t relevant to whether T1 is better than T2 in this situation, but is relevant to whether Bob should take notice of any particular reasons for one or the other.
Of course theory should inform experiment. Theory cannot replace experiment, however.
Alice does not need to merely assume it, Alice has evidence of it.
I guess it all comes down to this question: when Bob says T₁ is only strong in C, the nature of A is C, or C doesn’t apply to the problem, why that is the evidence for him to haven’t tried it, but not the evidence that he has actually tried it? You said that:
But I don’t see how saying “T₁ is only strong in C, the nature of A is C, C doesn’t apply to the problem” is to argue how better T₂ is, but only about how T₁ doesn’t help. Yes, the exact wording doesn’t fit, but I interpret the meaning isn’t much different. How is that different to “I tried T₁, and I realize that it only yields better results under condition C. But the problem doesn’t have C to begin with, so it didn’t help here”?
The meaning is very different.
To begin with, Bob actually starting this sentence with “In my understanding, …”. That is, Bob says that he has a mental model of how T1 works in relation to C (and A), and is specifically qualifying that this is purely Bob’s mental model. Most people who have actual practical experience demonstrating that something doesn’t work are more likely to say so instead of qualifying some theoretical statement with “in my understanding” and expectations under “if … then …” clauses.
It is still possible that Bob has actually tried T1 and that he’s just very bad at communicating, of course.
It’s also possible that Bob’s stated mental model is actually correct, and T1 isn’t strong when C doesn’t apply. That still isn’t a refutation of using technique T1, since there may be no better technique to use. So the only moderately strong argument against using T1 is Bob’s second sentence, which is what I was referring to in my sentence quoted.
Edit: This will be my last comment on the topic. It’s especially frustrating having to talk about a communication scenario in such vague generalities due to omission of almost all of the communication-relevant information from the original post.
Here is the full dialog, in case you are still interested.
[Bob posts a problem about data classification]
Alice: You should use LLM. It especially suites your problem
Bob: In my understanding LLM is only strong where the context is large. If the context is small then using regex gives better result? Also, regex has advantages of high accuracy, fast, understandable and debuggable?
Alice: Not really. Also, regex cannot work with synonyms and it must be in form. LLM is trained on multiple data, so if you make good prompt then it’s much better
Bob: But the nature of catching synonyms is still depending on context. As the context is small then there is not much synonyms at the beginning. If even human cannot get them then how can machine recognize them?
Alice: You should try it first. You are reasoning too much
There are some notes:
By “regex” Bob actually means rule-based approach. He thought in the context of NLP people generally understand regex and rule-based approach as one
He mistakes synonym with homonym. Had he been aware of that he might have not said “the nature of catching synonyms is still depending on context”
These info are only revealed later on.