LLMs hallucinate studies/papers so regularly you’re lucky to get a real one. That doesn’t have an impact on the truth of the facts they claimed beforehand. (Also, yes, Claude 3 Haiku is significantly less intelligent than 3.5 Sonnet.)
It doesn’t directly impact the truth of the facts they claimed, no—but the facts are produced by the same process that produces the studies. It’s easy to check whether or not a study exists; less-clearcut domains aren’t necessarily more reliable.
Yes, but that very same process has a high probability probability of producing correct facts (today’s LLMs are relatively reliable) and a very low probability of producing correct studies or papers.
The reliability of general facts could be checked by various benchmarks. The unreliability of specific studies and papers by personal experience, and by experiences of people I’ve read online.
I don’t understand why, except maybe rephrasing a true fact keeps it true, but rephrasing a study title and a journal title makes it false.
LLMs hallucinate studies/papers so regularly you’re lucky to get a real one. That doesn’t have an impact on the truth of the facts they claimed beforehand. (Also, yes, Claude 3 Haiku is significantly less intelligent than 3.5 Sonnet.)
It doesn’t directly impact the truth of the facts they claimed, no—but the facts are produced by the same process that produces the studies. It’s easy to check whether or not a study exists; less-clearcut domains aren’t necessarily more reliable.
Yes, but that very same process has a high probability probability of producing correct facts (today’s LLMs are relatively reliable) and a very low probability of producing correct studies or papers.
Source?
The reliability of general facts could be checked by various benchmarks. The unreliability of specific studies and papers by personal experience, and by experiences of people I’ve read online.
I don’t understand why, except maybe rephrasing a true fact keeps it true, but rephrasing a study title and a journal title makes it false.
According to Claude: green_leaf et al, 2024