LLMs are, simultaneously, (1) notoriously sycophantic, i. e. biased to answer the way they think the interlocutor wants them to, and (2) have “truesight”, i. e. a literally superhuman ability to suss out the interlocutor’s character (which is to say: the details of the latent structure generating the text) based on subtle details of phrasing. While the same could be said of humans as well – most humans would be biased towards assuaging their interlocutor’s worldview, rather than creating conflict – the problem of “leading questions” rises to a whole new level with LLMs, compared to humans.
You basically have to interpret an LLM being asked something as if a human were asked as biased a way to phrase this question as possible.
I wouldn’t say that my experience with ChatGPT is in total agreement with your conclusion yet you’re raising a good point and the distinction is helpful. I remember of conversations in which the chatbot would both acknowledge and challenge my viewpoint, which I must admit is quite appreciated and not systematic in the biological realm. On the other hand, indeed it is common that pushing the chatbot to buy my arguments and adopt my stance be fairly easy.
Somehow it’s very related to humanlike intelligence; that is, when training an LLM-based chatbot[1] by reinforcement, the positive (rewarding) feedback comes from both confirmation of the interlocutor’s beliefs and matters like veracity, ethics, … It’s also what we humans have been experiencing.
Why and how does it rise to a whole new level when it comes to AI? I tend to think that we must understand the technologies we are using, so it’s our responsibility to use chatbots properly and leverage their capabilities. When talking with a child, or a yound student, or generally someone you know is a newcomer, we adapt our questions, arguments, and the way we process their responses. It’s not an exact science for sure, but there’s no reason to expect so with chatbots.
LLMs are, simultaneously, (1) notoriously sycophantic, i. e. biased to answer the way they think the interlocutor wants them to, and (2) have “truesight”, i. e. a literally superhuman ability to suss out the interlocutor’s character (which is to say: the details of the latent structure generating the text) based on subtle details of phrasing. While the same could be said of humans as well – most humans would be biased towards assuaging their interlocutor’s worldview, rather than creating conflict – the problem of “leading questions” rises to a whole new level with LLMs, compared to humans.
You basically have to interpret an LLM being asked something as if a human were asked as biased a way to phrase this question as possible.
Why do you believe this?
See e. g. this and this, and it’s of course wholly unsurprising, since it’s literally what the base models are trained to do.
I wouldn’t say that my experience with ChatGPT is in total agreement with your conclusion yet you’re raising a good point and the distinction is helpful. I remember of conversations in which the chatbot would both acknowledge and challenge my viewpoint, which I must admit is quite appreciated and not systematic in the biological realm. On the other hand, indeed it is common that pushing the chatbot to buy my arguments and adopt my stance be fairly easy.
Somehow it’s very related to humanlike intelligence; that is, when training an LLM-based chatbot[1] by reinforcement, the positive (rewarding) feedback comes from both confirmation of the interlocutor’s beliefs and matters like veracity, ethics, … It’s also what we humans have been experiencing.
Why and how does it rise to a whole new level when it comes to AI? I tend to think that we must understand the technologies we are using, so it’s our responsibility to use chatbots properly and leverage their capabilities. When talking with a child, or a yound student, or generally someone you know is a newcomer, we adapt our questions, arguments, and the way we process their responses. It’s not an exact science for sure, but there’s no reason to expect so with chatbots.
It seems more accurate than LLMs as those have not yet been trained to have a chat with you