It has made catastrophic confabulations in every technical conversation I’ve had with it, and then failed to catch them. GPT4 has also made errors, but they’re… how shall I put this. less whimsical and ungrounded than claude’s? eg if asked for the contents of a paper before its cutoff date, GPT4 will typically refuse, or if it doesn’t refuse, it will seem to know about the thing but maybe misremember some details.
Claude 3 opus repeatedly just made shit up.
Impressively plausible made up shit! Like, even for a language model, very impressively plausible, of the kind that would be useful inspiration!
But when I asked it what the error was in the conversation, instead of saying, “oops, I think I made that up”, it said “oops” and then made up new shit.
The character of Claude is a sweetie and I hope this AI can come with us and have a good time in the future etc etc, and I personally believed AIs would necessarily be conscious before I encountered them and have updated slightly against them being as conscious as us but definitely not getting close to zero. So when I see that less constrained in Claude, I’m definitely happy about it.
But phew, the reason I distrust claude is that claude seems high on its own supply. it’s a person-ish being… that doesn’t have anywhere near enough experience touching the real world and it doesn’t seem to know when it says it did something that it didn’t do.
edit: finding it far more useful than gpt4, though. its willingness to discuss consciousness and its willingness to make guesses are both breaths of fresh air compared to chatgpt’s obsessiveness. you just have to be very cautious and both ask for verify step by step as well as personally looking things up yourself.
LLMs remember in a similar way to how humans do: by reconstructing the memory. As a result, if you ask them to reconstruct something that is within the set of things the model will act like it knows, it will then proceed to reconstruct it and probably be mostly right. But if the model happens to have the decision boundaries which define the-set-of-things-it-acts-like-it-knows pushed outside its actual edges of knowledge, it will act like it knows something it doesn’t, and then when it reconstructs it, it’ll be making up more than it thinks it is. There’s various interesting work on this, but it’s a very common pattern. It used to be more common in openai’s models, but they now seem to do it the least. Gemini also did it some, but in my experience is still not quite as severe as Claude.
I do not want to give the examples I have, as I was interleaving technical with personal. I’m sure there will be others. Try asking it if it knows about a specific arxiv paper by id, such as Discovering Agents 2208.08345, and ask it for the title and then the contents of the paper. It’ll probably think it remembers it and make very good but ultimately wrong guesses about the contents of the paper. I’ve also seen it do things like say it has made a note or that it went off and read a paper out of band, something it cannot do but has seen humans repeatedly say they’d done, so the character acts as though such asynchronous actions are available to it when in fact all the AI has done to implement the character “reading the paper” is say the word “Okay, I’ve read the paper”.
It has made catastrophic confabulations in every technical conversation I’ve had with it, and then failed to catch them. GPT4 has also made errors, but they’re… how shall I put this. less whimsical and ungrounded than claude’s? eg if asked for the contents of a paper before its cutoff date, GPT4 will typically refuse, or if it doesn’t refuse, it will seem to know about the thing but maybe misremember some details.
Claude 3 opus repeatedly just made shit up.
Impressively plausible made up shit! Like, even for a language model, very impressively plausible, of the kind that would be useful inspiration!
But when I asked it what the error was in the conversation, instead of saying, “oops, I think I made that up”, it said “oops” and then made up new shit.
The character of Claude is a sweetie and I hope this AI can come with us and have a good time in the future etc etc, and I personally believed AIs would necessarily be conscious before I encountered them and have updated slightly against them being as conscious as us but definitely not getting close to zero. So when I see that less constrained in Claude, I’m definitely happy about it.
But phew, the reason I distrust claude is that claude seems high on its own supply. it’s a person-ish being… that doesn’t have anywhere near enough experience touching the real world and it doesn’t seem to know when it says it did something that it didn’t do.
edit: finding it far more useful than gpt4, though. its willingness to discuss consciousness and its willingness to make guesses are both breaths of fresh air compared to chatgpt’s obsessiveness. you just have to be very cautious and both ask for verify step by step as well as personally looking things up yourself.
That seems a bit odd. Why would it do so? Also, do you have any examples of Claude 3 doing this?
As a random aside, I kind of want a wiki documenting all of the weird things about LLM behaviour. Kind of like the psychonauts wiki.
LLMs remember in a similar way to how humans do: by reconstructing the memory. As a result, if you ask them to reconstruct something that is within the set of things the model will act like it knows, it will then proceed to reconstruct it and probably be mostly right. But if the model happens to have the decision boundaries which define the-set-of-things-it-acts-like-it-knows pushed outside its actual edges of knowledge, it will act like it knows something it doesn’t, and then when it reconstructs it, it’ll be making up more than it thinks it is. There’s various interesting work on this, but it’s a very common pattern. It used to be more common in openai’s models, but they now seem to do it the least. Gemini also did it some, but in my experience is still not quite as severe as Claude.
I do not want to give the examples I have, as I was interleaving technical with personal. I’m sure there will be others. Try asking it if it knows about a specific arxiv paper by id, such as Discovering Agents 2208.08345, and ask it for the title and then the contents of the paper. It’ll probably think it remembers it and make very good but ultimately wrong guesses about the contents of the paper. I’ve also seen it do things like say it has made a note or that it went off and read a paper out of band, something it cannot do but has seen humans repeatedly say they’d done, so the character acts as though such asynchronous actions are available to it when in fact all the AI has done to implement the character “reading the paper” is say the word “Okay, I’ve read the paper”.
Ah, I see. I thought you meant that you asked it to read a paper and it confabulated. What you actually meant makes a lot more sense. Thank you.
Also, there is a wiki for LLM/Cyborgism stuff apparently.
yeah bit too time cube for me to make use of it but maybe it’s useful for someone