For hidden thoughts, I think this is very well defined.
Not for humans, and that’s what I was referring to. Sorry about the confusion.
“Thought” is just a common-sense idea. As far as I know, we don’t have a well-defined concept of that that’s stated in terms of brain states. Now, I believe Walter Freeman has conjectured that thoughts reflect states of global coherence across a large swath of cortex, perhaps a hemisphere, but that’s a whole other intellectual world.
If so, how can you say that came from a plan—it didn’t write the first half of the story!
But it read it, no? Why can’t it complete it according to it’s “plan” since it has no way of knowing the intentions of the person who wrote the first half.
Let me come at this a different way. I don’t know how many times I’ve read articles of the “computers for dummies” type where it said it’s all just ones and zeros. And that’s true. Source code may be human-readable, when when it’s compiled all the comments are stripped out and the rest is converted to runs and zeros. What does that tell you about a program? It depends on your point of view and what you know. From a very esoteric and abstract point of view, it tells you a lot. From the point of view of someone reading Digital Computing for Dummies, it doesn’t tell them much of anything.
I feel a bit like that about the assertion that LLMs are just next-token-predictors. Taking that in conjunction with the knowledge that they’re trained on zillions of tokens of text, those two things put together don’t tell you much either. If those two statements were deeply informative, then mechanistic interpretation would be trivial. It’s not. Saying that LLMs are next-token predictors puts a kind of boundary on mechanistic interpretation, but it doesn’t do much else. And saying it was trained on all these texts, that doesn’t tell you much about the structure the model has picked up.
On the contrary, you mainly seem to be claiming that thinking of LLMs as working one token at a time is misleading, but I’m not sure I understand any examples of misleading conclusions that you think people draw from it. Where do you think people go wrong?
Over there in another part of the universe there are people who are yelling that LLMs are “stochastic parrots.” Their intention is to discredit LLMs as dangerous evil devices Not too far away from those folks are those saying it’s “autocomplete on steroids.” That’s only marginally better.
Saying LLMs are “next word predictors” feeds into that. Now, I’m talking about rhetoric here, not intellectual substance. But rhetoric matters. There needs to be a better way of talking about these devices for a general audience.
For hidden thoughts, I think this is very well defined.
Not for humans, and that’s what I was referring to. Sorry about the confusion.
“Thought” is just a common-sense idea. As far as I know, we don’t have a well-defined concept of that that’s stated in terms of brain states. Now, I believe Walter Freeman has conjectured that thoughts reflect states of global coherence across a large swath of cortex, perhaps a hemisphere, but that’s a whole other intellectual world.
If so, how can you say that came from a plan—it didn’t write the first half of the story!
But it read it, no? Why can’t it complete it according to it’s “plan” since it has no way of knowing the intentions of the person who wrote the first half.
Let me come at this a different way. I don’t know how many times I’ve read articles of the “computers for dummies” type where it said it’s all just ones and zeros. And that’s true. Source code may be human-readable, when when it’s compiled all the comments are stripped out and the rest is converted to runs and zeros. What does that tell you about a program? It depends on your point of view and what you know. From a very esoteric and abstract point of view, it tells you a lot. From the point of view of someone reading Digital Computing for Dummies, it doesn’t tell them much of anything.
I feel a bit like that about the assertion that LLMs are just next-token-predictors. Taking that in conjunction with the knowledge that they’re trained on zillions of tokens of text, those two things put together don’t tell you much either. If those two statements were deeply informative, then mechanistic interpretation would be trivial. It’s not. Saying that LLMs are next-token predictors puts a kind of boundary on mechanistic interpretation, but it doesn’t do much else. And saying it was trained on all these texts, that doesn’t tell you much about the structure the model has picked up.
What intellectual work does that statement do?
I gave one example of the “work” this does: that GPT performs better when prompted to reason first rather than state the answer first. Another example is: https://www.lesswrong.com/posts/bwyKCQD7PFWKhELMr/by-default-gpts-think-in-plain-sight
On the contrary, you mainly seem to be claiming that thinking of LLMs as working one token at a time is misleading, but I’m not sure I understand any examples of misleading conclusions that you think people draw from it. Where do you think people go wrong?
Over there in another part of the universe there are people who are yelling that LLMs are “stochastic parrots.” Their intention is to discredit LLMs as dangerous evil devices Not too far away from those folks are those saying it’s “autocomplete on steroids.” That’s only marginally better.
Saying LLMs are “next word predictors” feeds into that. Now, I’m talking about rhetoric here, not intellectual substance. But rhetoric matters. There needs to be a better way of talking about these devices for a general audience.
Oh, thanks for the link. It looks interesting.