Chosen a number: no (though it does at temperature zero).
Has something approximating a plan for how the ‘conversation’ will go (including which questions are most favoured at each step and go with which numbers), yes to some extent. I do think “plan” is a misleading word, though I don’t have anything better.
I think the realization I’m coming to is that folks on this thread have a shared understanding of the basic mechanics (we seem to be agreed on what computations are occurring, we don’t seem to be making any different predictions), and we are unsure about interpretation. Do you agree?
For myself, I continue to maintain that viewing the system as a next-word sampler is not misleading, and that saying it has a “plan” is misleading—but I try to err very on the side of not anthropomorphizing / not taking an intentional stance (I also try to avoid saying the system “knows” or “understands” anything). I do agree that the system’s activation cache contain a lot of information that collectively biases the next word predictor towards producing the output it produces; I see how someone might reasonably call that a “plan” although I choose not to.
FWIW, I’m not wedded to “plan.” And as for anthropomorphizing, there are many times when anthropomorphic phrasing is easier and more straightforward, so I don’t want to waste time trying to work around it with more complex phrasing. The fact is these devices are fundamentally new and we need to come up with new ways of talking about them. That’s going to take awhile.
Chosen a number: no (though it does at temperature zero).
Has something approximating a plan for how the ‘conversation’ will go (including which questions are most favoured at each step and go with which numbers), yes to some extent. I do think “plan” is a misleading word, though I don’t have anything better.
Thank you, this is helpful.
I think the realization I’m coming to is that folks on this thread have a shared understanding of the basic mechanics (we seem to be agreed on what computations are occurring, we don’t seem to be making any different predictions), and we are unsure about interpretation. Do you agree?
For myself, I continue to maintain that viewing the system as a next-word sampler is not misleading, and that saying it has a “plan” is misleading—but I try to err very on the side of not anthropomorphizing / not taking an intentional stance (I also try to avoid saying the system “knows” or “understands” anything). I do agree that the system’s activation cache contain a lot of information that collectively biases the next word predictor towards producing the output it produces; I see how someone might reasonably call that a “plan” although I choose not to.
FWIW, I’m not wedded to “plan.” And as for anthropomorphizing, there are many times when anthropomorphic phrasing is easier and more straightforward, so I don’t want to waste time trying to work around it with more complex phrasing. The fact is these devices are fundamentally new and we need to come up with new ways of talking about them. That’s going to take awhile.