My main issue with the terms ‘simulator’, ‘simulation’, ‘simulacra’, etc is that a language model ‘simulating a simulacrum’ doesn’t correspond to a single simulation of reality, even in the high-capability limit. Instead, language model generation corresponds to a distribution over all the settings of latent variables which could have produced the previous tokens, aka “a prediction”.
The way I tend to think of ‘simulators’ is in simulating a distribution over worlds (i.e., latent variables) that increasingly collapses as prompt information determines specific processes with higher probability. I don’t think I’ve ever really thought of it as corresponding to a specific simulation of reality. Likewise with simulacra, I tend to think of them as any process that could contribute to changes in the behavioural logs of something in a simulation. (Related)
I’ve seen this mistake made frequently – for example, see this post (note that in this case the mistake doesn’t change the conclusion of the post).
[...]
this issue makes this terminology misleading.
I think that there were a lot of mistaken takes about GPT before Simulators, and that it’s plausible the count just went down. Certainly there have been a non-trivial number of people I’ve spoken to who were making pretty specific mistakes that the post cleared up for them—they may have had further mistakes, but thinking of models as predictors didn’t get them far enough to make those mistakes earlier. I think in general the reason I like the simulator framing so much is because it’s a very evocative frame, that gives you more accessible understanding about GPT mechanics. There have certainly been insights I’ve had about GPT in the last year that I don’t think thinking about next-token predictors would’ve evoked quite as easily.
The way I tend to think of ‘simulators’ is in simulating a distribution over worlds (i.e., latent variables) that increasingly collapses as prompt information determines specific processes with higher probability.
I agree this is the correct interpretation of the original post. It just doesn’t match typical usage of the world simulation imo. (I’m sorry my post is making such a narrow pedantic point).
I probably agree that simulators improved the thinking of people on lesswrong on average.
I don’t disagree that there aren’t people who came away with the wrong impression (though they’ve been at most a small minority of people I’ve talked to, you’ve plausibly spoken to more people). But I think that might be owed more to generative models being confusing to think about intrinsically. Speaking of them purely as predictive models probably nets you points for technical accuracy, but I’d bet it would still lead to a fair number of people thinking about them the wrong way.
The way I tend to think of ‘simulators’ is in simulating a distribution over worlds (i.e., latent variables) that increasingly collapses as prompt information determines specific processes with higher probability. I don’t think I’ve ever really thought of it as corresponding to a specific simulation of reality. Likewise with simulacra, I tend to think of them as any process that could contribute to changes in the behavioural logs of something in a simulation. (Related)
I think that there were a lot of mistaken takes about GPT before Simulators, and that it’s plausible the count just went down. Certainly there have been a non-trivial number of people I’ve spoken to who were making pretty specific mistakes that the post cleared up for them—they may have had further mistakes, but thinking of models as predictors didn’t get them far enough to make those mistakes earlier. I think in general the reason I like the simulator framing so much is because it’s a very evocative frame, that gives you more accessible understanding about GPT mechanics. There have certainly been insights I’ve had about GPT in the last year that I don’t think thinking about next-token predictors would’ve evoked quite as easily.
I agree this is the correct interpretation of the original post. It just doesn’t match typical usage of the world simulation imo. (I’m sorry my post is making such a narrow pedantic point).
I probably agree that simulators improved the thinking of people on lesswrong on average.
I don’t disagree that there aren’t people who came away with the wrong impression (though they’ve been at most a small minority of people I’ve talked to, you’ve plausibly spoken to more people). But I think that might be owed more to generative models being confusing to think about intrinsically. Speaking of them purely as predictive models probably nets you points for technical accuracy, but I’d bet it would still lead to a fair number of people thinking about them the wrong way.