I broadly agree with the points being made here, but allow me to nitpick the use of the word “predictive” here, and argue for the key advantage of the simulators framing over the prediction one:
Pretrained models don’t ‘simulate a character speaking’; they predict what comes next, which implicitly involves making predictions about the distribution of characters and what they would say next.
The simulators frame does make it very clear that there’s a distinction between the simulator/GPT-3 and the simulacra/characters or situations it’s making predictions about! On the other hand, using “prediction” can obscure the distinction, and end up with confused questions like “is GPT just an agent that just wants to minimize predictive loss?”
I broadly agree with the points being made here, but allow me to nitpick the use of the word “predictive” here, and argue for the key advantage of the simulators framing over the prediction one:
The simulators frame does make it very clear that there’s a distinction between the simulator/GPT-3 and the simulacra/characters or situations it’s making predictions about! On the other hand, using “prediction” can obscure the distinction, and end up with confused questions like “is GPT just an agent that just wants to minimize predictive loss?”