It seems as a result of this post, many people are saying that LLMs simulate people and so on. But I’m not sure that’s quite the right frame. It’s natural if you experience LLMs through chat-like interfaces, but from playing with them in a more raw form, like the RWKV playground, I get a different impression. For example, if I write something that sounds like the start of a quote, it’ll continue with what looks like a list of quotes from different people. Or if I write a short magazine article, it’ll happily tack on a publication date and “All rights reserved”. In other words it’s less like a simulation of some reality or set of realities, and more like a really fuzzy and hallucinatory search engine over the space of texts.
It is of course surprising that a search engine over the space of texts is able to write poems, take derivatives, and play chess. And it’s plausible that a stronger version of the same could outsmart us in more dangerous ways. I’m not trying to downplay the risk here. Just saying that, well, thinking in terms of the space of texts (and capabilities latent in it) feels to me more right than thinking about simulation.
Thinking further along this path, it may be that we don’t need to think much about AI architecture or training methods. What matters is the space of texts—the training dataset—and any additional structure on it that we provide to the AI (valuation, metric, etc). Maybe the solution to alignment, if it exists, could be described in terms of dataset alone, without reference to the AI’s architecture at all.
It seems as a result of this post, many people are saying that LLMs simulate people and so on. But I’m not sure that’s quite the right frame. It’s natural if you experience LLMs through chat-like interfaces, but from playing with them in a more raw form, like the RWKV playground, I get a different impression. For example, if I write something that sounds like the start of a quote, it’ll continue with what looks like a list of quotes from different people. Or if I write a short magazine article, it’ll happily tack on a publication date and “All rights reserved”. In other words it’s less like a simulation of some reality or set of realities, and more like a really fuzzy and hallucinatory search engine over the space of texts.
It is of course surprising that a search engine over the space of texts is able to write poems, take derivatives, and play chess. And it’s plausible that a stronger version of the same could outsmart us in more dangerous ways. I’m not trying to downplay the risk here. Just saying that, well, thinking in terms of the space of texts (and capabilities latent in it) feels to me more right than thinking about simulation.
Thinking further along this path, it may be that we don’t need to think much about AI architecture or training methods. What matters is the space of texts—the training dataset—and any additional structure on it that we provide to the AI (valuation, metric, etc). Maybe the solution to alignment, if it exists, could be described in terms of dataset alone, without reference to the AI’s architecture at all.