I’ve just taken a quick look. & have a quick and crude reaction.
Consider how natural language is learned. The infant & toddler is surrounded be people who speak. They begin to babble and eventually manage to babble in a way that intends meaning. So they’ve got a device for producing tokens as motor output that produces audio tokens that can intermingle with the audio input tokens being produced by others.
We’re now dealing with two token streams. There’s a large audio stream, with input from various sources. And the smaller motor stream, which is closely correlated with some of the tokens in the audio stream because it has ‘produced’ them.
You need to take a look at Lev Vygotsky’s account of language learning as a process of internalizing the speech streams of others. Here’s a quick intro. Also, think of language as an index over one’s conceptual space. & one LLM can index the space of another.
I’ve just taken a quick look. & have a quick and crude reaction.
Consider how natural language is learned. The infant & toddler is surrounded be people who speak. They begin to babble and eventually manage to babble in a way that intends meaning. So they’ve got a device for producing tokens as motor output that produces audio tokens that can intermingle with the audio input tokens being produced by others.
We’re now dealing with two token streams. There’s a large audio stream, with input from various sources. And the smaller motor stream, which is closely correlated with some of the tokens in the audio stream because it has ‘produced’ them.
You need to take a look at Lev Vygotsky’s account of language learning as a process of internalizing the speech streams of others. Here’s a quick intro. Also, think of language as an index over one’s conceptual space. & one LLM can index the space of another.