I think he’s referring to the understanding of the precise mechanics of how transistors worked, or why the particular first working prototypes functioned while all the others didn’t. Just from skimming https://en.wikipedia.org/wiki/History_of_the_transistor
That’s the current understanding for llms—people do know at a high level what an llm does and why it works, just like there were theories decades before working transistors on their function. But the details of why this system works but 50 other things tried didn’t is not known.
I think he’s referring to the understanding of the precise mechanics of how transistors worked, or why the particular first working prototypes functioned while all the others didn’t. Just from skimming https://en.wikipedia.org/wiki/History_of_the_transistor
That’s the current understanding for llms—people do know at a high level what an llm does and why it works, just like there were theories decades before working transistors on their function. But the details of why this system works but 50 other things tried didn’t is not known.