Obviously there’s nothing original in my writeup as opposed to the paper it’s about. The paper still seems like an important one, though I haven’t particularly followed the literature and wouldn’t know if it’s been refuted or built upon by other later work. In particular, in popular AI discourse one constantly hears things along the lines of “LLMs are just pushing symbols around and don’t have any sort of model of the actual world in them”, and this paper seems to me to be good evidence that transformer networks, even quite small ones, can build internal models that aren’t just symbol-pushing.
There’s a substantial error in the post as it stands (corrected in comments, but I never edited the post): I claim that the different legal-move-prediction abilities of the “network trained on a smallish number of good games” and “network trained on a much larger number of random games” cases is because the network isn’t big enough to capture both “legal” and “good strategy” well, when in fact it seems more likely that the difference is mostly because the random-game training set is so much larger. I’m not sure what the etiquette is around making edits now in such cases.
(Brief self-review for LW 2023 review.)
Obviously there’s nothing original in my writeup as opposed to the paper it’s about. The paper still seems like an important one, though I haven’t particularly followed the literature and wouldn’t know if it’s been refuted or built upon by other later work. In particular, in popular AI discourse one constantly hears things along the lines of “LLMs are just pushing symbols around and don’t have any sort of model of the actual world in them”, and this paper seems to me to be good evidence that transformer networks, even quite small ones, can build internal models that aren’t just symbol-pushing.
There’s a substantial error in the post as it stands (corrected in comments, but I never edited the post): I claim that the different legal-move-prediction abilities of the “network trained on a smallish number of good games” and “network trained on a much larger number of random games” cases is because the network isn’t big enough to capture both “legal” and “good strategy” well, when in fact it seems more likely that the difference is mostly because the random-game training set is so much larger. I’m not sure what the etiquette is around making edits now in such cases.