I was just in the middle of writing a draft revisiting some of his arguments, but in the meantime one claim that might be of particular interest to you is that: ”...[GPT-N type models] cannot take you more than a couple steps of inferential distance away from the span of concepts frequently used by humans in the training data”
Regarding “posts making a bearish case” against GPT-N, there’s Steve Byrnes’, Can you get AGI from a transformer.
I was just in the middle of writing a draft revisiting some of his arguments, but in the meantime one claim that might be of particular interest to you is that: ”...[GPT-N type models] cannot take you more than a couple steps of inferential distance away from the span of concepts frequently used by humans in the training data”
oo ok, thanks, I’ll take a look. The point about generative models being better is something I’ve been wanting to learn about, in particular.