Parameters will get outdated in the same way we don’t measure CPU performance using gigahertz anymore.
This was in the context of when he was asked about the number of parameters of GPT-4. He said that the big changes are not in the number of parameters but in the structure of the model. And he made a pretty excited and/or confident impression on me when he said it. I wouldn’t be surprised if the nxt GPT-N is much better without many more parameters.
This was in the context of when he was asked about the number of parameters of GPT-4. He said that the big changes are not in the number of parameters but in the structure of the model. And he made a pretty excited and/or confident impression on me when he said it. I wouldn’t be surprised if the nxt GPT-N is much better without many more parameters.