Yes. It’s more about the structure of language and cognition than about the mechanics of the models. The number of parameters and layers and functions assigned to layers shouldn’t change things, nor going multi-modal, either. Whatever the mechanics of the mechanics of the models, they have to deal with language as it is, and that’s not changing in any appreciable way.
To clarify: do you think in about 5 years we will be able to do such thing to then state of the art big models?
Yes. It’s more about the structure of language and cognition than about the mechanics of the models. The number of parameters and layers and functions assigned to layers shouldn’t change things, nor going multi-modal, either. Whatever the mechanics of the mechanics of the models, they have to deal with language as it is, and that’s not changing in any appreciable way.