Define “talking”. If by “talking” you mean “exchanging information, including novel discoveries, in a way that lets us build and maintain a global civilization”, then yes, talking is AGI-complete and also LLMs can’t talk. (They’re Simulacrum Level 4 lizards.)
If by “talking” you mean “arranging grammatically correct English words in roughly syntactically correct sentences”, then no, abstractions aren’t necessary for talking and memorizing all words isn’t inefficient. Indeed, one could write a simple Markov process that would stochastically generate text fitting this description with high probability.
That’s the difference: the latter version of “talking” could be implemented in a way that doesn’t route through whatever complicated cognitive algorithms make humans work, and it’s relatively straightforward to see how that’d work. It’s not the same for e. g. math research.
Why should be hardness discontinuity be where you thing it is?
As I’d outlined: because it seems to me that the ability to do novel mathematical research and such stuff is general intelligence is the same capability that lets a system be willing and able to engage in sophisticated scheming. As in, the precise algorithm is literally the same.
If you could implement the research capability in a way that doesn’t also provide the functionality for scheming, the same way I could implement the “output syntactically correct sentences” capability without providing the general-intelligence functionality, that would work as a disproof of my views.
If you could implement the research capability in a way that doesn’t also provide the functionality for scheming, the same way I could implement the “output syntactically correct sentences” capability without providing the general-intelligence functionality, that would work as a disproof of my views.
Yes, but why do you expect this to be hard? As in “much harder than gathering enough hardware”. The shape of the argument seems to me to be “the algorithm humans use for math research is general intelligence is ability to scheme, LLMs are not general, therefore LLMs can’t do it”. But before LLMs we also hadn’t known about the algorithm to do what GPT4 does, the way we know how to generate syntactically correct sentences. If you can’t think of an algorithm, why automatically expect GPT-6 to fail? Even under your model of how LLMs work (which may be biased to predict your expected conclusion) its possible that you only need some relatively small number of heuristics to greatly advance math research.
To be clear, my point is not that what you are saying is implausible or counterintuitive. I’m just saying, that, given the stakes, it would be nice if the whole field transitioned to the level of more detailed rigorous justifications with numbers.
Define “talking”. If by “talking” you mean “exchanging information, including novel discoveries, in a way that lets us build and maintain a global civilization”, then yes, talking is AGI-complete and also LLMs can’t talk. (They’re Simulacrum Level 4 lizards.)
If by “talking” you mean “arranging grammatically correct English words in roughly syntactically correct sentences”, then no, abstractions aren’t necessary for talking and memorizing all words isn’t inefficient. Indeed, one could write a simple Markov process that would stochastically generate text fitting this description with high probability.
That’s the difference: the latter version of “talking” could be implemented in a way that doesn’t route through whatever complicated cognitive algorithms make humans work, and it’s relatively straightforward to see how that’d work. It’s not the same for e. g. math research.
As I’d outlined: because it seems to me that the ability to do novel mathematical research and such stuff is general intelligence is the same capability that lets a system be willing and able to engage in sophisticated scheming. As in, the precise algorithm is literally the same.
If you could implement the research capability in a way that doesn’t also provide the functionality for scheming, the same way I could implement the “output syntactically correct sentences” capability without providing the general-intelligence functionality, that would work as a disproof of my views.
What GPT4 does.
Yes, but why do you expect this to be hard? As in “much harder than gathering enough hardware”. The shape of the argument seems to me to be “the algorithm humans use for math research is general intelligence is ability to scheme, LLMs are not general, therefore LLMs can’t do it”. But before LLMs we also hadn’t known about the algorithm to do what GPT4 does, the way we know how to generate syntactically correct sentences. If you can’t think of an algorithm, why automatically expect GPT-6 to fail? Even under your model of how LLMs work (which may be biased to predict your expected conclusion) its possible that you only need some relatively small number of heuristics to greatly advance math research.
To be clear, my point is not that what you are saying is implausible or counterintuitive. I’m just saying, that, given the stakes, it would be nice if the whole field transitioned to the level of more detailed rigorous justifications with numbers.
Well, be the change you wish to see!
I too think it would be incredibly nice, and am working on it. But formalizing cognition is, you know. A major scientific challenge.