Charlie’s quote is an excellent description of an important crux/challenge of getting useful difficult intellectual work out of GPTs.
Despite this, I think it’s possible in principle to train a GPT-like model to AGI or to solve problems at least as hard as humans can solve, for a combination of reasons:
I think it’s likely that GPTs implicitly perform search internally, to some extent, and will be able to perform more sophisticated search with scale.
It seems possible that a sufficiently powerful GPT trained on a massive corpus of human (medical + other) knowledge will learn better/more general abstractions than humans, so that in its ontology “a cure for Alzheimer’s” is an “intuitive” inference away, even if for humans it would require many logical steps and empirical research. I tend to think human knowledge implies a lot of low hanging fruit that we have not accessed because of insufficient exploration and because we haven’t compiled our data into the right abstractions. I don’t know how difficult a cure for Alzheimer’s is, and how close it is to being “implied” by the sum of human knowledge. Nor the solution to alignment. And eliciting this latent knowledge is another problem.
Of course, the models can do explicit search in simulated chains of thought. And if natural language in the wild doesn’t capture/imply the (right granularity of; right directed flow of evidence of) the search process that would be useful for attacking a given problem, it is still possible to record or construct data that does.
But it’s possible that the technical difficulties involved make SSL uncompetitive compared to other methods.
I also responded to Capybasilisk below, but I want to chime in here and use your own post against you, contra point 2 :P
It’s not so easy to get “latent knowledge” out of a simulator—it’s the simulands who have the knowledge, and they have to be somehow specified before you can step forward the simulation of them. When you get a text model to output a cure for Alzheimer’s in one step, without playing out the text of some chain of thought, it’s still simulating something to produce that output, and that something might be an optimization process that is going to find lots of unexpected and dangerous solutions to questions you might ask it.
Figuring out the alignment properties of simulated entities running in the “text laws of physics” seems like a challenge. Not an insurmountable challenge, maybe, and I’m curious about your current and future thoughts, but the sort of thing I want to see progress in before I put too much trust in attempts to use simulators to do superhuman abstraction-building.
If I was trying to have a human researcher cure Alzheimers, I’d give them a laboratory, lab assistants, a notebook, and likely also a computer. Similarly, if I wanted a simulacrum of a human researcher (or a great many simulacra of human researchers) to have a good chance of solving Alzheimer’s, I’d given them access to functionally equivalent resources, facilities and tools, crucially including the ability to design, carry out, and analyze the results of experiments in the real world.
Charlie’s quote is an excellent description of an important crux/challenge of getting useful difficult intellectual work out of GPTs.
Despite this, I think it’s possible in principle to train a GPT-like model to AGI or to solve problems at least as hard as humans can solve, for a combination of reasons:
I think it’s likely that GPTs implicitly perform search internally, to some extent, and will be able to perform more sophisticated search with scale.
It seems possible that a sufficiently powerful GPT trained on a massive corpus of human (medical + other) knowledge will learn better/more general abstractions than humans, so that in its ontology “a cure for Alzheimer’s” is an “intuitive” inference away, even if for humans it would require many logical steps and empirical research. I tend to think human knowledge implies a lot of low hanging fruit that we have not accessed because of insufficient exploration and because we haven’t compiled our data into the right abstractions. I don’t know how difficult a cure for Alzheimer’s is, and how close it is to being “implied” by the sum of human knowledge. Nor the solution to alignment. And eliciting this latent knowledge is another problem.
Of course, the models can do explicit search in simulated chains of thought. And if natural language in the wild doesn’t capture/imply the (right granularity of; right directed flow of evidence of) the search process that would be useful for attacking a given problem, it is still possible to record or construct data that does.
But it’s possible that the technical difficulties involved make SSL uncompetitive compared to other methods.
I also responded to Capybasilisk below, but I want to chime in here and use your own post against you, contra point 2 :P
It’s not so easy to get “latent knowledge” out of a simulator—it’s the simulands who have the knowledge, and they have to be somehow specified before you can step forward the simulation of them. When you get a text model to output a cure for Alzheimer’s in one step, without playing out the text of some chain of thought, it’s still simulating something to produce that output, and that something might be an optimization process that is going to find lots of unexpected and dangerous solutions to questions you might ask it.
Figuring out the alignment properties of simulated entities running in the “text laws of physics” seems like a challenge. Not an insurmountable challenge, maybe, and I’m curious about your current and future thoughts, but the sort of thing I want to see progress in before I put too much trust in attempts to use simulators to do superhuman abstraction-building.
If I was trying to have a human researcher cure Alzheimers, I’d give them a laboratory, lab assistants, a notebook, and likely also a computer. Similarly, if I wanted a simulacrum of a human researcher (or a great many simulacra of human researchers) to have a good chance of solving Alzheimer’s, I’d given them access to functionally equivalent resources, facilities and tools, crucially including the ability to design, carry out, and analyze the results of experiments in the real world.