I’m talking about the AI’s ability to learn / figure out a new system / idea / domain on the fly. It’s hard to point to a particular “task” that specifically tests this ability (in the way that people normally use the term “task”), because for any possible task, maybe the AI happens to already know how to do it.
You could filter the training data, but doing that in practice might be kinda tricky because “the AI already knows how to do X” is distinct from “the AI has already seen examples of X in the training data”. LLMs “already know how to do” lots of things that are not superficially in the training data, just as humans “already know how to do” lots of things that are superficially unlike anything they’ve seen before—e.g. I can ask a random human to imagine a purple colander falling out of an airplane and answer simple questions about it, and they’ll do it skillfully and instantaneously. That’s the inference algorithm, not the learning algorithm.
Well, getting an AI to invent a new scientific field would work as such a task, because it’s not in the training data by definition. But that’s such a high bar as to be unhelpful in practice. Maybe tasks that we think of as more suited to RL, like low-level robot control, or skillfully playing games that aren’t like anything in the training data?
Separately, I think there are lots of domains where “just generate synthetic data” is not a thing you can do. If an AI doesn’t fully ‘understand’ the physics concept of “superradiance” based on all existing human writing, how would it generate synthetic data to get better? If an AI is making errors in its analysis of the tax code, how would it generate synthetic data to get better? (If you or anyone has a good answer to those questions, maybe you shouldn’t publish them!! :-P )
If an AI doesn’t fully ‘understand’ the physics concept of “superradiance” based on all existing human writing, how would it generate synthetic data to get better?
I think “doesn’t fully understand the concept of superradiance” is a phrase that smuggles in too many assumptions here. If you rephrase it as “can determine when superradiance will occur, but makes inaccurate predictions about physical systems will do in those situations” / “makes imprecise predictions in such cases” / “has trouble distinguishing cases where superradiance will occur vs cases where it will not”, all of those suggest pretty obvious ways of generating training data.
GPT-4 can already “figure out a new system on the fly” in the sense of taking some repeatable phenomenon it can observe, and predicting things about that phenomenon, because it can write standard machine learning pipelines, design APIs with documentation, and interact with documented APIs. However, the process of doing that is very slow and expensive, and resembles “build a tool and then use the tool” rather than “augment its own native intelligence”.
Which makes sense. The story of human capabilities advances doesn’t look like “find clever ways to configure unprocess rocks and branches from the environment in ways which accomplish our goals”, it looks like “build a bunch of tools, and figure out which ones are most useful and how they are best used, and then use our best tools to build better tools, and so on, and then use the much-improved tools to do the things we want”.
I feel quite confident that all the leading AI labs are already thinking and talking internally about this stuff, and that what we are saying here adds approximately nothing to their conversations. So I don’t think it matters whether we discuss this or not. That simply isn’t a lever of control we have over the world.
There are potentially secret things people might know which shouldn’t be divulged, but I doubt this conversation is anywhere near technical enough to be advancing the frontier in any way.
I’m talking about the AI’s ability to learn / figure out a new system / idea / domain on the fly. It’s hard to point to a particular “task” that specifically tests this ability (in the way that people normally use the term “task”), because for any possible task, maybe the AI happens to already know how to do it.
You could filter the training data, but doing that in practice might be kinda tricky because “the AI already knows how to do X” is distinct from “the AI has already seen examples of X in the training data”. LLMs “already know how to do” lots of things that are not superficially in the training data, just as humans “already know how to do” lots of things that are superficially unlike anything they’ve seen before—e.g. I can ask a random human to imagine a purple colander falling out of an airplane and answer simple questions about it, and they’ll do it skillfully and instantaneously. That’s the inference algorithm, not the learning algorithm.
Well, getting an AI to invent a new scientific field would work as such a task, because it’s not in the training data by definition. But that’s such a high bar as to be unhelpful in practice. Maybe tasks that we think of as more suited to RL, like low-level robot control, or skillfully playing games that aren’t like anything in the training data?
Separately, I think there are lots of domains where “just generate synthetic data” is not a thing you can do. If an AI doesn’t fully ‘understand’ the physics concept of “superradiance” based on all existing human writing, how would it generate synthetic data to get better? If an AI is making errors in its analysis of the tax code, how would it generate synthetic data to get better? (If you or anyone has a good answer to those questions, maybe you shouldn’t publish them!! :-P )
I think “doesn’t fully understand the concept of superradiance” is a phrase that smuggles in too many assumptions here. If you rephrase it as “can determine when superradiance will occur, but makes inaccurate predictions about physical systems will do in those situations” / “makes imprecise predictions in such cases” / “has trouble distinguishing cases where superradiance will occur vs cases where it will not”, all of those suggest pretty obvious ways of generating training data.
GPT-4 can already “figure out a new system on the fly” in the sense of taking some repeatable phenomenon it can observe, and predicting things about that phenomenon, because it can write standard machine learning pipelines, design APIs with documentation, and interact with documented APIs. However, the process of doing that is very slow and expensive, and resembles “build a tool and then use the tool” rather than “augment its own native intelligence”.
Which makes sense. The story of human capabilities advances doesn’t look like “find clever ways to configure unprocess rocks and branches from the environment in ways which accomplish our goals”, it looks like “build a bunch of tools, and figure out which ones are most useful and how they are best used, and then use our best tools to build better tools, and so on, and then use the much-improved tools to do the things we want”.
I don’t know how I feel about pushing this conversation further. A lot of people read this forum now.
I feel quite confident that all the leading AI labs are already thinking and talking internally about this stuff, and that what we are saying here adds approximately nothing to their conversations. So I don’t think it matters whether we discuss this or not. That simply isn’t a lever of control we have over the world.
There are potentially secret things people might know which shouldn’t be divulged, but I doubt this conversation is anywhere near technical enough to be advancing the frontier in any way.
Perhaps.