Interesting and useful concept, technological leverage.
I’m curious what Googetasoft is?
OK I can see a strong AI algorithm being able to do many things we consider intelligence, and I can see how the technological leverage it would have in our increasingly digital / networked world would be far greater than many previous technologies.
This is the story of all new technological advancements, bigger benefits as well as bigger problems and dangers that need to be addressed or solved or else bigger bad things can happen. There will be no end to these types of problems going forward if we are to continue to progress, and there is no guarantee we can solve them, but there is no law of physics saying we can’t.
The efforts on this front are good, necessary, and should demand our attention, but I think this whole effort isn’t really about AGI.
I guess I don’t understand how scaling up or tweaking the current approach will lead AI’s that are uncontrollable or “run away” from us? I’m actually rather skeptical of this.
I agree regular AI can generate new knowledge but only an AGI will do so creatively and and recognize it as so. I don’t think we are close to creating that kind of AGI yet with the current approach as we don’t really understand how creativity works.
That being said, it can’t be that hard if evolution was able to figure it out.
The unholy spiritual merger of Google, Meta, Microsoft, and all the other large organizations pushing capabilities.
I guess I don’t understand how scaling up or tweaking the current approach will lead AI’s that are uncontrollable or “run away” from us? I’m actually rather skeptical of this.
It’s possible that the current approach (that is, token predicting large language models using transformers like we use them now) won’t go somewhere potentially dangerous, because they won’t be capable enough. It’s hard to make this claim with high certainty, though- GPT-3 already does a huge amount with very little. If Chinchilla was 1,000x larger and trained across 1,000x more data (say, the entirety of youtube), what is it going to be able to do? It wouldn’t be surprising if it could predict a video of two humans sitting down in a restaurant having a conversation. It probably would have a decent model of how newtonian physics works, since everything filmed in the real world would benefit from that understanding. Might it also learn more subtle things? Detailed mental models of humans, because it needs to predict tokens from the slightest quirk of an eyebrow, or a tremor in a person’s voice? How much of chemistry, nuclear physics, or biology could it learn? I don’t know, but I really can’t assign a significant probability to it just failing completely given what we’ve already observed.
Critically, we cannot make assumptions about what it can and can’t learn based on what we think its dataset is about. Consider that GPT-3′s dataset didn’t have a bunch of text about how to predict tokens- it learned to predict tokens because of the loss function. Everything it knows, everything it can do, was learned because it increased the probability that the next predicted token will be correct. If there’s some detail- maybe something about physics, or how humans work- that helps it predict tokens better, we should not just assume that it will be inaccessible to even simple token predictors. Remember, the AI is much, much better than you at predicting tokens, and you’re not doing the same thing it is.
In other words...
I don’t think we are close to creating that kind of AGI yet with the current approach as we don’t really understand how creativity works.
We don’t have a good understanding of how any of this works. We don’t need to have a good understanding of how it works to make it happen, apparently. This is the ridiculous truth of machine learning that’s slapped me in the face several times over the last 5-10 years. And yes, evolution managing to solve it definitely doesn’t give me warm fuzzies about it being hard.
And we’re not even slightly bottlenecked on… anything, really. Transformers and token predictors aren’t the endgame. There are really obvious steps forward, and even tiny changes to how we use existing architectures massively increase capability (just look at prompt engineering, or how Minerva worked, and so on).
Going back to the idea of the AI being uncontrollable- we just don’t know how to do it yet. Token predictors just predict tokens, but even there, we struggle to figure out what the AI can actually do because it’s not “interested” in giving you correct answers. It just predicts tokens. So we get the entire subfield of prompt engineering that tries to elicit its skills and knowledge by… asking it nicely???
(It may seem like a token predictor is safer in some ways, which I’d agree with in principle. The outer behavior of the AI isn’t agentlike. But it can predict tokens associated with agents. And the more capable it is, the more capable the simulated agents are. This is just one trivial example of how an oracle/tool AI can easily get turned into something dangerous.)
An obvious guess might be something like reinforcement learning. Just reward it for doing the things you want, right? Not a bad first stab at the problem… but it doesn’t really work. This isn’t just at theoretical concern- it fails in practice. And we don’t know how to fix it rigorously yet.
It could be that the problem is easy, and there’s a natural basin of safe solution space that AI will fall into as they become more capable. That would be very helpful, since it would mean there are far more paths to good outcomes. But we don’t know if that’s how reality actually works, and the very obvious theoretical and practical failure modes of some architectures (like maximizers) are worrying. I definitely don’t want to bet humanity’s survival on “we happen to live in a reality where the problem is super easy.”
I’d feel a lot better about our chances if anyone ever outlined how we would actually, concretely, do it. So far, every proposal seems either obviously broken, or it relies on reality being on easymode. (Edit: or they’re still being worked on!)
Interesting and useful concept, technological leverage.
I’m curious what Googetasoft is?
OK I can see a strong AI algorithm being able to do many things we consider intelligence, and I can see how the technological leverage it would have in our increasingly digital / networked world would be far greater than many previous technologies.
This is the story of all new technological advancements, bigger benefits as well as bigger problems and dangers that need to be addressed or solved or else bigger bad things can happen. There will be no end to these types of problems going forward if we are to continue to progress, and there is no guarantee we can solve them, but there is no law of physics saying we can’t.
The efforts on this front are good, necessary, and should demand our attention, but I think this whole effort isn’t really about AGI.
I guess I don’t understand how scaling up or tweaking the current approach will lead AI’s that are uncontrollable or “run away” from us? I’m actually rather skeptical of this.
I agree regular AI can generate new knowledge but only an AGI will do so creatively and and recognize it as so. I don’t think we are close to creating that kind of AGI yet with the current approach as we don’t really understand how creativity works.
That being said, it can’t be that hard if evolution was able to figure it out.
The unholy spiritual merger of Google, Meta, Microsoft, and all the other large organizations pushing capabilities.
It’s possible that the current approach (that is, token predicting large language models using transformers like we use them now) won’t go somewhere potentially dangerous, because they won’t be capable enough. It’s hard to make this claim with high certainty, though- GPT-3 already does a huge amount with very little. If Chinchilla was 1,000x larger and trained across 1,000x more data (say, the entirety of youtube), what is it going to be able to do? It wouldn’t be surprising if it could predict a video of two humans sitting down in a restaurant having a conversation. It probably would have a decent model of how newtonian physics works, since everything filmed in the real world would benefit from that understanding. Might it also learn more subtle things? Detailed mental models of humans, because it needs to predict tokens from the slightest quirk of an eyebrow, or a tremor in a person’s voice? How much of chemistry, nuclear physics, or biology could it learn? I don’t know, but I really can’t assign a significant probability to it just failing completely given what we’ve already observed.
Critically, we cannot make assumptions about what it can and can’t learn based on what we think its dataset is about. Consider that GPT-3′s dataset didn’t have a bunch of text about how to predict tokens- it learned to predict tokens because of the loss function. Everything it knows, everything it can do, was learned because it increased the probability that the next predicted token will be correct. If there’s some detail- maybe something about physics, or how humans work- that helps it predict tokens better, we should not just assume that it will be inaccessible to even simple token predictors. Remember, the AI is much, much better than you at predicting tokens, and you’re not doing the same thing it is.
In other words...
We don’t have a good understanding of how any of this works. We don’t need to have a good understanding of how it works to make it happen, apparently. This is the ridiculous truth of machine learning that’s slapped me in the face several times over the last 5-10 years. And yes, evolution managing to solve it definitely doesn’t give me warm fuzzies about it being hard.
And we’re not even slightly bottlenecked on… anything, really. Transformers and token predictors aren’t the endgame. There are really obvious steps forward, and even tiny changes to how we use existing architectures massively increase capability (just look at prompt engineering, or how Minerva worked, and so on).
Going back to the idea of the AI being uncontrollable- we just don’t know how to do it yet. Token predictors just predict tokens, but even there, we struggle to figure out what the AI can actually do because it’s not “interested” in giving you correct answers. It just predicts tokens. So we get the entire subfield of prompt engineering that tries to elicit its skills and knowledge by… asking it nicely???
(It may seem like a token predictor is safer in some ways, which I’d agree with in principle. The outer behavior of the AI isn’t agentlike. But it can predict tokens associated with agents. And the more capable it is, the more capable the simulated agents are. This is just one trivial example of how an oracle/tool AI can easily get turned into something dangerous.)
An obvious guess might be something like reinforcement learning. Just reward it for doing the things you want, right? Not a bad first stab at the problem… but it doesn’t really work. This isn’t just at theoretical concern- it fails in practice. And we don’t know how to fix it rigorously yet.
It could be that the problem is easy, and there’s a natural basin of safe solution space that AI will fall into as they become more capable. That would be very helpful, since it would mean there are far more paths to good outcomes. But we don’t know if that’s how reality actually works, and the very obvious theoretical and practical failure modes of some architectures (like maximizers) are worrying. I definitely don’t want to bet humanity’s survival on “we happen to live in a reality where the problem is super easy.”
I’d feel a lot better about our chances if anyone ever outlined how we would actually, concretely, do it. So far, every proposal seems either obviously broken, or it relies on reality being on easymode. (Edit: or they’re still being worked on!)