I am confused about why people are building systems in the current machine learning paradigm and trying to make them more and more capable, without realizing that this can be dangerous. I basically think the arguments that Eliezer is making seem likely and should be taken seriously, but I expect most of the people working on bleeding edge systems don’t even know these arguments.
For example, the argument that if you have a training process that trains a system to perform well on a text prediction task, then that doesn’t necessarily mean that the resulting system will “just do text prediction”. It seems quite likely to me that, as Eliezer says, intelligence is just a useful thing to have in order to perform better on the task of predicting text from the Internet. Therefore, at some point, as the systems become more and more capable, we should expect that through this optimization pressure, general intelligence will arise even for a task that seems as energetic as predicting text.
How much permission do AI developers need to get from society before irrevocably changing society?
Right now, to me it seems, like people are steering straight towards the doom. And nobody really ever approved this. But the problem is that most people, even the people doing this, don’t realize that that’s what they’re doing. At least that’s how it seems from my perspective.
Does progress always demand heterodox strategies?
I found it weird that you thought it would be weird if we got continuous learning systems. Because it seems very likely to me that if we get really capable systems at some point, will do active learning. Clearly, gradient descent is a pretty dumb optimization process that you can improve upon. Maybe we can get to the point without continuous learning where the systems improve themselves. This could then actually also be seen as a form of active learning. But at that point we the systems can improve themselves better than humans can are probably dead very very quickly.
Related to this, the thing I am working on is trying to figure out how we can do learning without using SGD. The hope is that if we find an algorithm that can learn, which we can just write down explicitly and understand, then that would make this algorithm pretty straightforward to align, especially if during the design process of the algorithm you build it such that it would be easy to align already.
Here is a response I wrote to the Import AI 337
I am confused about why people are building systems in the current machine learning paradigm and trying to make them more and more capable, without realizing that this can be dangerous. I basically think the arguments that Eliezer is making seem likely and should be taken seriously, but I expect most of the people working on bleeding edge systems don’t even know these arguments.
For example, the argument that if you have a training process that trains a system to perform well on a text prediction task, then that doesn’t necessarily mean that the resulting system will “just do text prediction”. It seems quite likely to me that, as Eliezer says, intelligence is just a useful thing to have in order to perform better on the task of predicting text from the Internet. Therefore, at some point, as the systems become more and more capable, we should expect that through this optimization pressure, general intelligence will arise even for a task that seems as energetic as predicting text.
Right now, to me it seems, like people are steering straight towards the doom. And nobody really ever approved this. But the problem is that most people, even the people doing this, don’t realize that that’s what they’re doing. At least that’s how it seems from my perspective.
I found it weird that you thought it would be weird if we got continuous learning systems. Because it seems very likely to me that if we get really capable systems at some point, will do active learning. Clearly, gradient descent is a pretty dumb optimization process that you can improve upon. Maybe we can get to the point without continuous learning where the systems improve themselves. This could then actually also be seen as a form of active learning. But at that point we the systems can improve themselves better than humans can are probably dead very very quickly.
Related to this, the thing I am working on is trying to figure out how we can do learning without using SGD. The hope is that if we find an algorithm that can learn, which we can just write down explicitly and understand, then that would make this algorithm pretty straightforward to align, especially if during the design process of the algorithm you build it such that it would be easy to align already.
We cannot follow that link into Gmail unless you give us your Gmail username and password.
LOL, what a dumb mistake. Fixed. Thanks.
Your link to Import AI 337 currently links to the email, it should be this: https://importai.substack.com/p/import-ai-337-why-i-am-confused-about