150,000 people die every day. That’s not a small price for any delays to AGI development. Now, we need to do this right: AGI without alignment just kills everyone; it doesn’t solve anything. But the faster we get aligned AI, the better. And trying to slow down capabilities research without much thought into the endgame seems remarkably callous.
Eliezer has mentioned the idea of trying to invent a new paradigm for AI, outside of the conventional neural net/backpropagation model. The context was more “what would you do with unlimited time and money” than “what do you intend irl”, but this seems to be his ideal play. Now, I wish him the best of luck with the endeavor if he tries it, but do we have any evidence that another paradigm is possible?
Evolved minds use something remarkably close to the backprop model, and the only other model we’ve seen work is highly mechanistic AI like Deep Blue. The Deep Blue model doesn’t generalize well, nor is it capable of much creativity. A priori, it seems somewhat unlikely that any other AI paradigm exists: why would math just happen to permit it? And if we oppose capabilities research until we find something like a new AI model, there’s a good chance that we oppose it all the way to the singularity, rather than ever contributing to a Friendly system. That’s not an outcome anyone wants, and it seems to be the default outcome of incautious pessimism.
Another general way to look at it is think about what a policy IS.
A policy is the set of rules any AI system uses to derive the input from the output. It’s how a human or humanoid robot walks, or talks, or any intelligent act. (Since there are rules that update the rules)
Well any non trivial task you realize the policy HAS to account for thousands of variables, including intermediates generated during the policy calculation. It trivially, for any “competitive” policy that does complex things, can exceed the complexity that a human being can grok.
So no matter the method you use to generate a policy it will exceed your ability to review it, and COULD contain “if condition do bad thing” in it.
150,000 people die every day. That’s not a small price for any delays to AGI development. Now, we need to do this right: AGI without alignment just kills everyone; it doesn’t solve anything. But the faster we get aligned AI, the better. And trying to slow down capabilities research without much thought into the endgame seems remarkably callous.
Eliezer has mentioned the idea of trying to invent a new paradigm for AI, outside of the conventional neural net/backpropagation model. The context was more “what would you do with unlimited time and money” than “what do you intend irl”, but this seems to be his ideal play. Now, I wish him the best of luck with the endeavor if he tries it, but do we have any evidence that another paradigm is possible?
Evolved minds use something remarkably close to the backprop model, and the only other model we’ve seen work is highly mechanistic AI like Deep Blue. The Deep Blue model doesn’t generalize well, nor is it capable of much creativity. A priori, it seems somewhat unlikely that any other AI paradigm exists: why would math just happen to permit it? And if we oppose capabilities research until we find something like a new AI model, there’s a good chance that we oppose it all the way to the singularity, rather than ever contributing to a Friendly system. That’s not an outcome anyone wants, and it seems to be the default outcome of incautious pessimism.
Another general way to look at it is think about what a policy IS.
A policy is the set of rules any AI system uses to derive the input from the output. It’s how a human or humanoid robot walks, or talks, or any intelligent act. (Since there are rules that update the rules)
Well any non trivial task you realize the policy HAS to account for thousands of variables, including intermediates generated during the policy calculation. It trivially, for any “competitive” policy that does complex things, can exceed the complexity that a human being can grok.
So no matter the method you use to generate a policy it will exceed your ability to review it, and COULD contain “if condition do bad thing” in it.