EY seems to have interpreted AlphaGo Zero as strong evidence for his view in the AI-foom debate
I don’t think CAIS takes much of a position on the AI-foom debate. CAIS seems entirely compatible with very fast progress in AI.
I don’t think CAIS would anti-predict AlphaGo Zero, though plausibly it doesn’t predict as strongly as EY’s position does.
conditional on there being breakthroughs involving some domain-specific hacks, and major labs keep working on them, they will somewhat quickly superseded by breakthroughs with general-seming architectures
This is a prediction I make, with “general-seeming” replaced by “more general”, and I think of this as a prediction inspired much more by CAIS than by EY/Bostrom.
The equivalent of the “foom scenario” for CAIS would be rapidly improving basic AI capabilities due to automated AI R&D services, such that the aggregate “soup of services” is quickly able to do more and more complex tasks with constantly improving performance. If you look at the “soup” as an aggregate, this looks like a thing that is quickly becoming superintelligent by self-improving.
The main difference from the classical AI foom scenario is that the thing that’s improving cannot easily be modeled as pursuing a single goal. Also, there are more safety affordances: there can still be humans in the loop for services that have large real world consequences, you can monitor the interactions between services to make sure they aren’t doing anything unexpected, etc.
This is a prediction I make, with “general-seeming” replaced by “more general”, and I think of this as a prediction inspired much more by CAIS than by EY/Bostrom.
I notice I’m confused. My model of CAIS predicts that there would be poor returns to building general services compared to specialised ones (though this might be more of a claim about economics than a claim about the nature of intelligence).
My model of CAIS predicts that there would be poor returns to building general services compared to specialised ones
Depends what you mean by “general”. If you mean that there would be poor returns to building an AGI that has a broad understanding of the world that you then ask to always perform surgery, I agree that that’s not going to be as good as creating a system that is specialized for surgeries. If you mean that there would be poor returns to building a machine translation system that uses end-to-end trained neural nets, I can just point to Google Translate using those neural nets instead of more specialized systems that built parse trees before translating. When you say “domain-specific hacks”, I think much more of the latter than the former.
Another way of putting it is that CAIS says that there are poor returns to building task-general AI systems, but does not say that there are poor returns to building general AI building blocks. In fact, I think CAIS says that you really do make very general AI building blocks—the premise of recursive technological improvement is that AI systems can autonomously perform AI R&D which makes better AI building blocks which makes all of the other services better.
All of that said, Eric and I probably do disagree on how important generality is, though I’m not sure exactly what the disagreement is, so to the extent that you’re trying to use Eric’s conception of CAIS you might want to downweight these particular beliefs of mine.
I don’t think CAIS takes much of a position on the AI-foom debate. CAIS seems entirely compatible with very fast progress in AI.
I don’t think CAIS would anti-predict AlphaGo Zero, though plausibly it doesn’t predict as strongly as EY’s position does.
This is a prediction I make, with “general-seeming” replaced by “more general”, and I think of this as a prediction inspired much more by CAIS than by EY/Bostrom.
Isn’t the “foom scenario” referring to an individual AI that quickly gains ASI status by self-improving?
The equivalent of the “foom scenario” for CAIS would be rapidly improving basic AI capabilities due to automated AI R&D services, such that the aggregate “soup of services” is quickly able to do more and more complex tasks with constantly improving performance. If you look at the “soup” as an aggregate, this looks like a thing that is quickly becoming superintelligent by self-improving.
The main difference from the classical AI foom scenario is that the thing that’s improving cannot easily be modeled as pursuing a single goal. Also, there are more safety affordances: there can still be humans in the loop for services that have large real world consequences, you can monitor the interactions between services to make sure they aren’t doing anything unexpected, etc.
I notice I’m confused. My model of CAIS predicts that there would be poor returns to building general services compared to specialised ones (though this might be more of a claim about economics than a claim about the nature of intelligence).
Depends what you mean by “general”. If you mean that there would be poor returns to building an AGI that has a broad understanding of the world that you then ask to always perform surgery, I agree that that’s not going to be as good as creating a system that is specialized for surgeries. If you mean that there would be poor returns to building a machine translation system that uses end-to-end trained neural nets, I can just point to Google Translate using those neural nets instead of more specialized systems that built parse trees before translating. When you say “domain-specific hacks”, I think much more of the latter than the former.
Another way of putting it is that CAIS says that there are poor returns to building task-general AI systems, but does not say that there are poor returns to building general AI building blocks. In fact, I think CAIS says that you really do make very general AI building blocks—the premise of recursive technological improvement is that AI systems can autonomously perform AI R&D which makes better AI building blocks which makes all of the other services better.
All of that said, Eric and I probably do disagree on how important generality is, though I’m not sure exactly what the disagreement is, so to the extent that you’re trying to use Eric’s conception of CAIS you might want to downweight these particular beliefs of mine.