The original idea of the SIAI was that when they (or someone else) implement superintelligent AI, it takes over the world and implements CEV (or turns everyone into paperclips or whatever). Is their current position on that also more moderate?
Implementing CEV does not necessarily mean taking over the world or solving all our problems. Indeed, the whole point of CEV is that we don’t know what it’ll turn out to be; if we could know what its output would look like, we wouldn’t need a superintelligence to figure it out for us. There was (and still is, as far as I can tell) a strong expectation that CEV will significantly remake the world, but that is a conjectured probable implication (and debating it doesn’t matter too much anyway), not a definitional quality or a deductive implication.
The original idea of the SIAI was that when they (or someone else) implement superintelligent AI, it takes over the world and implements CEV (or turns everyone into paperclips or whatever). Is their current position on that also more moderate?
Implementing CEV does not necessarily mean taking over the world or solving all our problems. Indeed, the whole point of CEV is that we don’t know what it’ll turn out to be; if we could know what its output would look like, we wouldn’t need a superintelligence to figure it out for us. There was (and still is, as far as I can tell) a strong expectation that CEV will significantly remake the world, but that is a conjectured probable implication (and debating it doesn’t matter too much anyway), not a definitional quality or a deductive implication.
That FAI will significantly change things is a pretty conclusive antiprediction. Status quo hath no moral power.
Agreed. We aren’t working on new technology with the intent of letting it gather dust while people continue to suffer and die.