Verification seems like a strictly simpler problem. If we can’t prove properties for a web server, how are we going to do anything about a completely unspecified AI?
The AI take over scenarios I’ve head almost always involve some kind of hacking, because today hacking is easy. I don’t see why that would necessarily be the case a decade from now. We could prove some operating system security guarantees for instance.
Yes, verification is a strictly simpler problem, and one that’s fairly thoroughly addressed by existing research—which is why people working specifically on AI safety are paying attention to other things.
(Maybe they should actually be working on doing verification better first, but that doesn’t seem obviously a superior strategy.)
Some AI takeover scenarios involve hacking (by the AI, of other systems). We might hope to make AI safer by making that harder, but that would require securing all the other important computer systems in the world. Even though making an AI safe is really hard, it may well be easier than that.
Yes, verification is a strictly simpler problem, and one that’s fairly thoroughly addressed by existing research—which is why people working specifically on AI safety are paying attention to other things.
This doesn’t really seem true to me. We are currently pretty bad at software verification, only able to deal with either fairly simple properties or fairly simple programs. I also think that people in verification do care about the “specification problem”, which is roughly problem 2 above (although I don’t think anyone really has that many ideas for how to address it).
Verification seems like a strictly simpler problem. If we can’t prove properties for a web server, how are we going to do anything about a completely unspecified AI?
The AI take over scenarios I’ve head almost always involve some kind of hacking, because today hacking is easy. I don’t see why that would necessarily be the case a decade from now. We could prove some operating system security guarantees for instance.
Yes, verification is a strictly simpler problem, and one that’s fairly thoroughly addressed by existing research—which is why people working specifically on AI safety are paying attention to other things.
(Maybe they should actually be working on doing verification better first, but that doesn’t seem obviously a superior strategy.)
Some AI takeover scenarios involve hacking (by the AI, of other systems). We might hope to make AI safer by making that harder, but that would require securing all the other important computer systems in the world. Even though making an AI safe is really hard, it may well be easier than that.
This doesn’t really seem true to me. We are currently pretty bad at software verification, only able to deal with either fairly simple properties or fairly simple programs. I also think that people in verification do care about the “specification problem”, which is roughly problem 2 above (although I don’t think anyone really has that many ideas for how to address it).
I would be somewhat more convinced that MIRI was up to it’s mission if they could contribute to much simpler problems in prerequisite fields.