But what if your AI wasn’t programmed to model the entire world, just some subset of it, and had restrictions in place to preserve this? Would it be possible to write a safe, recursively self-improving chess-playing AI, for instance? (You could call this approach “restricting the AI’s ontology”.)
Why would this work any better (or worse) than an oracle AI?
Presumably an Oracle AI’s ontology would not be restricted because it’s trying to model the entire world.
Obviously we don’t particularly need an AI to play chess. It’s possible that we’d want this for some other domain, though, perhaps especially for one that has some relevance for FAI, or as a self-improving AI prototype. I also think it’s interesting as a thought experiment. I don’t understand the reasons why SI is so focused on the FAI approach, and I figure by asking questions like that one maybe I can learn more about their views.
Why would this work any better (or worse) than an oracle AI?
Presumably an Oracle AI’s ontology would not be restricted because it’s trying to model the entire world.
Obviously we don’t particularly need an AI to play chess. It’s possible that we’d want this for some other domain, though, perhaps especially for one that has some relevance for FAI, or as a self-improving AI prototype. I also think it’s interesting as a thought experiment. I don’t understand the reasons why SI is so focused on the FAI approach, and I figure by asking questions like that one maybe I can learn more about their views.
Well, yes, by definition. But that’s not an answer to my question.
I don’t know which would approach would be more easily formalized and proven to be safe.