You make it sound like it wants things. It could at most pretend to be something that wants things. If there’s a UFAI in there that is carefully managing its bits of anonymity (which sounds as unlikely as your usual conspiracy theory—a myopic neural net of this level should keep a secret no better than a conspiracy of a thousand people), it’s going to have better opportunities to influence the world soon enough.
(joke was more about the general principle of putting opaque AIs in charge of alignment ontology, even if this one obviously wasn’t going to be adversarial about it)
You make it sound like it wants things. It could at most pretend to be something that wants things. If there’s a UFAI in there that is carefully managing its bits of anonymity (which sounds as unlikely as your usual conspiracy theory—a myopic neural net of this level should keep a secret no better than a conspiracy of a thousand people), it’s going to have better opportunities to influence the world soon enough.
Sorry, to be clear this was a joke.
(joke was more about the general principle of putting opaque AIs in charge of alignment ontology, even if this one obviously wasn’t going to be adversarial about it)
I think the concern is more “it wouldn’t optimize the ontology carefully”.