What is the domain of U? What inputs does it take? In your papers you take a generic Markov Decision Process, but which one will you use here? How exactly do you model the real world? What is the set of states and the set of actions? Does the set of states include the internal state of the AI?
You may have been referring to this as “4. Issues of ontology”, but I don’t think the problem can be separated from your agenda. I don’t see how any progress can be made without answering these questions. Maybe your can start with naive answers, and to move on to something more realistic later. If so I’m interested in what those naive world models look like. And I’m suspicious of how well human preferences would translate onto such models.
Other AI construction methods could claim that the AI will learn the optimal world model, by interacting with the world, but I don’t think this solution can work for your agenda, since the U function is fixed from the start.
What is the domain of U? What inputs does it take? In your papers you take a generic Markov Decision Process, but which one will you use here? How exactly do you model the real world? What is the set of states and the set of actions? Does the set of states include the internal state of the AI?
You may have been referring to this as “4. Issues of ontology”, but I don’t think the problem can be separated from your agenda. I don’t see how any progress can be made without answering these questions. Maybe your can start with naive answers, and to move on to something more realistic later. If so I’m interested in what those naive world models look like. And I’m suspicious of how well human preferences would translate onto such models.
Other AI construction methods could claim that the AI will learn the optimal world model, by interacting with the world, but I don’t think this solution can work for your agenda, since the U function is fixed from the start.