That’s right. What I mainly have in mind is a vector of Q-learned values V and a scalarization function that combines them in some (probably non-linear) way. Note that in our technical work, the combination occurs during action selection, not during reward assignment and learning.
I guess whether one calls this “multi-objective RL” is semantic. Because objectives are combined during action selection, not during learning itself, I would not call it “single objective RL with a complicated objective”. If you combined objectives during reward, then I could call it that.
re: your example of real-time control during hunger, I think yours is a pretty reasonable model. I haven’t thought about homeostatic processes in this project (my upcoming paper is all about them!). Definitely am not suggesting that our particular implementation of “MORL” (if we can call it that) is the only or even the best sort of MORL. I’m just trying to get started on understanding it! I really like the way you put it. It makes me think that perhaps the brain is a sort of multi-objective decision-making system with no single combinatory mechanism at all except for the emergent winner of whatever kind of output happens in a particular context—that could plausibly be different depending on whether an action is moving limbs, talking, or mentally setting an intention for a long term plan.
That’s right. What I mainly have in mind is a vector of Q-learned values V and a scalarization function that combines them in some (probably non-linear) way. Note that in our technical work, the combination occurs during action selection, not during reward assignment and learning.
I guess whether one calls this “multi-objective RL” is semantic. Because objectives are combined during action selection, not during learning itself, I would not call it “single objective RL with a complicated objective”. If you combined objectives during reward, then I could call it that.
re: your example of real-time control during hunger, I think yours is a pretty reasonable model. I haven’t thought about homeostatic processes in this project (my upcoming paper is all about them!). Definitely am not suggesting that our particular implementation of “MORL” (if we can call it that) is the only or even the best sort of MORL. I’m just trying to get started on understanding it! I really like the way you put it. It makes me think that perhaps the brain is a sort of multi-objective decision-making system with no single combinatory mechanism at all except for the emergent winner of whatever kind of output happens in a particular context—that could plausibly be different depending on whether an action is moving limbs, talking, or mentally setting an intention for a long term plan.