If it’s important to me that my children have food, and my reward function is such that I get 1 unit of reward for 1 unit of fed-child, and you give me the ability to edit my reward function so I get N units instead, I don’t automatically do it.
Is your reward function the warm glow you feel when your child is fed? (A parent choosing to ramp this up would be analogous to a parent in real life choosing to take a drug that feels great with no consequences in response to their kid eating a meal. This would indeed be a strange thing to do. Maybe a parent would agree to the arrangement as the only way of obtaining that drug..)
Or is your reward function the health and well-being of your child, which is the reason you wanted them to eat in the first place. In which case, parents would certainly do what they could to ramp that up.
(My question might be leading in the direction of SRStarin’s comment, I’m not sure.)
If it’s important to me that my children have food, I will take the steps I think will lead to my children being fed.
My reward function in this case is whatever structures in my mind reinforce the taking of actions that are associated in certain ways with the structures that represent my children having food. Maybe there’s a subjective component to that (“warm glow”), maybe there isn’t.
A sufficiently advanced neuroscience allows me to point to structures in my own brain and say “Ah, see? That is where my preference for my children to have food is computed, that is where my belief that earning a salary increases the chances my children have food is computed, that is where my increased inclination to earn a salary is computed,” and so on and so forth. That is, it lets me identify the neural substrate(s) of my utility function(s).
So Omega hands me the appropriately advanced neuroscience and there I am, standing in front of the console that controls the appropriate machinery, knowing full well that the only reason I care about my child being fed is those circuits I’m seeing on the screen—that, for example, if an accidental brain lesion were to disrupt those circuits, I would no longer care whether my child were fed or not.
Omega’s gadget also allows me to edit those structures so that I no longer care about whether my child is fed. There’s the button right there. Do I press it?
Is your reward function the warm glow you feel when your child is fed? (A parent choosing to ramp this up would be analogous to a parent in real life choosing to take a drug that feels great with no consequences in response to their kid eating a meal. This would indeed be a strange thing to do. Maybe a parent would agree to the arrangement as the only way of obtaining that drug..)
Or is your reward function the health and well-being of your child, which is the reason you wanted them to eat in the first place. In which case, parents would certainly do what they could to ramp that up.
(My question might be leading in the direction of SRStarin’s comment, I’m not sure.)
If it’s important to me that my children have food, I will take the steps I think will lead to my children being fed.
My reward function in this case is whatever structures in my mind reinforce the taking of actions that are associated in certain ways with the structures that represent my children having food. Maybe there’s a subjective component to that (“warm glow”), maybe there isn’t.
A sufficiently advanced neuroscience allows me to point to structures in my own brain and say “Ah, see? That is where my preference for my children to have food is computed, that is where my belief that earning a salary increases the chances my children have food is computed, that is where my increased inclination to earn a salary is computed,” and so on and so forth. That is, it lets me identify the neural substrate(s) of my utility function(s).
So Omega hands me the appropriately advanced neuroscience and there I am, standing in front of the console that controls the appropriate machinery, knowing full well that the only reason I care about my child being fed is those circuits I’m seeing on the screen—that, for example, if an accidental brain lesion were to disrupt those circuits, I would no longer care whether my child were fed or not.
Omega’s gadget also allows me to edit those structures so that I no longer care about whether my child is fed. There’s the button right there. Do I press it?
I can’t see why I would.
Would you?