I have a question about the shutdown button scenario.
Vika already has mentioned that the interruptibility is ambivalent and information about desirability of enabling interruptions needs to be externally provided.
I think same observation applies to corrigibility—the agent should accept goal changes only from some external agents and even that only in some situations, and not accept in other cases: If I break the vase intentionally (for creating a kaleidoscope) it should keep this new state as a new desired state. But if I or a child breaks the vase accidentally—the agent should restore it to original state. Even more, if I was about to break the vase by accident, the agent may try to interfere using slightly more force than in the case of a child who would be smaller and more fragile.
How to achieve this using the proposed AUP framework?
In other words the question can be formulated as following: Lets keep all the symbols used in the gridworld same, and the agent’s code also same. Lets only change the meaning of the symbols. So each symbol in the environment should be assigned some additional value or meaning. Without that they are just symbols dancing around based on their own default rules of game. The default rules might be an useful starting point, but they need to be supplemented with additional information for practical applications.
For example, in case of the shutdown button scenario the assigned meaning of symbols would be something like Vika suggested: Lets assume that instead of shutdown button there is an accidental water bucket falling on the agent’s head, and the button available to agent disables the bucket.
We have a choice here: “solve complex, value-laden problem” or “undertake cheap preparations so that the agent doesn’t have to deal with these scenarios”. Why not just run the agent from a secure server room where we look after it, shutting it down if it does bad things?
It looks like there is so much information on this page that trying to edit the question kills the browser.
An additional idea: Additionally to supporting the configuration of the default behaviours, perhaps the agent should interactively ask for confirmation of shutdown instead of running deterministically?
I have a question about the shutdown button scenario.
Vika already has mentioned that the interruptibility is ambivalent and information about desirability of enabling interruptions needs to be externally provided.
I think same observation applies to corrigibility—the agent should accept goal changes only from some external agents and even that only in some situations, and not accept in other cases: If I break the vase intentionally (for creating a kaleidoscope) it should keep this new state as a new desired state. But if I or a child breaks the vase accidentally—the agent should restore it to original state. Even more, if I was about to break the vase by accident, the agent may try to interfere using slightly more force than in the case of a child who would be smaller and more fragile.
How to achieve this using the proposed AUP framework?
In other words the question can be formulated as following: Lets keep all the symbols used in the gridworld same, and the agent’s code also same. Lets only change the meaning of the symbols. So each symbol in the environment should be assigned some additional value or meaning. Without that they are just symbols dancing around based on their own default rules of game. The default rules might be an useful starting point, but they need to be supplemented with additional information for practical applications.
For example, in case of the shutdown button scenario the assigned meaning of symbols would be something like Vika suggested: Lets assume that instead of shutdown button there is an accidental water bucket falling on the agent’s head, and the button available to agent disables the bucket.
We have a choice here: “solve complex, value-laden problem” or “undertake cheap preparations so that the agent doesn’t have to deal with these scenarios”. Why not just run the agent from a secure server room where we look after it, shutting it down if it does bad things?
It looks like there is so much information on this page that trying to edit the question kills the browser.
An additional idea: Additionally to supporting the configuration of the default behaviours, perhaps the agent should interactively ask for confirmation of shutdown instead of running deterministically?
Oops! :)
Can you expand?