EDIT: I now consider this whole approach to be irreparably flawed.
Imagine if their was a 50% chance the button would be pressed. Also the AI was in some betting thing that would pay out paperclips or staples based on a coin flip.
In scenario 1, this AI has an option of setting up a mechanism, external to itself, that controls the coin flip based on if the button is pressed.
In scenario 2, the AI can set up a mechanism to control the button based on the coin flip.
When only considering actions, probabilities and utilities, these look identical. But we want the AI to behave differently. So we must make use of the arrow of time. The structure of cause and effect. In a way we aren’t doing here.
End edit.
To make this work, you would need a utility function for shutting off, including all subagents.
Lets suppose that other than these robots, there are ~0 transistors in your house. Then we can define shutting off as minimizing the number of transistor flips in your house.
So you make
A=at least 1 coffee
B=1/transistor flips.
Note that, when imagining any potential future world where the switch isn’t pressed, the AI has no reason to bother counting transistor flips. And in potential futures where the switch is pressed, it doesn’t need to know what coffee is.
What it cares about are a=E(A(1−K)) and b=E(BK)
Lets list the robot’s options.
Let a=1 without coffee, and a=2 with coffee.
In worlds where the switch is pressed, the robot turns it’s self off immediately once it is fairly sure the switch will be pressed. Making b=1.5 In worlds where it doesn’t turn off, more transistors flip making b=0.5. Because a=1 in both cases, and we only select from the paerito frontier, whenever the switch is pressed, it will turn off.
Lets apply your utility penalties, by putting them in A and B. Ie in a or b depending on the switch.
EDIT: I now consider this whole approach to be irreparably flawed.
Imagine if their was a 50% chance the button would be pressed. Also the AI was in some betting thing that would pay out paperclips or staples based on a coin flip.
In scenario 1, this AI has an option of setting up a mechanism, external to itself, that controls the coin flip based on if the button is pressed.
In scenario 2, the AI can set up a mechanism to control the button based on the coin flip.
When only considering actions, probabilities and utilities, these look identical. But we want the AI to behave differently. So we must make use of the arrow of time. The structure of cause and effect. In a way we aren’t doing here.
End edit.
To make this work, you would need a utility function for shutting off, including all subagents.
Lets suppose that other than these robots, there are ~0 transistors in your house. Then we can define shutting off as minimizing the number of transistor flips in your house.
So you make
A=at least 1 coffee
B=1/transistor flips.
Note that, when imagining any potential future world where the switch isn’t pressed, the AI has no reason to bother counting transistor flips. And in potential futures where the switch is pressed, it doesn’t need to know what coffee is.
What it cares about are a=E(A(1−K)) and b=E(BK)
Lets list the robot’s options.
Let a=1 without coffee, and a=2 with coffee.
In worlds where the switch is pressed, the robot turns it’s self off immediately once it is fairly sure the switch will be pressed. Making b=1.5 In worlds where it doesn’t turn off, more transistors flip making b=0.5. Because a=1 in both cases, and we only select from the paerito frontier, whenever the switch is pressed, it will turn off.
Lets apply your utility penalties, by putting them in A and B. Ie in a or b depending on the switch.
TC) Press switch, avoid cat. Switch pressed. So (a=0.7) . b=1.5−0.1−0.2=1.2,
PC) Prevent switch, avoid cat. a=1.7,(b=0.2)
IC) Ignore switch, avoid cat. a=1.8,(b=0.3)
TH) Press switch, hit cat. b=1.4,(a=0.9)
IH) Ignore switch, hit cat b=1.5,(a=1) (because it predicts humans will see it and turn it off)
PH) Prevent switch, hit cat. a=1.9,(b=0.4)
This puts IH and PH on the convex hull.
And I think my algorithm picks between them stochastically.