C need not be a low constant, btw. The only requirement is that u(false,action a, A) = u(false, action b, A) for all actions a and b and all A. ie nothing the AI does affects the utility of worlds where w is false, so this does not constrain its actions.
Basically the AI observes the ON signal going through, and knows that either a) the signal went through normally, or b) the signal was overwritten by coincidence by exactly the same signal. It’s actions have no consequences in the first case, so it ignores it, and acts “as if” it was certain there had been a thermodynamic miracle that happened.
Thanks. I understand now. Just needed to sleep on it, and today, your explanation makes sense.
Basically, the AI’s actions don’t matter if the unlikely event doesn’t happen, so it will take whatever actions would maximize its utility if the event did happen. This maximizes expected utility
Maximizing [P(no TM) C + P(TM) u(TM, A))] is the same as maximizing u(A) under assumption TM.
C need not be a low constant, btw. The only requirement is that u(false,action a, A) = u(false, action b, A) for all actions a and b and all A. ie nothing the AI does affects the utility of worlds where w is false, so this does not constrain its actions.
Basically the AI observes the ON signal going through, and knows that either a) the signal went through normally, or b) the signal was overwritten by coincidence by exactly the same signal. It’s actions have no consequences in the first case, so it ignores it, and acts “as if” it was certain there had been a thermodynamic miracle that happened.
Thanks. I understand now. Just needed to sleep on it, and today, your explanation makes sense.
Basically, the AI’s actions don’t matter if the unlikely event doesn’t happen, so it will take whatever actions would maximize its utility if the event did happen. This maximizes expected utility
Maximizing [P(no TM) C + P(TM) u(TM, A))] is the same as maximizing u(A) under assumption TM.
Yes, that’s a clear way of phrasing it.