My-Moral-Philosophy is either as true as 2+2=4 or as true as 2+2=5, I’m not sure. or 0.0001*1>0.
If it is wrong, then it’s still decent as philosophy goes, and I just won’t try to use math to talk about it. Though I’d probably think more about another system I looked at, because it seems like more fun.
But just because it’s what a primate wants doesn’t mean it’s the right answer.
@Ian C and Tiiba:
Doing nothing or picking randomly are also choices, you would need a reason for them to be the correct rational choice. ‘Doing nothing’ in particular is the kind of thing we would design into an agent as a safe default, but ‘set all motors to 0’ is as much a choice as ‘set all motors to 1’. Doing at random is no more correct than doing each potential option sequentially.
Elizer has us suppose he proved it, but if you were to experience such a situation, what is the probability that he tricked you into accepting a faulty proof, or that you are suffering some other cognitive failure?
To me, that leaves a nonzero probability of some utility.
In general, I’d go back to being an amoralist.
My-Moral-Philosophy is either as true as 2+2=4 or as true as 2+2=5, I’m not sure. or 0.0001*1>0.
If it is wrong, then it’s still decent as philosophy goes, and I just won’t try to use math to talk about it. Though I’d probably think more about another system I looked at, because it seems like more fun.
But just because it’s what a primate wants doesn’t mean it’s the right answer.
@Ian C and Tiiba: Doing nothing or picking randomly are also choices, you would need a reason for them to be the correct rational choice. ‘Doing nothing’ in particular is the kind of thing we would design into an agent as a safe default, but ‘set all motors to 0’ is as much a choice as ‘set all motors to 1’. Doing at random is no more correct than doing each potential option sequentially.
Elizer has us suppose he proved it, but if you were to experience such a situation, what is the probability that he tricked you into accepting a faulty proof, or that you are suffering some other cognitive failure?
To me, that leaves a nonzero probability of some utility.