The fate of a maximiser depends a great deal on its strength relative to other maximisers. It’s utility function is not the only issue—and maximisers with any utility function can easily be eaten by other, more powerful maximisers.
If you look at biology, replicators have survived so far for billions of years with other utility functions. Do you really think biology is “ensuring continued co-existence”—rather than doing the things described in my references? If so, why do you think that? - the view doesn’t seem to make any sense.
Yes Tim—as I pointed out earlier however, under reasonable assumptions an AI will upon self reflection on the circumstances leading to its existence as well as its utility function conclude that a strictly literal interpretation of its utility function would have to be against the implicit wishes of its originator.
The fate of a maximiser depends a great deal on its strength relative to other maximisers. It’s utility function is not the only issue—and maximisers with any utility function can easily be eaten by other, more powerful maximisers.
If you look at biology, replicators have survived so far for billions of years with other utility functions. Do you really think biology is “ensuring continued co-existence”—rather than doing the things described in my references? If so, why do you think that? - the view doesn’t seem to make any sense.
Yes Tim—as I pointed out earlier however, under reasonable assumptions an AI will upon self reflection on the circumstances leading to its existence as well as its utility function conclude that a strictly literal interpretation of its utility function would have to be against the implicit wishes of its originator.