It seems better in principle to find a way to respect human intuitions about which things to be updateless about. Getting something wrong in the too-updateless direction can give up control of the AI to entities which we don’t think of as existing; getting something wrong in the too-updateful direction can miss out on multiverse-wide coorination via superrationality.
It seems better in principle to find a way to respect human intuitions about which things to be updateless about. Getting something wrong in the too-updateless direction can give up control of the AI to entities which we don’t think of as existing; getting something wrong in the too-updateful direction can miss out on multiverse-wide coorination via superrationality.