That is the real question, yes. That kind of self-modification is already cropping up, in certain fringe cases as mentioned; it will get more prevalent over time. You need a lot of information and resources in order to be able to generally self-modify like that, but once you can… should you? It’s similar to the idea of wireheading, but deeper… instead of generalized pleasure, it can be ‘whatever you want’… provided that there’s anything you want more than truth.
That is the real question, yes. That kind of self-modification is already cropping up, in certain fringe cases as mentioned; it will get more prevalent over time. You need a lot of information and resources in order to be able to generally self-modify like that, but once you can… should you? It’s similar to the idea of wireheading, but deeper… instead of generalized pleasure, it can be ‘whatever you want’… provided that there’s anything you want more than truth.