Oh, you wanted to know my utility function? It goes something like this:
I noticed there was no explicit reference to truth (as in accurate mappings of the territory, not as in truth with a capital T). Am I to assume that goes along with protecting life, stopping extinction, and possibly as a subset of other people’s pursuits of happiness?
Yes—I probably value keeping people well-informed, in addition to what I mentioned. If nothing else, this would stop me from assigning a very high utility to “everyone gets wireheaded”.
Keeping myself well-informed is more of a sub-goal than an end value. I can’t improve the territory unless my map is accurate.
I noticed there was no explicit reference to truth (as in accurate mappings of the territory, not as in truth with a capital T). Am I to assume that goes along with protecting life, stopping extinction, and possibly as a subset of other people’s pursuits of happiness?
Yes—I probably value keeping people well-informed, in addition to what I mentioned. If nothing else, this would stop me from assigning a very high utility to “everyone gets wireheaded”.
Keeping myself well-informed is more of a sub-goal than an end value. I can’t improve the territory unless my map is accurate.