Set aside different preferences, because I have no idea how to deal with that. It seems like just having different source codes makes things very difficult.
Where do you think preference lives? It’s but a property of the source code, or rather, a way of looking at the source code. If you change the source code, preference changes as well (to some extent).
Where do you think preference lives? It’s but a property of the source code, or rather, a way of looking at the source code.
I agree that preferences live in the source code. Having different preferences implies having different source codes. I also agree that preferences are but a way to interpret source code, so that any two source codes might be said to have the same preferences with respect to some preference scheme.
So my comment was assuming that we already have an implicit way of mapping source codes to preference schemes, and that that mapping is not injective. That is, different source codes might have the same preference scheme. But these different source codes might still use different internal representation schemes for representing their input.
I think that my issue disappears if each agent has access to the other’s source code.
I also agree that preferences are but a way to interpret source code, so that any two source codes might be said to have the same preferences with respect to some preference scheme.
I don’t understand what the above means (and so can’t readily be in agreement with it). (“Preference scheme”?)
So my comment was assuming that we already have an implicit way of mapping source codes to preference schemes, and that that mapping is not injective.
I expect preference mapping to be “essentially” injective (possibly disregarding only stuff that doesn’t play any role in the algorithm, including the way it computes as well as the externally visible behavior). Preference is what the program does, and it’s everything that the program does. (But what the program does isn’t necessarily preferable according to that same program seen as preference.)
I don’t understand what the above means (and so can’t readily be in agreement with it). (“Preference scheme”?)
At this date almost four years later, I’m not exactly sure what I meant, which is a pretty strong indictment of my ability to write clearly.
I think that I meant the following: Take the set of all possible outcomes, partition this set of outcomes into equivalence classes, and then put an order on the resulting set of equivalence classes. The ordered set of equivalence classes is what I was calling a “preference scheme”. (… I think.)
ETA: On further thought, I don’t think that that’s exactly what I meant, but I honestly don’t know what I did mean. I’d like to think that I could have explained myself better at the time, but I didn’t write enough so that even I-now could reconstruct what that explanation would have been.
Where do you think preference lives? It’s but a property of the source code, or rather, a way of looking at the source code. If you change the source code, preference changes as well (to some extent).
I agree that preferences live in the source code. Having different preferences implies having different source codes. I also agree that preferences are but a way to interpret source code, so that any two source codes might be said to have the same preferences with respect to some preference scheme.
So my comment was assuming that we already have an implicit way of mapping source codes to preference schemes, and that that mapping is not injective. That is, different source codes might have the same preference scheme. But these different source codes might still use different internal representation schemes for representing their input.
I think that my issue disappears if each agent has access to the other’s source code.
I don’t understand what the above means (and so can’t readily be in agreement with it). (“Preference scheme”?)
I expect preference mapping to be “essentially” injective (possibly disregarding only stuff that doesn’t play any role in the algorithm, including the way it computes as well as the externally visible behavior). Preference is what the program does, and it’s everything that the program does. (But what the program does isn’t necessarily preferable according to that same program seen as preference.)
At this date almost four years later, I’m not exactly sure what I meant, which is a pretty strong indictment of my ability to write clearly.
I think that I meant the following: Take the set of all possible outcomes, partition this set of outcomes into equivalence classes, and then put an order on the resulting set of equivalence classes. The ordered set of equivalence classes is what I was calling a “preference scheme”. (… I think.)
ETA: On further thought, I don’t think that that’s exactly what I meant, but I honestly don’t know what I did mean. I’d like to think that I could have explained myself better at the time, but I didn’t write enough so that even I-now could reconstruct what that explanation would have been.