If we can copy values, or represent them as digital information, I haven’t heard about it.
The closest thing I’ve seen is tools for exporting values into an intersubjective format like speech, writing, art, or behavior. As you point out, the subsequent corresponding import often fails… whether that’s because of explicit defense mechanisms, or because the exported data structure lacks key data, or because the import process is defective in some way, or for some other reason, is hard to tease out.
Maybe you mean something different from me by the term ‘values’. The values I was referring to are fairly simple to write down. Many of them are so codified in legal systems and religious traditions.
If I tell you that I like mulberries more than blackberries, then that’s information about my values represented digitally. The guts of the value information really is in there. Consequently, you can accurately make predictions about what I will do if presented with various food choices—without actually personally adopting my values.
Yeah, we apparently mean different things. To my mind the statement “I like mulberries more than blackberries” is not even close to being a high-fidelity copy of my relative preferences for mulberries and blackberries; I could not reconstruct the latter given only the former.
I would classify it as being value information which can be accurately copied. I never meant to suggest that it was an accurate copy of what was in your mind. For instance, your mental representation could be considered to include additional information about what mulberries and blackberries are—broadly related to what might be found in an encyclopedia of berries. The point is that we can represent people’s values digitally—to the point where we can make quite good predictions about what choices they will make under controlled conditions involving value-derived choices. Values aren’t especially mysterious, they are just what people want—and we have a big mountain of information about that which can be represented digitally.
You mean “values aren’t especially mysterious”, I expect.
I agree that they’re not mysterious. More specifically, I agree that what it means to capture my value information about X and Y is to capture the information someone else would need in order to accurately and reliably predict my relative preferences for X and Y under a wide range of conditions. And, yes, a large chunk of that information is what you describe here as encyclopedic knowledge.
So for (X,Y)=(mulberry, blackberry) a high-fidelity copy of my values, in conjunction with a suitable encyclopedia of berries, would allow you to reliably predict which one I would prefer to eat with chocolate ice cream, which one I would prefer to spread as jam on rye bread, which one I would prefer to decorate a cake with, which one I would prefer to receive a pint of as a gift, how many pints of one I’d exchange for a pint of the other, etc., etc., etc.
Yes?
Assuming I’ve gotten that right… so, when you say:
We do have high fidelity copying today. We can accurately copy anything we can represent as digital information—including values ...do you mean to suggest that we can, today, create a high-fidelity copy of my values with respect to mulberries and blackberries as described above?
(Obviously, this is a very simple problem in degenerate cases like “I like blackberries and hate mulberries,” but that’s not all that interesting.)
If so, do you know of any examples of that sort of high-fidelity copy of someone’s values with respect to some non-degenerate (X,Y) pair actually having been created? Can you point me at one?
I can’t meet your “complex value extraction” challenge. I never meant to imply “complete” extraction—just that we can extract value information (like this) and then copy it around with high fidelity. Revealed preferences can be good, but I wouldn’t like to get into quantifying their accuracy here.
OK. I certainly agree that any information we know how to digitally encode in the first place, we can copy around with high fidelity. But we don’t know how to digitally encode our values in the first place, so we don’t know how to copy them. That’s not because value is some kind of mysterious abstract ethereal “whatness of the if”… we can define it concretely as the stuff that informs, and in principle allows an observer to predict, our revealed preferences… but because it’s complicated. I’m inclined to agree with Wei_Dai that high-fidelity value sharing would represent a significant breakthrough in our understanding of and our ability to engineer human psychology, and would likely be a game-changer.
But we don’t know how to digitally encode our values in the first place, so we don’t know how to copy them.
Well, we do have the idea of revealed preference. Also, if you want to know what people value, you can often try asking them. Between them, these ideas work quite well.
What we can’t do is build a machine that optimises them—so there is something missing, but it’s mostly not value information. We can’t automatically perform inductive inference very well, for one thing.
I suspect I agree with you about what information we can encode today, and you seem to agree with me that there’s additional information in our brains (for example, information about berries) that we use to make those judgments which revealed preferences (and to a lesser extent explicitly articulated preferences) report on, which we don’t yet know how to encode.
I don’t really care whether we call that additional information “value information” or not; I thought initially you were claiming that we could in practice encode it. Thank you for clarifying.
Also agreed that there are operations our brains perform that we don’t know how to automate.
If we can copy values, or represent them as digital information, I haven’t heard about it.
The closest thing I’ve seen is tools for exporting values into an intersubjective format like speech, writing, art, or behavior. As you point out, the subsequent corresponding import often fails… whether that’s because of explicit defense mechanisms, or because the exported data structure lacks key data, or because the import process is defective in some way, or for some other reason, is hard to tease out.
Maybe you mean something different from me by the term ‘values’. The values I was referring to are fairly simple to write down. Many of them are so codified in legal systems and religious traditions.
If I tell you that I like mulberries more than blackberries, then that’s information about my values represented digitally. The guts of the value information really is in there. Consequently, you can accurately make predictions about what I will do if presented with various food choices—without actually personally adopting my values.
Yeah, we apparently mean different things. To my mind the statement “I like mulberries more than blackberries” is not even close to being a high-fidelity copy of my relative preferences for mulberries and blackberries; I could not reconstruct the latter given only the former.
I would classify it as being value information which can be accurately copied. I never meant to suggest that it was an accurate copy of what was in your mind. For instance, your mental representation could be considered to include additional information about what mulberries and blackberries are—broadly related to what might be found in an encyclopedia of berries. The point is that we can represent people’s values digitally—to the point where we can make quite good predictions about what choices they will make under controlled conditions involving value-derived choices. Values aren’t especially mysterious, they are just what people want—and we have a big mountain of information about that which can be represented digitally.
You mean “values aren’t especially mysterious”, I expect.
I agree that they’re not mysterious. More specifically, I agree that what it means to capture my value information about X and Y is to capture the information someone else would need in order to accurately and reliably predict my relative preferences for X and Y under a wide range of conditions. And, yes, a large chunk of that information is what you describe here as encyclopedic knowledge.
So for (X,Y)=(mulberry, blackberry) a high-fidelity copy of my values, in conjunction with a suitable encyclopedia of berries, would allow you to reliably predict which one I would prefer to eat with chocolate ice cream, which one I would prefer to spread as jam on rye bread, which one I would prefer to decorate a cake with, which one I would prefer to receive a pint of as a gift, how many pints of one I’d exchange for a pint of the other, etc., etc., etc.
Yes?
Assuming I’ve gotten that right… so, when you say:
(Obviously, this is a very simple problem in degenerate cases like “I like blackberries and hate mulberries,” but that’s not all that interesting.)
If so, do you know of any examples of that sort of high-fidelity copy of someone’s values with respect to some non-degenerate (X,Y) pair actually having been created? Can you point me at one?
I can’t meet your “complex value extraction” challenge. I never meant to imply “complete” extraction—just that we can extract value information (like this) and then copy it around with high fidelity. Revealed preferences can be good, but I wouldn’t like to get into quantifying their accuracy here.
OK.
I certainly agree that any information we know how to digitally encode in the first place, we can copy around with high fidelity.
But we don’t know how to digitally encode our values in the first place, so we don’t know how to copy them. That’s not because value is some kind of mysterious abstract ethereal “whatness of the if”… we can define it concretely as the stuff that informs, and in principle allows an observer to predict, our revealed preferences… but because it’s complicated.
I’m inclined to agree with Wei_Dai that high-fidelity value sharing would represent a significant breakthrough in our understanding of and our ability to engineer human psychology, and would likely be a game-changer.
Well, we do have the idea of revealed preference. Also, if you want to know what people value, you can often try asking them. Between them, these ideas work quite well.
What we can’t do is build a machine that optimises them—so there is something missing, but it’s mostly not value information. We can’t automatically perform inductive inference very well, for one thing.
I suspect I agree with you about what information we can encode today, and you seem to agree with me that there’s additional information in our brains (for example, information about berries) that we use to make those judgments which revealed preferences (and to a lesser extent explicitly articulated preferences) report on, which we don’t yet know how to encode.
I don’t really care whether we call that additional information “value information” or not; I thought initially you were claiming that we could in practice encode it. Thank you for clarifying.
Also agreed that there are operations our brains perform that we don’t know how to automate.