But we don’t know how to digitally encode our values in the first place, so we don’t know how to copy them.
Well, we do have the idea of revealed preference. Also, if you want to know what people value, you can often try asking them. Between them, these ideas work quite well.
What we can’t do is build a machine that optimises them—so there is something missing, but it’s mostly not value information. We can’t automatically perform inductive inference very well, for one thing.
I suspect I agree with you about what information we can encode today, and you seem to agree with me that there’s additional information in our brains (for example, information about berries) that we use to make those judgments which revealed preferences (and to a lesser extent explicitly articulated preferences) report on, which we don’t yet know how to encode.
I don’t really care whether we call that additional information “value information” or not; I thought initially you were claiming that we could in practice encode it. Thank you for clarifying.
Also agreed that there are operations our brains perform that we don’t know how to automate.
Well, we do have the idea of revealed preference. Also, if you want to know what people value, you can often try asking them. Between them, these ideas work quite well.
What we can’t do is build a machine that optimises them—so there is something missing, but it’s mostly not value information. We can’t automatically perform inductive inference very well, for one thing.
I suspect I agree with you about what information we can encode today, and you seem to agree with me that there’s additional information in our brains (for example, information about berries) that we use to make those judgments which revealed preferences (and to a lesser extent explicitly articulated preferences) report on, which we don’t yet know how to encode.
I don’t really care whether we call that additional information “value information” or not; I thought initially you were claiming that we could in practice encode it. Thank you for clarifying.
Also agreed that there are operations our brains perform that we don’t know how to automate.