I think a lot of the values we care about are cultural, not just genetic. A human raised without culture isn’t even clearly going to be generally intelligent (in the way humans are), so why assume they’d share our values?
Estimations of the information content of this part are discussed by Eric Baum in What is Thought?, although I do not recall the details.
I find that plausible, a priori. Mostly doesn’t affect the stuff in the talk, since that would still come from the environment, and the same principles would apply to culturally-derived values as to environment-derived values more generally. Assuming the hardwired part is figured out, we should still be able to get an estimate of human values within the typical-human-value-distribution-for-a-given-culture from data which is within the typical-human-environment-distribution-for-that-culture.
I think a lot of the values we care about are cultural, not just genetic. A human raised without culture isn’t even clearly going to be generally intelligent (in the way humans are), so why assume they’d share our values?
Estimations of the information content of this part are discussed by Eric Baum in What is Thought?, although I do not recall the details.
I find that plausible, a priori. Mostly doesn’t affect the stuff in the talk, since that would still come from the environment, and the same principles would apply to culturally-derived values as to environment-derived values more generally. Assuming the hardwired part is figured out, we should still be able to get an estimate of human values within the typical-human-value-distribution-for-a-given-culture from data which is within the typical-human-environment-distribution-for-that-culture.