the idea that you put together a 3d physics module and a calculus module and a social module and a vision module and a language module and you get something that venerates Mickey Mouse shapes is… just bizarre.
Is it any less bizzare to put together a bunch of modules that would work for any goal, and get out of them something that values all four of humor, cute kittens, friendship, and movies? What I mean by this is that precisely human values are as contingent and non-special as a broad class of other values.
Is it any less bizzare to put together a bunch of modules that would work for any goal, and get out of them something that values all four of humor, cute kittens, friendship, and movies?
Yes. Think about it.
What I mean by this is that precisely human values are as contingent and non-special as a broad class of other values.
Human values are fragmentary subvalues of one value, which is what one would expect from a bunch of modules that each contribute to reproduction in a different way. The idea of putting together a bunch of different modules to get a single, overriding value, is bizarre. (The only possible exemption here is ‘make more of myself,’ but the modules are probably going to implement subvalues for that, rather than that as an explicit value. As far as single values go, that one’s special, whereas things like Mickey Mouse faces are not.)
Is it any less bizzare to put together a bunch of modules that would work for any goal, and get out of them something that values all four of humor, cute kittens, friendship, and movies? What I mean by this is that precisely human values are as contingent and non-special as a broad class of other values.
Yes. Think about it.
Human values are fragmentary subvalues of one value, which is what one would expect from a bunch of modules that each contribute to reproduction in a different way. The idea of putting together a bunch of different modules to get a single, overriding value, is bizarre. (The only possible exemption here is ‘make more of myself,’ but the modules are probably going to implement subvalues for that, rather than that as an explicit value. As far as single values go, that one’s special, whereas things like Mickey Mouse faces are not.)