This relates to a theoretical question which is more interesting to me: are “individual extrapolated volitions” in/compatible with a “collective extrapolated volition” (a la CEV)? Are there “natural classes” of individuals who, in collection, form their own coherent extrapolated volition which is compatible among themselves, but incompatible with others extrapolated volitions, e.g. men vs. women, smart people vs. dumb people, individualists vs. collectivists, all of humanity happens to be compatible, or every person individually is incompatible with everyone else...? In other words, how much natural coherence is there among individual extrapolated volitions, and how much would CEV require to sacrifice individual extrapolated volitions for the collective extrapolated volition?
If it happened to work out that the entire collective of humanity could have a coherent extrapolated volition in which no individual extrapolated volition is sacrificed, that would be perfect and would relieve these uncomfortable questions. If that’s not the case—should be proceed with CEV anyway? Or is it possible we may want to favor one group over another?
right. this raises the point that the programmer of the FAI would rationally not write it with CEV. He would use his own individual extrapolated volition. Whether that happened to be coherent with the extrapolated volitions of the rest of humanity is irrelevant (to him) - because obviously he would rationally favor himself or his group in any instance of incoherence.
This relates to a theoretical question which is more interesting to me: are “individual extrapolated volitions” in/compatible with a “collective extrapolated volition” (a la CEV)? Are there “natural classes” of individuals who, in collection, form their own coherent extrapolated volition which is compatible among themselves, but incompatible with others extrapolated volitions, e.g. men vs. women, smart people vs. dumb people, individualists vs. collectivists, all of humanity happens to be compatible, or every person individually is incompatible with everyone else...? In other words, how much natural coherence is there among individual extrapolated volitions, and how much would CEV require to sacrifice individual extrapolated volitions for the collective extrapolated volition?
If it happened to work out that the entire collective of humanity could have a coherent extrapolated volition in which no individual extrapolated volition is sacrificed, that would be perfect and would relieve these uncomfortable questions. If that’s not the case—should be proceed with CEV anyway? Or is it possible we may want to favor one group over another?
Of course we want to favor the group we are part of. Otherwise our CEV wouldn’t differ.
right. this raises the point that the programmer of the FAI would rationally not write it with CEV. He would use his own individual extrapolated volition. Whether that happened to be coherent with the extrapolated volitions of the rest of humanity is irrelevant (to him) - because obviously he would rationally favor himself or his group in any instance of incoherence.