Are my terminal values “insufficiently extrapolated?” Or just not coherent with yours?
I don’t think it’s either. We both have the general value, “experience interesting stories,” it’s just expressed in slightly different ways. I don’t think that really really specific preferences for art consumption would be something that CEV extrapolates. I think CEV is meant to figure out what general things humans value, not really specific things (i.e. a CEV might say, “you want to experience fun adventure stories,” it would not say “read Green Lantern #26” or “read King Solomon’s Mines”). The impression I get is that CEV is more about general things like “How should we treat others?” and “How much effort should we devote to liking activities vs. approving ones?”
I don’t think our values are incoherent, you don’t want to stop me from watching horror movies and I don’t want to make you watch them. In fact, I think a CEV would probably say “It’s good to have many people who like different activities because that makes life more interesting and fun.” Some questions (like “Is it okay to torture people”) likely only have one true, or very few true, CEVs, but others, like matters of personal taste, probably vary from person to person. I think a FAI would probably order everyone not to torture toddlers, but I doubt it would order us all to watch “Animal House” at 9:00pm this coming Friday.
I don’t think it’s either. We both have the general value, “experience interesting stories,” it’s just expressed in slightly different ways. I don’t think that really really specific preferences for art consumption would be something that CEV extrapolates. I think CEV is meant to figure out what general things humans value, not really specific things (i.e. a CEV might say, “you want to experience fun adventure stories,” it would not say “read Green Lantern #26” or “read King Solomon’s Mines”). The impression I get is that CEV is more about general things like “How should we treat others?” and “How much effort should we devote to liking activities vs. approving ones?”
I don’t think our values are incoherent, you don’t want to stop me from watching horror movies and I don’t want to make you watch them. In fact, I think a CEV would probably say “It’s good to have many people who like different activities because that makes life more interesting and fun.” Some questions (like “Is it okay to torture people”) likely only have one true, or very few true, CEVs, but others, like matters of personal taste, probably vary from person to person. I think a FAI would probably order everyone not to torture toddlers, but I doubt it would order us all to watch “Animal House” at 9:00pm this coming Friday.