I think you may have misunderstood my comment. I’m not saying CEV is simpler overall, I’m saying it’s not clear to me why your idea is simpler, if you’re including the “feature” of allowing people inside the VR to change the AI’s values. That seems to introduce problems that are analogous to the kinds of problems that CEV has. Basically you have to design your VR universe to guarantee that people who live inside them will avoid value drift and eventually reach correct conclusions about what their values are. That’s where the main difficulty in CEV lies also, at least in my view. What do you think are some of the philosophical progress that CEV requires that your idea avoids?
The way I imagined it, people inside the VR wouldn’t be able to change the AI’s values. Population ethics seems like a problem that people can solve by themselves, negotiating with each other under the VR’s rules, without help from AI.
CEV requires extracting all human preferences, extrapolating them, determining coherence, and finding a general way to map them to physics. (We need to either do it ourselves, or teach the AI how to do it, the difference doesn’t matter to the argument.) The approach in my post skips most of these tasks, by letting humans describe a nice normal world directly, and requires mapping only one thing (consciousness) to physics. Though I agree with you that the loss of potential utility is huge, the idea is intended as a kind of lower bound.
I think you may have misunderstood my comment. I’m not saying CEV is simpler overall, I’m saying it’s not clear to me why your idea is simpler, if you’re including the “feature” of allowing people inside the VR to change the AI’s values. That seems to introduce problems that are analogous to the kinds of problems that CEV has. Basically you have to design your VR universe to guarantee that people who live inside them will avoid value drift and eventually reach correct conclusions about what their values are. That’s where the main difficulty in CEV lies also, at least in my view. What do you think are some of the philosophical progress that CEV requires that your idea avoids?
The way I imagined it, people inside the VR wouldn’t be able to change the AI’s values. Population ethics seems like a problem that people can solve by themselves, negotiating with each other under the VR’s rules, without help from AI.
CEV requires extracting all human preferences, extrapolating them, determining coherence, and finding a general way to map them to physics. (We need to either do it ourselves, or teach the AI how to do it, the difference doesn’t matter to the argument.) The approach in my post skips most of these tasks, by letting humans describe a nice normal world directly, and requires mapping only one thing (consciousness) to physics. Though I agree with you that the loss of potential utility is huge, the idea is intended as a kind of lower bound.