CEV is way too far from a useful design, or even a big-picture sketch. It’s a non-technical vague description of something that doesn’t automatically fail for obvious reasons, as compared to essentially all other descriptions of human decision problem (friendliness content) published elsewhere. But “running CEV” is like running the sketch of da Vinci’s flying machine.
CEV is a reasonable description on the level where a sketch of a plane doesn’t insist on the plane having a beak or being made entirely out of feathers. It does look very good in comparison to other published sketches. But we don’t have laws of aerodynamics or wind tunnels. “Running the sketch” is never a plan (which provokes protestations such as this). One (much preferable) way of going forward is to figure out the fundamental laws, that’s decision theory/philosophy/math path. Another is to copy a bird in some sense, collecting all of its properties in as much detail as possible (metaphorically speaking, so that it’s about copying goals and not about emulating brains); that’s the neuroscience path, which I expect isn’t viable no matter how much time is given, since we don’t really know how to learn about goals by looking at brains or behavior.
(Perhaps when we figure out the fundamental laws, it’ll turn out that we want a helicopter, and to the dustbin goes the original sketch.)
I agree that CEV needs conceptual and technical fleshing-out; when I said “run CEV”, I meant “run some suitably fleshed-out version of CEV”. You seem to be saying that to do this fleshing-out, we will need knowledge of some large subset of the details of human value. I’m not saying that’s false, but I’m trying to get at what sort of details you think those are; what variables we’re trying to find out the value of. Again, surely it’s not all the details, or we wouldn’t need to run CEV in the first place.
CEV is way too far from a useful design, or even a big-picture sketch. It’s a non-technical vague description of something that doesn’t automatically fail for obvious reasons, as compared to essentially all other descriptions of human decision problem (friendliness content) published elsewhere. But “running CEV” is like running the sketch of da Vinci’s flying machine.
CEV is a reasonable description on the level where a sketch of a plane doesn’t insist on the plane having a beak or being made entirely out of feathers. It does look very good in comparison to other published sketches. But we don’t have laws of aerodynamics or wind tunnels. “Running the sketch” is never a plan (which provokes protestations such as this). One (much preferable) way of going forward is to figure out the fundamental laws, that’s decision theory/philosophy/math path. Another is to copy a bird in some sense, collecting all of its properties in as much detail as possible (metaphorically speaking, so that it’s about copying goals and not about emulating brains); that’s the neuroscience path, which I expect isn’t viable no matter how much time is given, since we don’t really know how to learn about goals by looking at brains or behavior.
(Perhaps when we figure out the fundamental laws, it’ll turn out that we want a helicopter, and to the dustbin goes the original sketch.)
I agree that CEV needs conceptual and technical fleshing-out; when I said “run CEV”, I meant “run some suitably fleshed-out version of CEV”. You seem to be saying that to do this fleshing-out, we will need knowledge of some large subset of the details of human value. I’m not saying that’s false, but I’m trying to get at what sort of details you think those are; what variables we’re trying to find out the value of. Again, surely it’s not all the details, or we wouldn’t need to run CEV in the first place.