Well, I guess this is as good a time and place as any to give a long winded, speculative response. Just skip this if you doubt my speculation is going to be interesting or useful.
Let’s assume that there exist some sort of objective right, no matter what that actually means. If humans desire to be right, isn’t it the sort of human value that a friendly AI would seek to protect and cultivate?
What does it even mean for there to be an “objective right” given the view that morals are a result of the blind forces of natural selection?
As I understand it, the idea of CEV is to somehow determine whether there is an executable protocol that would uniformally raise or leave the same everyone’s utility without significantly (not sure how to determine this) lowering the utility of any minority group.
My more detailed (and still hopelessly vague at this point) understanding is that the utility function of a person is a neurological fact about them, and that the idea is sort of like taking a simulation of a perfect Bayesian utility maximizer for each person and equipping this maximizer with the individual’s utility function, and then running the set of Bayesian utility maximizers on a set of possible futures (determined how? I don’t know. The CEV paper seemed extremely general, differentiating between ‘nice place to live’ and the more general goal of the CEV) to see what they converge on (again I have no clue as to how this convergence is determined).
On the other hand; I assume that UDT has been deemed necessary to the implementation of whatever computation is going to determine which actions must be taken in order to achieve the outcome converged upon, and indeed for figuring out how to compute the convergence itself (since the spread and muddle factors and the like must be taken into consideration).
It seems like the confusing part of all of this isn’t so much figuring out what people like ( that seems like a very solvable problem given enough understanding about neuroscience), but figuring out how to (1) make preferences converge and (2) decide which actions are going to be acceptable (minimizing ‘spread’ and the effects of ‘muddle’ at each step).
All of these things considered, my guess is that lukeprog wants to promote CEV indirectly by promoting the idea that meta-ethics is fundamentally solvable by studying human neurology and evolutionary psychology. That solving meta-ethics is akin to discerning the human utility function and coming up with a theory for how to reconcile competing human utility functions.
Of course all of this might be totally off base, I don’t really know, I’m still kind of new to all of this and I’m trying to infer quite a bit.
Well, I guess this is as good a time and place as any to give a long winded, speculative response. Just skip this if you doubt my speculation is going to be interesting or useful.
What does it even mean for there to be an “objective right” given the view that morals are a result of the blind forces of natural selection?
As I understand it, the idea of CEV is to somehow determine whether there is an executable protocol that would uniformally raise or leave the same everyone’s utility without significantly (not sure how to determine this) lowering the utility of any minority group.
My more detailed (and still hopelessly vague at this point) understanding is that the utility function of a person is a neurological fact about them, and that the idea is sort of like taking a simulation of a perfect Bayesian utility maximizer for each person and equipping this maximizer with the individual’s utility function, and then running the set of Bayesian utility maximizers on a set of possible futures (determined how? I don’t know. The CEV paper seemed extremely general, differentiating between ‘nice place to live’ and the more general goal of the CEV) to see what they converge on (again I have no clue as to how this convergence is determined).
On the other hand; I assume that UDT has been deemed necessary to the implementation of whatever computation is going to determine which actions must be taken in order to achieve the outcome converged upon, and indeed for figuring out how to compute the convergence itself (since the spread and muddle factors and the like must be taken into consideration).
It seems like the confusing part of all of this isn’t so much figuring out what people like ( that seems like a very solvable problem given enough understanding about neuroscience), but figuring out how to (1) make preferences converge and (2) decide which actions are going to be acceptable (minimizing ‘spread’ and the effects of ‘muddle’ at each step).
All of these things considered, my guess is that lukeprog wants to promote CEV indirectly by promoting the idea that meta-ethics is fundamentally solvable by studying human neurology and evolutionary psychology. That solving meta-ethics is akin to discerning the human utility function and coming up with a theory for how to reconcile competing human utility functions.
Of course all of this might be totally off base, I don’t really know, I’m still kind of new to all of this and I’m trying to infer quite a bit.