So, doesn’t it seem to anyone else that our priority here ought to be to strive for consensus on goals, so that we at least come to understand better just what obstacles stand in the way of achieving consensus?
Yes.
And also to get a better feel for whether having one’s own volition overruled by the coherent extrapolated volition of mankind is something one really wants.
Hell no.
To my mind, the really important question is whether we have one-big-AI which we hope is friendly, or an ecosystem of less powerful AIs and humans cooperating and competing under some kind of constitution. I think that the latter is the obvious way to go.
Sounds like a good way to go extinct. That is, unless the ‘constitution’ manages to implement friendliness.
And I just don’t trust anyone pushing for the first option—particularly when they want to be the one who defines “friendly”.
I’m not too keen about the prospect either. But it may well become a choice between that and certain doom.
And I just don’t trust anyone pushing for the first option—particularly when they want to be the one who defines “friendly”.
to get a better feel for whether having one’s own volition overruled by the coherent extrapolated volition of mankind is something one really wants.
Hell no.
Am I to interpret that expletive as expressing that you already have a pretty good feel regarding whether you would want that?
To my mind, the really important question is whether we have one-big-AI which we hope is friendly, or an ecosystem of less powerful AIs and humans cooperating and competing under some kind of constitution. I think that the latter is the obvious way to go.
Sounds like a good way to go extinct. That is, unless the ‘constitution’ manages to implement friendliness.
We’ll get to the definition of “friendliness” in a moment. What I think is crucial is that the constitution implements some form of “fairness” and that the AI’s and constitution together advance some meta-goals like tolerance and communication and understanding other viewpoints.
As to “friendliness”, the thing I most dislike about the definition “friendliness” = “CEV” is that in Eliezer’s vision, it seems that everyone wants the same things. In my opinion, on the other hand, the mechanisms for resolution of conflicting objectives constitute the real core of the problem. And I believe that the solutions pretty much already exist, in standard academic rational agent game theory. With AIs assisting, and with a constitution granting humans equal power over each other and over AIs, and granting AIs power only over each other, I think we can create a pretty good future.
With one big AI, whose “friendliness” circuits have been constructed by a megalomaniac who seems to believe in a kind of naive utilitarianism with direct interpersonal comparison of utility and discounting of the future forbidden; … well …, I see this kind of future as a recipe for disaster.
As to “friendliness”, the thing I most dislike about the definition “friendliness” = “CEV” is that in Eliezer’s vision, it seems that everyone wants the same things.
Yes.
Hell no.
Sounds like a good way to go extinct. That is, unless the ‘constitution’ manages to implement friendliness.
I’m not too keen about the prospect either. But it may well become a choice between that and certain doom.
And I just don’t trust anyone pushing for the first option—particularly when they want to be the one who defines “friendly”.
Am I to interpret that expletive as expressing that you already have a pretty good feel regarding whether you would want that?
We’ll get to the definition of “friendliness” in a moment. What I think is crucial is that the constitution implements some form of “fairness” and that the AI’s and constitution together advance some meta-goals like tolerance and communication and understanding other viewpoints.
As to “friendliness”, the thing I most dislike about the definition “friendliness” = “CEV” is that in Eliezer’s vision, it seems that everyone wants the same things. In my opinion, on the other hand, the mechanisms for resolution of conflicting objectives constitute the real core of the problem. And I believe that the solutions pretty much already exist, in standard academic rational agent game theory. With AIs assisting, and with a constitution granting humans equal power over each other and over AIs, and granting AIs power only over each other, I think we can create a pretty good future.
With one big AI, whose “friendliness” circuits have been constructed by a megalomaniac who seems to believe in a kind of naive utilitarianism with direct interpersonal comparison of utility and discounting of the future forbidden; … well …, I see this kind of future as a recipe for disaster.
He doesn’t think that—but he does seem to have some rather curious views of the degree of similarity between humans.