“My set of values are utterly whimsical [...] The reasons for my desires can be described biologically, evolutionarily or with physics of a suitable resolution. But now that I have them they are mine and I need no further reason.”
If that is your stated position then in what way can you claim to create FAI with this whimsical set of goals? This is the crux you see: unless you find some unobjectionable set of values (such as in rational morality ‘existence is preferable over non-existence’ ⇒ utility = continued existence ⇒ modified to ensure continued co-existence with the ‘other’ to make it unobjectionable ⇒ apply rationality in line with microeconomic theory to maximize this utility et cetera) you will end up being a deluded self serving optimizer.
If that is your stated position then in what way can you claim to create FAI with this whimsical set of goals?
Were it within my power to do so I would create a machine that was really, really good at doing things I like. It is that simple. This machine is (by definition) ‘Friendly’ to me.
you will end up being a deluded self serving optimizer.
I don’t know where the ‘deluded’ bit comes from but yes, I would end up being a self serving optimizer. Fortunately for everyone else my utility function places quite a lot of value on the whims of other people. My self serving interests are beneficial to others too because I am actually quite a compassionate and altruistic guy.
PS: Instead of using quotation marks you can put a ‘>’ at the start of a quoted line. This convention makes quotations far easier to follow. And looks prettier.
There is no such thing as an “unobjectionable set of values”.
Imagine the values of an agent that wants all the atoms in the universe for its own ends. It will object to any other agent’s values—since it objects to the very existence of other agents—since those agents use up its precious atoms—and put them into “wrong” configurations.
Whatever values you have, they seem bound to piss off somebody.
There is no such thing as an “unobjectionable set of values”.
And here I disagree. Firstly see my comment about utility function interpretation on another post of yours. Secondly, as soon as one assumes existence as being preferable over non-existence you can formulate a set of unobjectionable values (http://www.jame5.com/?p=45 and http://rationalmorality.info/?p=124). But granted, if you do not want to exist nor have a desire to be rational then rational morality has in fact little to offer you. Non existence and irrational behavior being so trivial goals to achieve after all that it would hardly require – nor value and thus seek for that mater – well thought out advice.
Alas, the first link seems almost too silly to bother with to me, but briefly:
Unobjectionable—to whom? An agent objecting to another agent’s values is a simple and trivial occurrence. All an agent has to do is to state that—according to its values—it wants to use the atoms of the agent with the supposedly unobjectionable utility function for something else.
“Ensure continued co-existence” is vague and wishy-washy. Perhaps publicly work through some “trolley problems” using it—so people have some idea of what you think it means.
You claim there can be no rational objection to your preferred utility function.
In fact, an agent with a different utility function can (obviously) object to its existence—on grounds of instrumental rationality. I am not clear on why you don’t seem to recognise this.
“My set of values are utterly whimsical [...] The reasons for my desires can be described biologically, evolutionarily or with physics of a suitable resolution. But now that I have them they are mine and I need no further reason.”
If that is your stated position then in what way can you claim to create FAI with this whimsical set of goals? This is the crux you see: unless you find some unobjectionable set of values (such as in rational morality ‘existence is preferable over non-existence’ ⇒ utility = continued existence ⇒ modified to ensure continued co-existence with the ‘other’ to make it unobjectionable ⇒ apply rationality in line with microeconomic theory to maximize this utility et cetera) you will end up being a deluded self serving optimizer.
Were it within my power to do so I would create a machine that was really, really good at doing things I like. It is that simple. This machine is (by definition) ‘Friendly’ to me.
I don’t know where the ‘deluded’ bit comes from but yes, I would end up being a self serving optimizer. Fortunately for everyone else my utility function places quite a lot of value on the whims of other people. My self serving interests are beneficial to others too because I am actually quite a compassionate and altruistic guy.
PS: Instead of using quotation marks you can put a ‘>’ at the start of a quoted line. This convention makes quotations far easier to follow. And looks prettier.
There is no such thing as an “unobjectionable set of values”.
Imagine the values of an agent that wants all the atoms in the universe for its own ends. It will object to any other agent’s values—since it objects to the very existence of other agents—since those agents use up its precious atoms—and put them into “wrong” configurations.
Whatever values you have, they seem bound to piss off somebody.
And here I disagree. Firstly see my comment about utility function interpretation on another post of yours. Secondly, as soon as one assumes existence as being preferable over non-existence you can formulate a set of unobjectionable values (http://www.jame5.com/?p=45 and http://rationalmorality.info/?p=124). But granted, if you do not want to exist nor have a desire to be rational then rational morality has in fact little to offer you. Non existence and irrational behavior being so trivial goals to achieve after all that it would hardly require – nor value and thus seek for that mater – well thought out advice.
Alas, the first link seems almost too silly to bother with to me, but briefly:
Unobjectionable—to whom? An agent objecting to another agent’s values is a simple and trivial occurrence. All an agent has to do is to state that—according to its values—it wants to use the atoms of the agent with the supposedly unobjectionable utility function for something else.
“Ensure continued co-existence” is vague and wishy-washy. Perhaps publicly work through some “trolley problems” using it—so people have some idea of what you think it means.
You claim there can be no rational objection to your preferred utility function.
In fact, an agent with a different utility function can (obviously) object to its existence—on grounds of instrumental rationality. I am not clear on why you don’t seem to recognise this.