Not really an answer to your question, but it seems to me a lot depends on what position I take wrt value drift and the subject-dependence of values.
At one extreme: if I believe that whatever I happen to value right now is what I value, and what I value tomorrow is what I value tomorrow, and it simply doesn’t matter how those things relate to each other, I just want to optimize my environment for what I value at any given moment, then it makes sense to concentrate on security without reference to goals. More precisely, it makes sense to concentrate on mechanisms for optimizing my environment for any given value, and security is a very important part of that.
At another extreme: if I believe that there is One True Value Set that ought to be optimized for (even if I don’t happen to know what that is, or even if I don’t particularly value it [1] ), thinking about goals is valuable only insofar as it leads to systems better able to implement the OTVS.
Only if I believe that my values are the important ones, believe my values can change, and endorse my current values over my values at other times, does working out a way to preserve my current values against value-shifts (either intentionally imposed shifts, as in your examples, or natural drift) start to seem important.
I know lots of people who don’t seem to believe that their current values are more important than their later values, at least not in any way that consistently constrains their planning. That is, they seem to prefer to avoid committing to their current values, and to instead keep their options open.
And I can see how that sort of thinking leads to the idea that “secure property rights” (and, relatedly, reliably enforced consensual contracts) are the most important thing.
[1] EDIT: in retrospect, this is a somewhat confused condition; what I really mean is more like “even if I’m not particularly aware of myself valuing it”, or “even if my valuation of it is not reflectively consistent” or something of that sort.
More precisely, it makes sense to concentrate on mechanisms for optimizing my environment for any given value, and security is a very important part of that.
Wouldn’t that be a bad idea? If you change your mind as to what you value, then Future!you will optimize for something Present!you doesn’t want. Since you’re only worried about Present!you’s goals, that would be bad.
Sure, if I’m only worried about Present!me’s goals, then the entire rest of the paragraph you didn’t bother quoting is of course false, and the sentence you quote for which that paragraph was intended as context is also false.
I don’t understand how your hypothetical beliefs of paragraph two differ from those of paragraph four. Or don’t they? Please elaborate. Are you saying that Nick Szabo’s position depends on (or at least is helped by) viewing one’s later values as quite possibly better than current ones?
What I’m referring to in paragraphs 2 and 4 are similar enough that what differences may exist between them don’t especially matter to any point I’m making.
Are you saying that Nick Szabo’s position depends on (or at least is helped by) viewing one’s later values as quite possibly better than current ones?
No, and in fact I don’t believe that. Better to say that, insofar as an important component of Friendliness research is working out ways to avoid value drift, the OP’s preference for Friendliness research over security research is reinforced by a model of the world in which value drift is a problem to be avoided/fixed rather than simply a neutral feature of the world.
Not really an answer to your question, but it seems to me a lot depends on what position I take wrt value drift and the subject-dependence of values.
At one extreme: if I believe that whatever I happen to value right now is what I value, and what I value tomorrow is what I value tomorrow, and it simply doesn’t matter how those things relate to each other, I just want to optimize my environment for what I value at any given moment, then it makes sense to concentrate on security without reference to goals. More precisely, it makes sense to concentrate on mechanisms for optimizing my environment for any given value, and security is a very important part of that.
At another extreme: if I believe that there is One True Value Set that ought to be optimized for (even if I don’t happen to know what that is, or even if I don’t particularly value it [1] ), thinking about goals is valuable only insofar as it leads to systems better able to implement the OTVS.
Only if I believe that my values are the important ones, believe my values can change, and endorse my current values over my values at other times, does working out a way to preserve my current values against value-shifts (either intentionally imposed shifts, as in your examples, or natural drift) start to seem important.
I know lots of people who don’t seem to believe that their current values are more important than their later values, at least not in any way that consistently constrains their planning. That is, they seem to prefer to avoid committing to their current values, and to instead keep their options open.
And I can see how that sort of thinking leads to the idea that “secure property rights” (and, relatedly, reliably enforced consensual contracts) are the most important thing.
[1] EDIT: in retrospect, this is a somewhat confused condition; what I really mean is more like “even if I’m not particularly aware of myself valuing it”, or “even if my valuation of it is not reflectively consistent” or something of that sort.
Wouldn’t that be a bad idea? If you change your mind as to what you value, then Future!you will optimize for something Present!you doesn’t want. Since you’re only worried about Present!you’s goals, that would be bad.
Sure, if I’m only worried about Present!me’s goals, then the entire rest of the paragraph you didn’t bother quoting is of course false, and the sentence you quote for which that paragraph was intended as context is also false.
Sorry. I missed a word when I read it the first time.
I don’t understand how your hypothetical beliefs of paragraph two differ from those of paragraph four. Or don’t they? Please elaborate. Are you saying that Nick Szabo’s position depends on (or at least is helped by) viewing one’s later values as quite possibly better than current ones?
What I’m referring to in paragraphs 2 and 4 are similar enough that what differences may exist between them don’t especially matter to any point I’m making.
No, and in fact I don’t believe that. Better to say that, insofar as an important component of Friendliness research is working out ways to avoid value drift, the OP’s preference for Friendliness research over security research is reinforced by a model of the world in which value drift is a problem to be avoided/fixed rather than simply a neutral feature of the world.