Isn’t there a hidden problem with values taking other values as their arguments? If I can value having a particular value, I can possibly construct self-referential paradoxical values like V = value of not having value V, and empty values like V = value of having value V. A value system including value W = value of preserving other values, W included seems to be of that kind, and I am not sure whether it can be transformed into a consistent decision algorithm.
On the other hand, look at what happens if we avoid self-reference by introducing distinct types of values, where the 0th order values can speak only about external world states and n-th order meta-values can speak only about the external world and m<n-th order values. I assume that the Christian (or more generally any traditional) values all belong to the 0th order rank. Let’s also assume that we have a 1st-order meta-value of preserving all 0th-order values. But what prevents this meta-value from drifting away? Seems that without an infinite chain of meta-values there is no way how to prevent drift. Can people hold an infinite chain of meta-values?
Now, let’s leave the possibility of the value-preservation meta-value drift aside, and look at how this meta-value is exactly implemented. Does it say “try to minimise the integral of Abs(d value/dt)”, or does it say “try to minimise the integral of Abs(value(t)-value(t0)), where t0 is some arbitrary instant”? If the former is the case, the meta-value doesn’t encourage you to periodically revise your values with respect to some written stable standard. If an infinitesimal drift accidentally slips through your value-preservation guards, once the guards wake up they shall protect the new value set, not try to return back to the original one. Only if the latter is the case the funny argument holds water. But I suspect that the former description is more accurate, evidenced by how few people strive to return to their past values.
I expect that in most cases having an object-level value V should make you behave as if you were also protecting the meta-value V’ of your valuing V, because your switching away from V can be detrimental to V. Also see my reply to torekp for a reason why you might want to return to your past values even though they don’t coincide with the current ones.
It certainly seems that valuing V implies valuing valuing V and valuing^3 V and so on. But, if we try to formalise a bit the notion of value, doesn’t it produce some unexpected paradoxes? I haven’t thought about it in detail, so perhaps there is no problem, but I am not convinced.
I don’t understand the relevance of your reply to torekp.
I don’t see how paradoxes could arise. For example, if you have a value V of having value V, that’s a perfectly well-defined function on future states of the world and you know what to do to maximize it. (You can remove explicit self-reference by using quining), aka the diagonal lemma.) Likewise for the value W of not having value W. The actions of an agent having such a value will be pretty bizarre, but Bayesian-rational.
I don’t understand the relevance of your reply to torekp.
It shows a possible reason why you might want to return to your past values once you approach TDT-ish reflective consistency, even if you don’t want that in your current state. I’m not sure it’s correct, though.
Isn’t there a hidden problem with values taking other values as their arguments? If I can value having a particular value, I can possibly construct self-referential paradoxical values like V = value of not having value V, and empty values like V = value of having value V. A value system including value W = value of preserving other values, W included seems to be of that kind, and I am not sure whether it can be transformed into a consistent decision algorithm.
On the other hand, look at what happens if we avoid self-reference by introducing distinct types of values, where the 0th order values can speak only about external world states and n-th order meta-values can speak only about the external world and m<n-th order values. I assume that the Christian (or more generally any traditional) values all belong to the 0th order rank. Let’s also assume that we have a 1st-order meta-value of preserving all 0th-order values. But what prevents this meta-value from drifting away? Seems that without an infinite chain of meta-values there is no way how to prevent drift. Can people hold an infinite chain of meta-values?
Now, let’s leave the possibility of the value-preservation meta-value drift aside, and look at how this meta-value is exactly implemented. Does it say “try to minimise the integral of Abs(d value/dt)”, or does it say “try to minimise the integral of Abs(value(t)-value(t0)), where t0 is some arbitrary instant”? If the former is the case, the meta-value doesn’t encourage you to periodically revise your values with respect to some written stable standard. If an infinitesimal drift accidentally slips through your value-preservation guards, once the guards wake up they shall protect the new value set, not try to return back to the original one. Only if the latter is the case the funny argument holds water. But I suspect that the former description is more accurate, evidenced by how few people strive to return to their past values.
I expect that in most cases having an object-level value V should make you behave as if you were also protecting the meta-value V’ of your valuing V, because your switching away from V can be detrimental to V. Also see my reply to torekp for a reason why you might want to return to your past values even though they don’t coincide with the current ones.
It certainly seems that valuing V implies valuing valuing V and valuing^3 V and so on. But, if we try to formalise a bit the notion of value, doesn’t it produce some unexpected paradoxes? I haven’t thought about it in detail, so perhaps there is no problem, but I am not convinced.
I don’t understand the relevance of your reply to torekp.
I don’t see how paradoxes could arise. For example, if you have a value V of having value V, that’s a perfectly well-defined function on future states of the world and you know what to do to maximize it. (You can remove explicit self-reference by using quining), aka the diagonal lemma.) Likewise for the value W of not having value W. The actions of an agent having such a value will be pretty bizarre, but Bayesian-rational.
It shows a possible reason why you might want to return to your past values once you approach TDT-ish reflective consistency, even if you don’t want that in your current state. I’m not sure it’s correct, though.