It is not obvious why creating a causal chain that goes on indefinitely is uniquely morally relevant. (Nor is it obvious that the concept is meaningful in reality—a causal chain with a starting point can be unboundedly long but at no actual point in time will it be infinite.) I do see it as valuable to look for ways to escape this space-time continuum, because I presently want (and think I will continue to want) (post)humanity to continue existing and growing indefinitely, but I don’t believe there is any universal validity to this value. (If values like this form attractors for complex—i.e. not paperclip-maximizing—intelligences I suppose they would be in a sense “objective”, but would not acquire any more normative force, whatever that is.) I don’t see this value as “unreal” because it’s subjective, though. My subjectivity is very real to me.
Saying that only indefinitely-long causal chains are important does not tell us which indefinitely-long causal chains are good and which ones are evil.
This was Eliezer’s point: how could you ever recognize which ones are good and which ones are evil? How could you even recognize a process for recognizing objective good and evil?
I’m still reading.
It is not obvious why creating a causal chain that goes on indefinitely is uniquely morally relevant. (Nor is it obvious that the concept is meaningful in reality—a causal chain with a starting point can be unboundedly long but at no actual point in time will it be infinite.) I do see it as valuable to look for ways to escape this space-time continuum, because I presently want (and think I will continue to want) (post)humanity to continue existing and growing indefinitely, but I don’t believe there is any universal validity to this value. (If values like this form attractors for complex—i.e. not paperclip-maximizing—intelligences I suppose they would be in a sense “objective”, but would not acquire any more normative force, whatever that is.) I don’t see this value as “unreal” because it’s subjective, though. My subjectivity is very real to me.
Saying that only indefinitely-long causal chains are important does not tell us which indefinitely-long causal chains are good and which ones are evil.
This was Eliezer’s point: how could you ever recognize which ones are good and which ones are evil? How could you even recognize a process for recognizing objective good and evil?