J Thomas—”in principle you ought to consider the entire state of the future universe when you set a terminal value.”
Yes, and in practice we don’t. But as I look further into the future to see the consequences of my terminal value(s), that’s when the trouble begins.
igor—I want to defend Eliezer’s bias against boredom. It seems that many of the ‘most moral’ terminal values (total freedom, complete knowledge, endless bliss...) would end up in a condition of hideous boredom.
Maybe that’s why we don’t achieve them.
Richard- I read your post. I agree with the conclusions to a large extent, but totally disagree with the premises. (For example- I think the only valueable thing is subjective experience) Isn’t that amazing?
J Thomas—”in principle you ought to consider the entire state of the future universe when you set a terminal value.” Yes, and in practice we don’t. But as I look further into the future to see the consequences of my terminal value(s), that’s when the trouble begins.
igor—I want to defend Eliezer’s bias against boredom. It seems that many of the ‘most moral’ terminal values (total freedom, complete knowledge, endless bliss...) would end up in a condition of hideous boredom. Maybe that’s why we don’t achieve them.
Richard- I read your post. I agree with the conclusions to a large extent, but totally disagree with the premises. (For example- I think the only valueable thing is subjective experience) Isn’t that amazing?