“That is why we should try to arrange deals across time, where for example, we agree to invest for the future, and they agree to remember and revere us.”
Consider an agent that at any time t does not discount benefits received between t and t+1 year, discounts benefits between t+1 years and t+100 years by half, and does not value benefits realized after t+100 years. If the agent is capable of self-modification, then at any particular time it will want to self-modify to replace the variable ‘t’ with a constant, the time of self-modification, locking in its preferences over world histories for its future selves. The future selves will then expend all available resources in rushed consumption over the next 100 years. So I would expect the bargaining position of the future to get progressively worse with advancing technology.
Robin,
“That is why we should try to arrange deals across time, where for example, we agree to invest for the future, and they agree to remember and revere us.” Consider an agent that at any time t does not discount benefits received between t and t+1 year, discounts benefits between t+1 years and t+100 years by half, and does not value benefits realized after t+100 years. If the agent is capable of self-modification, then at any particular time it will want to self-modify to replace the variable ‘t’ with a constant, the time of self-modification, locking in its preferences over world histories for its future selves. The future selves will then expend all available resources in rushed consumption over the next 100 years. So I would expect the bargaining position of the future to get progressively worse with advancing technology.