Put baldly, the main underlying question is : how do you compare the value of
(a) a unit of work expended now, today, on the well-being of a person alive, now, today,
with the value of
(b) the same unit of work expended now, today, for the well-being of 500 potential people who might be alive in 500 years’ time,
given that units of work are in limited supply.
I suspect any attempt at a mathematical answer to that would only be an expression of a subjective emotional preference.
What is more, the mathematical answer wouldn’t be a discount function, it would be a compounding function, as it would be the result of comparing all the AI units of work available between now and time t in the future, with the units of work required between now and time t to address all the potential needs of humanity and trans-humanity between now and the end of time, which looks seriously like infinity.
Put baldly, the main underlying question is : how do you compare the value of (a) a unit of work expended now, today, on the well-being of a person alive, now, today, with the value of (b) the same unit of work expended now, today, for the well-being of 500 potential people who might be alive in 500 years’ time, given that units of work are in limited supply. I suspect any attempt at a mathematical answer to that would only be an expression of a subjective emotional preference. What is more, the mathematical answer wouldn’t be a discount function, it would be a compounding function, as it would be the result of comparing all the AI units of work available between now and time t in the future, with the units of work required between now and time t to address all the potential needs of humanity and trans-humanity between now and the end of time, which looks seriously like infinity.