A simple way of rating the scenarios above is to describe them as you have and ask humans what they think.
In a way… but I expect that what we actually need to solve is just how to make a narrow AI faithfully generate AI papers and AI safety papers that humans would have come up with given time.
The CEV paper has gone into this, but indeed human utility functions will have to be aggregated in some manner, and the manner in which to do this and allocate ressources can’t be derived from first principles. Fortunately human utility functions are logarithmic enough and enough people care about enough other people that the basin of acceptable solutions is quite large, especially if we get the possible future AIs to acausally trade with each other.
The universe is finite, and has to be distributed in some manner.
Some people prefer interactions with the people alive today to ones with heavenly replicas of them. You might claim that there is no difference, but I say that in the end it’s all atoms, all the meaning is made up anyway, and we know exactly why those people would not approve if we described virtual heavens to them so we shouldn’t just do them anyway.
Some people care about what other people do in their virtual heavens. You could deontologically tell them to fuck off, but I’d expect the model of dictator lottery + acausal trade to arrive at another solution.
I see this not as a question to ask now, but later, on many levels of detail, when the omnipotent singleton is deciding what to do with the world. Of course we will have to figure out the correct way to pose such questions before deployment, but this can be deferred until we can generate research.
A simple way of rating the scenarios above is to describe them as you have and ask humans what they think.
In a way… but I expect that what we actually need to solve is just how to make a narrow AI faithfully generate AI papers and AI safety papers that humans would have come up with given time.
The CEV paper has gone into this, but indeed human utility functions will have to be aggregated in some manner, and the manner in which to do this and allocate ressources can’t be derived from first principles. Fortunately human utility functions are logarithmic enough and enough people care about enough other people that the basin of acceptable solutions is quite large, especially if we get the possible future AIs to acausally trade with each other.
I do not see why that should be the case? Assuming virtual heavens, why couldn’t each individuals personal preferences be fullfilled?
The universe is finite, and has to be distributed in some manner.
Some people prefer interactions with the people alive today to ones with heavenly replicas of them. You might claim that there is no difference, but I say that in the end it’s all atoms, all the meaning is made up anyway, and we know exactly why those people would not approve if we described virtual heavens to them so we shouldn’t just do them anyway.
Some people care about what other people do in their virtual heavens. You could deontologically tell them to fuck off, but I’d expect the model of dictator lottery + acausal trade to arrive at another solution.
Do you think this is worth doing?
I thought that
either this was done a billion times and I just missed it
or this is neither important nor interesting to anyone but me
I see this not as a question to ask now, but later, on many levels of detail, when the omnipotent singleton is deciding what to do with the world. Of course we will have to figure out the correct way to pose such questions before deployment, but this can be deferred until we can generate research.