we have the proposal of simulating smaller Universes and less coordinated humans, which makes the AI think that the simulators might be richer and have a better chance of solving alignment
This only matters if the AIs are CDT or dumb about decision theory etc.
I usually defer to you in things like this, but I don’t see why this would be the case. I think the proposal of simulating less competent civilizations is equivalent to the idea of us deciding now, when we don’t really know yet how competent a civilization we are, to bail out less competent alien civilizations in the multiverse if we succeed. In return, we hope that this decision is logically correlated with more competent civilization (who were also unsure in their infancy about how competent they are), deciding to bail out less competent civilizations, including us. My understanding from your comments is that you believe this likely works, how is my proposal of simulating less-coordinated civilizations different?
The story about simulating smaller Universes is more confusing. That would be equivalent to bailing out aliens in smaller Universes for a tiny fraction of our Universe, in the hope that larger Universes also bail us out for a tiny fraction of their Universe. This is very confusing if there are infinite levels of bigger and bigger Universes, I don’t know what to do with infinite ethics. If there are finite levels, but the young civilizations don’t yet have a good prior over the distribution of Universe-sizes, all can reasonably think that there all levels above them, and all their decisions are correlated, so everyone bails out the inhabitants of the smaller Universes, in the hope that they get bailed out by a bigger Universe. Once they learn the correct prior over Universe-sizes, and biggest Universe realizes that no bigger Universe’s actions correlate with theirs, all of this fails (though they can still bail each other out from charity). But this is similar to the previous case, where once the civilizations learn their competence level, the most competent ones are no longer incentivized to enter into insurance contracts, but the hope is that in a sense they enter into a contract while they are still behind the veil of ignorance.
Hmm, maybe I misunderstood your point. I thought you were talking about using simulations to anthropically capture AIs. As in, creating more observer moments where AIs take over less competent civilizations but are actually in a simulation run by us.
If you’re happy to replace “simulation” with “prediction in a way that doesn’t create observer moments” and think the argument goes through either way then I think I agree.
I agree that paying out to less competent civilizations if we find out we’re competent and avoid takeover might be what you should do (as part of a post-hoc insurance deal via UDT or as part of a commitment or whatever). As in, this would help avoid getting killed if you ended up being a less competent civilization.
The smaller thing won’t work exactly for getting us bailed out. I think infinite ethics should be resolvable and end up getting resolved with something roughly similar to some notion of reality-fluid and this implies that you just have to pay more for higher measure places. (Of course people might disagree about the measure etc.)
I’m happy to replace “simulation” with “prediction in a way that doesn’t create observer moments” if we assume we are dealing with UDT agents (which I’m unsure about) and that it’s possible to run accurate predictions about the decisions of complex agents without creating observer moments (which I’m also unsure about). I think running simulations, by some meaning of “simulation” is not really more expensive than getting the accurate predictions, and he cost of running the sims is likely small compared to the size of the payment anyway. So I like talking about running sims, in case we get an AI that takes sims more seriously than prediction-based acausal trade, but I try to pay attention that all my proposals make sense from the perspective of a UDT agent too with predictions instead of simulations. (Exception is the Can we get more than this? proposal which relies on the AI not being UDT, and I agree it’s likely to fail for various reasons, but I decided it was still worth including in the post, in case we get an AI for which this actually works, which I still don’t find that extremely unlikely.)
This only matters if the AIs are CDT or dumb about decision theory etc.
I usually defer to you in things like this, but I don’t see why this would be the case. I think the proposal of simulating less competent civilizations is equivalent to the idea of us deciding now, when we don’t really know yet how competent a civilization we are, to bail out less competent alien civilizations in the multiverse if we succeed. In return, we hope that this decision is logically correlated with more competent civilization (who were also unsure in their infancy about how competent they are), deciding to bail out less competent civilizations, including us. My understanding from your comments is that you believe this likely works, how is my proposal of simulating less-coordinated civilizations different?
The story about simulating smaller Universes is more confusing. That would be equivalent to bailing out aliens in smaller Universes for a tiny fraction of our Universe, in the hope that larger Universes also bail us out for a tiny fraction of their Universe. This is very confusing if there are infinite levels of bigger and bigger Universes, I don’t know what to do with infinite ethics. If there are finite levels, but the young civilizations don’t yet have a good prior over the distribution of Universe-sizes, all can reasonably think that there all levels above them, and all their decisions are correlated, so everyone bails out the inhabitants of the smaller Universes, in the hope that they get bailed out by a bigger Universe. Once they learn the correct prior over Universe-sizes, and biggest Universe realizes that no bigger Universe’s actions correlate with theirs, all of this fails (though they can still bail each other out from charity). But this is similar to the previous case, where once the civilizations learn their competence level, the most competent ones are no longer incentivized to enter into insurance contracts, but the hope is that in a sense they enter into a contract while they are still behind the veil of ignorance.
Hmm, maybe I misunderstood your point. I thought you were talking about using simulations to anthropically capture AIs. As in, creating more observer moments where AIs take over less competent civilizations but are actually in a simulation run by us.
If you’re happy to replace “simulation” with “prediction in a way that doesn’t create observer moments” and think the argument goes through either way then I think I agree.
I agree that paying out to less competent civilizations if we find out we’re competent and avoid takeover might be what you should do (as part of a post-hoc insurance deal via UDT or as part of a commitment or whatever). As in, this would help avoid getting killed if you ended up being a less competent civilization.
The smaller thing won’t work exactly for getting us bailed out. I think infinite ethics should be resolvable and end up getting resolved with something roughly similar to some notion of reality-fluid and this implies that you just have to pay more for higher measure places. (Of course people might disagree about the measure etc.)
I’m happy to replace “simulation” with “prediction in a way that doesn’t create observer moments” if we assume we are dealing with UDT agents (which I’m unsure about) and that it’s possible to run accurate predictions about the decisions of complex agents without creating observer moments (which I’m also unsure about). I think running simulations, by some meaning of “simulation” is not really more expensive than getting the accurate predictions, and he cost of running the sims is likely small compared to the size of the payment anyway. So I like talking about running sims, in case we get an AI that takes sims more seriously than prediction-based acausal trade, but I try to pay attention that all my proposals make sense from the perspective of a UDT agent too with predictions instead of simulations. (Exception is the Can we get more than this? proposal which relies on the AI not being UDT, and I agree it’s likely to fail for various reasons, but I decided it was still worth including in the post, in case we get an AI for which this actually works, which I still don’t find that extremely unlikely.)