I think I might be missing something, because the argument you attribute to Dávid still looks wrong to me. You say:
The entropy of the simulators’ distribution need not be more than the entropy of the (square of the) wave function in any relevant sense. Despite the fact that subjective entropy may be huge, physical entropy is still low (because the simulations happen on a high-amplitude ridge of the wave function, after all).
Doesn’t this argument imply that the supermajority of simulations within the simulators’ subjective distribution over universe histories are not instantiated anywhere within the quantum multiverse?
I think it does. And, if you accept this, then (unless for some reason you think the simulators’ choice of which histories to instantiate is biased towards histories that correspond to other “high-amplitude ridges” of the wave function, which makes no sense because any such bias should have already been encoded within the simulators’ subjective distribution over universe histories) you should also expect, a priori, that the simulations instantiated by the simulators should not be indistinguishable from physical reality, because such simulations comprise a vanishingly small proportion of the simulators’ subjective probability distribution over universe histories.
What this in turn means, however, is that prior to observation, a Solomonoff inductor (SI) must spread out much of its own subjective probability mass across hypotheses that predict finding itself within a noticeably simulated environment. Those are among the possibilities it must take into account—meaning, if you stipulate that it doesn’t find itself in an environment corresponding to any of those hypotheses, you’ve ruled out all of the “high-amplitude ridges” corresponding to instantiated simulations in the crossent of the simulators’ subjective distribution and reality’s distribution.
We can make this very stark: suppose our SI finds itself in an environment which, according to its prior over the quantum multiverse, corresponds to one high-amplitude ridge of the physical wave function, and zero high-amplitude ridges containing simulators that happened to instantiate that exact environment (either because no branches of the quantum multiverse happened to give rise to simulators that would have instantiated that environment, or because the environment in question simply wasn’t a member of any simulators’ subjective distributions over reality to begin with). Then the SI would immediately (correctly) conclude that it cannot be in a simulation.
Now, of course, the argument as I’ve presented it here is heavily reliant on the idea of our SI being an SI, in such a way that it’s not clear how exactly the argument carries over to the logically non-omniscient case. In particular, it relies on the SI being capable of discerning differences between very good simulations and perfect simulations, a feat which bounded reasoners cannot replicate; and it relies on the notion that our inability as bounded reasoners to distinguish between hypotheses at this level of granularity is best modeled in the SI case by stipulating that the SI’s actual observations are in fact consistent with its being instantiated within a base-level, high-amplitude ridge of the physical wave function—i.e. that our subjective inability to tell whether we’re in a simulation should be viewed as analogous to an SI being unable to tell whether it’s in a simulation because its observations actually fail to distinguish. I think this is the relevant analogy, but I’m open to being told (by you or by Dávid) why I’m wrong.
I agree that in real life the entropy argument is an argument in favor of it being actually pretty hard to fool a superintelligence into thinking it might be early in Tegmark III when it’s not (even if you yourself are a superintelligence, unless you’re doing a huge amount of intercepting its internal sanity checks (which puts significant strain on the trade possibilities and which flirts with being a technical-threat)). And I agree that if you can’t fool a superintelligence into thinking it might be early in Tegmark III when it’s not, then the purchasing power of simulators drops dramatically, except in cases where they’re trolling local aliens. (But the point seems basically moot, as ‘troll local aliens’ is still an option, and so afaict this does all essentially iron out to “maybe we’ll get sold to aliens”.)
I think I might be missing something, because the argument you attribute to Dávid still looks wrong to me. You say:
Doesn’t this argument imply that the supermajority of simulations within the simulators’ subjective distribution over universe histories are not instantiated anywhere within the quantum multiverse?
I think it does. And, if you accept this, then (unless for some reason you think the simulators’ choice of which histories to instantiate is biased towards histories that correspond to other “high-amplitude ridges” of the wave function, which makes no sense because any such bias should have already been encoded within the simulators’ subjective distribution over universe histories) you should also expect, a priori, that the simulations instantiated by the simulators should not be indistinguishable from physical reality, because such simulations comprise a vanishingly small proportion of the simulators’ subjective probability distribution over universe histories.
What this in turn means, however, is that prior to observation, a Solomonoff inductor (SI) must spread out much of its own subjective probability mass across hypotheses that predict finding itself within a noticeably simulated environment. Those are among the possibilities it must take into account—meaning, if you stipulate that it doesn’t find itself in an environment corresponding to any of those hypotheses, you’ve ruled out all of the “high-amplitude ridges” corresponding to instantiated simulations in the crossent of the simulators’ subjective distribution and reality’s distribution.
We can make this very stark: suppose our SI finds itself in an environment which, according to its prior over the quantum multiverse, corresponds to one high-amplitude ridge of the physical wave function, and zero high-amplitude ridges containing simulators that happened to instantiate that exact environment (either because no branches of the quantum multiverse happened to give rise to simulators that would have instantiated that environment, or because the environment in question simply wasn’t a member of any simulators’ subjective distributions over reality to begin with). Then the SI would immediately (correctly) conclude that it cannot be in a simulation.
Now, of course, the argument as I’ve presented it here is heavily reliant on the idea of our SI being an SI, in such a way that it’s not clear how exactly the argument carries over to the logically non-omniscient case. In particular, it relies on the SI being capable of discerning differences between very good simulations and perfect simulations, a feat which bounded reasoners cannot replicate; and it relies on the notion that our inability as bounded reasoners to distinguish between hypotheses at this level of granularity is best modeled in the SI case by stipulating that the SI’s actual observations are in fact consistent with its being instantiated within a base-level, high-amplitude ridge of the physical wave function—i.e. that our subjective inability to tell whether we’re in a simulation should be viewed as analogous to an SI being unable to tell whether it’s in a simulation because its observations actually fail to distinguish. I think this is the relevant analogy, but I’m open to being told (by you or by Dávid) why I’m wrong.
I agree that in real life the entropy argument is an argument in favor of it being actually pretty hard to fool a superintelligence into thinking it might be early in Tegmark III when it’s not (even if you yourself are a superintelligence, unless you’re doing a huge amount of intercepting its internal sanity checks (which puts significant strain on the trade possibilities and which flirts with being a technical-threat)). And I agree that if you can’t fool a superintelligence into thinking it might be early in Tegmark III when it’s not, then the purchasing power of simulators drops dramatically, except in cases where they’re trolling local aliens. (But the point seems basically moot, as ‘troll local aliens’ is still an option, and so afaict this does all essentially iron out to “maybe we’ll get sold to aliens”.)