I really feel like “you aren’t in a superintelligent reality warper’s simulation” is kind of the assumption that’s necessary for any AI safety proposal to work.
If we assume no limits, then the whole project becomes pointless.
I’m not sure I completely understand your comment.
If you are talking about us actually living in a simulation, Vanessa doesn’t say “maybe we live in a simulation, and then the AGI will notice”. She says, “independently of the epistemological status and empirical credence of simulation hypotheses, the AGI’s model might converge on them (because of the way in which we punish complexity, which is necessary to arrive at the laws of Physics), and this is a problem”.
If on the contrary you are talking about instilling into the AGI the assumption that simulation hypotheses are false, then this would be great but we can’t do it easily, because of the problem of ontology identification and other complications. Or in other words, how would you specify which reality counts as a simulation?
Specifically, if there’s a high chance we are in an adversary’s simulation, that’s equivalent to showing that you can’t actually win. We are no more able to deal with such simulators than we are able to deal with real life Avengers or Justice League coming from the comics to attack us.
Thus, the exercise is pointless: no AI safety proposal could survive such forces.
You might not have understood my above comment. A simulation hypothesis having high credence (let alone being the case) is not necessary for acausal attacks to be a problem for PreDCA. That is, this worry is independent of whether we actually live in a simulation (and whether you know that).
I really feel like “you aren’t in a superintelligent reality warper’s simulation” is kind of the assumption that’s necessary for any AI safety proposal to work.
If we assume no limits, then the whole project becomes pointless.
I’m not sure I completely understand your comment.
If you are talking about us actually living in a simulation, Vanessa doesn’t say “maybe we live in a simulation, and then the AGI will notice”. She says, “independently of the epistemological status and empirical credence of simulation hypotheses, the AGI’s model might converge on them (because of the way in which we punish complexity, which is necessary to arrive at the laws of Physics), and this is a problem”.
If on the contrary you are talking about instilling into the AGI the assumption that simulation hypotheses are false, then this would be great but we can’t do it easily, because of the problem of ontology identification and other complications. Or in other words, how would you specify which reality counts as a simulation?
Specifically, if there’s a high chance we are in an adversary’s simulation, that’s equivalent to showing that you can’t actually win. We are no more able to deal with such simulators than we are able to deal with real life Avengers or Justice League coming from the comics to attack us.
Thus, the exercise is pointless: no AI safety proposal could survive such forces.
You might not have understood my above comment. A simulation hypothesis having high credence (let alone being the case) is not necessary for acausal attacks to be a problem for PreDCA. That is, this worry is independent of whether we actually live in a simulation (and whether you know that).
Thank you for clarifying things, since I got pretty confused on the acausal attack issue.