It’s the same reason for why we can’t break out of the simulation IRL, except we don’t have to face adversarial cognition, so the AI’s task is even harder than our task.
This seems like it contains several ungrounded claims. Maybe I’m misreading you? But it seems weight-bearing for your overall argument, so I want to clarify.
We may not be in a simulation, in which case not being able to break out is no evidence of the ease of preventing breakout.
If we are in a simulation, then we only know that the simulation has been good enough to keep us in it so far, for a very short time since we even considered the possiblity that we were in a simulation, with barely any effort put into trying to determine whether we’re in one or how to break out. We might find a way to break out once we put more effort into it, or once science and technology advance a bit further, or once we’re a bit smarter than current-human.
We have no idea whether we’re facing adversarial cognition.
If we are in a simulation, it’s probably being run by beings much smarter than human, or at least much more advanced (certainly humans aren’t anywhere remotely close to being able to simulate an entire universe containing billions of sentient minds). For the analogy to hold, the AI would have to be way below human level, and by hypothesis it’s not (since we’re talking about AI smart enough to be dangerous).
If we are in a simulation, it’s probably being run by beings much smarter than human, or at least much more advanced (certainly humans aren’t anywhere remotely close to being able to simulate an entire universe containing billions of sentient minds). For the analogy to hold, the AI would have to be way below human level, and by hypothesis it’s not (since we’re talking about AI smart enough to be dangerous).
Okay, my simulation point was admittedly a bit of colorful analogy, but 1 thing to keep in mind:
We don’t have to simulate entire universes at the level of detail of our universe, and only need to make them realistic enough such that whoever grew up in the simulation crafted by us wouldn’t know it’s a simulation, and given that the AI is likely trained solely through synthetic data, it may well be unable to distinguish between deployment and training at all, especially if AI labs use synthetic data very early in training.
For our purposes, the synthetic data simulation doesn’t need to be all that realistic and can even include exploits/viruses that don’t work in our reality.
Plus, the AI has no reason to elevate the hypothesis that it’s in training/simulation until the very end of training, where a sim-to real phase is implemented to ground them in our reality, because it’s in a world of solely synthetic data, and it has little knowledge of whether something is real or whether we are trying to trick it to reveal something about it’s nature.
Not critical for my response, but to address it for completion purposes
We have no idea whether we’re facing adversarial cognition.
My claim for humans is a conditional claim, in that if the presumed aliens wanted to get rid of us at some point if we broke out of our simulation and had adversarial cognition, humanity just totally loses and it’s game over.
If we are in a simulation, then we only know that the simulation has been good enough to keep us in it so far, for a very short time since we even considered the possiblity that we were in a simulation, with barely any effort put into trying to determine whether we’re in one or how to break out. We might find a way to break out once we put more effort into it, or once science and technology advance a bit further, or once we’re a bit smarter than current-human.
Conditional on us being able to break out of the simulation, it would require massive, massively more advancements in tech before we could do it, and we have no plan of attack on how to break out of the simulation even if it could be done.
We may not be in a simulation, in which case not being able to break out is no evidence of the ease of preventing breakout.
I used to think this way, but I no longer do, because the simulation hypothesis is a non-predictive phenomenon that predicts nothing other than the universe is a computer, which doesn’t restrain expectations at all, since basically everything can be predicted by that hypothesis, and there are theoretical computers which are ridiculously powerful and compute every well-founded set, which means you can’t rule out anything from the hypothesis alone.
One of the reasons I hate simulation hypothesis discourse is people seem to think that it predicts far more than it does:
Okay, my simulation point was admittedly a bit of colorful analogy
Fair enough; if it’s not load-bearing for your view, that’s fine. I do remain skeptical, and can sketch out why if it’s of interest, but feel no particular need to continue.
That’s fine, I was using the word simulation in a much, much looser sense than is usually used, and is closer to @janus’s use of the word simulator/simulation than what physicists/chemists use.
This seems like it contains several ungrounded claims. Maybe I’m misreading you? But it seems weight-bearing for your overall argument, so I want to clarify.
We may not be in a simulation, in which case not being able to break out is no evidence of the ease of preventing breakout.
If we are in a simulation, then we only know that the simulation has been good enough to keep us in it so far, for a very short time since we even considered the possiblity that we were in a simulation, with barely any effort put into trying to determine whether we’re in one or how to break out. We might find a way to break out once we put more effort into it, or once science and technology advance a bit further, or once we’re a bit smarter than current-human.
We have no idea whether we’re facing adversarial cognition.
If we are in a simulation, it’s probably being run by beings much smarter than human, or at least much more advanced (certainly humans aren’t anywhere remotely close to being able to simulate an entire universe containing billions of sentient minds). For the analogy to hold, the AI would have to be way below human level, and by hypothesis it’s not (since we’re talking about AI smart enough to be dangerous).
To address this:
Okay, my simulation point was admittedly a bit of colorful analogy, but 1 thing to keep in mind:
We don’t have to simulate entire universes at the level of detail of our universe, and only need to make them realistic enough such that whoever grew up in the simulation crafted by us wouldn’t know it’s a simulation, and given that the AI is likely trained solely through synthetic data, it may well be unable to distinguish between deployment and training at all, especially if AI labs use synthetic data very early in training.
For our purposes, the synthetic data simulation doesn’t need to be all that realistic and can even include exploits/viruses that don’t work in our reality.
Plus, the AI has no reason to elevate the hypothesis that it’s in training/simulation until the very end of training, where a sim-to real phase is implemented to ground them in our reality, because it’s in a world of solely synthetic data, and it has little knowledge of whether something is real or whether we are trying to trick it to reveal something about it’s nature.
Not critical for my response, but to address it for completion purposes
My claim for humans is a conditional claim, in that if the presumed aliens wanted to get rid of us at some point if we broke out of our simulation and had adversarial cognition, humanity just totally loses and it’s game over.
Conditional on us being able to break out of the simulation, it would require massive, massively more advancements in tech before we could do it, and we have no plan of attack on how to break out of the simulation even if it could be done.
I used to think this way, but I no longer do, because the simulation hypothesis is a non-predictive phenomenon that predicts nothing other than the universe is a computer, which doesn’t restrain expectations at all, since basically everything can be predicted by that hypothesis, and there are theoretical computers which are ridiculously powerful and compute every well-founded set, which means you can’t rule out anything from the hypothesis alone.
One of the reasons I hate simulation hypothesis discourse is people seem to think that it predicts far more than it does:
https://arxiv.org/abs/1806.08747
Fair enough; if it’s not load-bearing for your view, that’s fine. I do remain skeptical, and can sketch out why if it’s of interest, but feel no particular need to continue.
That’s fine, I was using the word simulation in a much, much looser sense than is usually used, and is closer to @janus’s use of the word simulator/simulation than what physicists/chemists use.
We can discuss this another time.