Well, the only absolute guarantee the AI can make is that the underlying reality supports computation.
But it can still probabilistically infer other things about it. Specifically, the AI knows not only that the underlying reality supports computation, but also that there was some underlying process that actually created the simulation it’s in. Even though Conway’s Game of Life can allow for arbitrary computation, many possible configurations of the world state would result in no AI simulations being made. The configurations that would result in AI simulations being made would likely involve some sort of intelligent civilization creating the simulations. So the AI could potentially predict the existence of this civilization and infer some things about it.
Regardless, even if the AI can’t infer anything else about outside reality, I don’t see how this is a fault of not having a notion of base-level reality. I mean, if you’re correct, then it’s not clear to me how an AI with a notion of base-level reality would do inferentially better.
How can it conceive of the idea of percepts misleading about reality if it literally can’t conceive of any distinction between models (which are a special case of abstractions) and reality?
Well, as I said before, the AI could still consider the possibility that the world is composed entirely of hats (minus the AI simulation). The AI could also have a model of Bayesian inference and infer that the Bayesian probability that would be rational to assign to “the world is all hats” is low and its evidence makes it even lower. So, by combining these two models, the AI can come up with a model that says, “The world is all hats, even though everything I’ve seen, according to probability theory, makes it seem like this isn’t the case”. That sounds like a model about the idea of percepts misleading about reality.
I know we’ve been going back and forth a lot, but I think these are pretty interesting things to talk about, so I thank you for the discussion.
It might help if you try to describe a specific situation in which the AI makes the wrong prediction or takes the wrong action for its goals. This could help be better understand what you’re thinking about.
Well, as I said before, the AI could still consider the possibility that the world is composed entirely of hats (minus the AI simulation).
At this point I’m not sure there’s much point in discussing further. You’re using words in ways that seem self-contradictory to me.
You said “the AI could still consider the possibility that the world is composed of [...]”. Considering a possibility is creating a model. Models can be constructed about all sorts of things: mathematical statements, future sensory inputs, hypothetical AIs in simulated worlds, and so on. In this case, the AI’s model is about “the world”, that is to say, reality.
So it is using a concept of model, and a concept of reality. It is only considering the model as a possibility, so it knows that not everything true in the model is automatically true in reality and vice versa. Therefore it is distinguishing between them. But you posited that it can’t do that.
To me, this is a blatant contradiction. My model of you is that you are unlikely to post blatant contradictions, so I am left with the likelihood that what you mean by your statements is wholly unlike the meaning I assign to the same statements. This does not bode well for effective communication.
Yeah, it might be best to wrap up the discussion. It seems we aren’t really understanding what the other means.
So it is using a concept of model, and a concept of reality. It is only considering the model as a possibility, so it knows that not everything true in the model is automatically true in reality and vice versa. Therefore it is distinguishing between them. But you posited that it can’t do that.
Well, I can’t say I’m really following you there. The AI would still have a notion of reality. It just would consider abstractions like chairs and tables to be part of reality.
There is one thing I want to say though. We’ve been discussing the question of if a notion of base-level reality is necessary to avoid severe limitations in reasoning ability. And to see why I think it’s not, just consider regular humans. They often don’t have a distinction between base-level reality and abstractions. And yet, they can still reason about the possibility of life-long illusions as well as function well to accomplish their goals. And if you taught someone the concept of “base-level reality”, I’m not sure it would help them much.
Well, the only absolute guarantee the AI can make is that the underlying reality supports computation.
But it can still probabilistically infer other things about it. Specifically, the AI knows not only that the underlying reality supports computation, but also that there was some underlying process that actually created the simulation it’s in. Even though Conway’s Game of Life can allow for arbitrary computation, many possible configurations of the world state would result in no AI simulations being made. The configurations that would result in AI simulations being made would likely involve some sort of intelligent civilization creating the simulations. So the AI could potentially predict the existence of this civilization and infer some things about it.
Regardless, even if the AI can’t infer anything else about outside reality, I don’t see how this is a fault of not having a notion of base-level reality. I mean, if you’re correct, then it’s not clear to me how an AI with a notion of base-level reality would do inferentially better.
I know we’ve been going back and forth a lot, but I think these are pretty interesting things to talk about, so I thank you for the discussion.
It might help if you try to describe a specific situation in which the AI makes the wrong prediction or takes the wrong action for its goals. This could help be better understand what you’re thinking about.
At this point I’m not sure there’s much point in discussing further. You’re using words in ways that seem self-contradictory to me.
You said “the AI could still consider the possibility that the world is composed of [...]”. Considering a possibility is creating a model. Models can be constructed about all sorts of things: mathematical statements, future sensory inputs, hypothetical AIs in simulated worlds, and so on. In this case, the AI’s model is about “the world”, that is to say, reality.
So it is using a concept of model, and a concept of reality. It is only considering the model as a possibility, so it knows that not everything true in the model is automatically true in reality and vice versa. Therefore it is distinguishing between them. But you posited that it can’t do that.
To me, this is a blatant contradiction. My model of you is that you are unlikely to post blatant contradictions, so I am left with the likelihood that what you mean by your statements is wholly unlike the meaning I assign to the same statements. This does not bode well for effective communication.
Yeah, it might be best to wrap up the discussion. It seems we aren’t really understanding what the other means.
Well, I can’t say I’m really following you there. The AI would still have a notion of reality. It just would consider abstractions like chairs and tables to be part of reality.
There is one thing I want to say though. We’ve been discussing the question of if a notion of base-level reality is necessary to avoid severe limitations in reasoning ability. And to see why I think it’s not, just consider regular humans. They often don’t have a distinction between base-level reality and abstractions. And yet, they can still reason about the possibility of life-long illusions as well as function well to accomplish their goals. And if you taught someone the concept of “base-level reality”, I’m not sure it would help them much.