But I didn’t understand what you meant in the paragraph starting with “What stops us from just saying...” What does stop us from just saying this, and how come some desires successfully result in action and others result in wishful thinking? Can you predict when wishful thinking would be more likely to occur?
On a similar note, if “the final goal of a plan is a belief”, would you expect me to be indifferent between saving the world and taking a pill that caused me to believe that the world was saved, or is that confusing levels?
The algorithm you use to build your plan won’t let you believe a step in the plan is successful until you can satisfy its preconditions. The problem is that “satisfy its preconditions” can be done in a one-sided, non-Bayesian manner, which doesn’t work as well for inference as for action.
Re. the pill—that’s a good question. To avoid taking the pill, you’d need to have a representation that distinguishes between causing X and causing believes(X), from the viewpoint of an outside observer. What I said in the post needs to be revised or clarified to account for this.
Your goal is X, a truth in the external world. When constructing a plan, you operate in a belief space representing the external world’s viewpoint. You predict a plan will be successful if, in simulating it, you find it leads to the assertion X within that simulated belief space; not if it leads to finding believes(you, X) there. believes(you, X) in that belief space maps to X in your “root” belief space (which I’ll call your mind); X in that belief space maps to X being true in the external world.
Successfully executing that plan would result in finding X in your mind. To an external observer, the X in your mind means believes(you, X), not X. That’s because, to that observer, your mind is a belief space, just like the belief space you use when simulating a plan.
To represent the pill-taking action this way:
A = action(eat(me, pill)), precondition(A, have(me, pill)) , consequence(A, goal).
is not right, because that represents that you believe that eating the pill makes X true in the external world.
At first, it appears that it would also be wrong to represent it as
consequence(A, believes(me, goal))
because it appears that eating the pill would cause you to add believes(me, X) instead of X to your knowledge base, whereas you actually will add X.
However! Your inference engine is not the world. The representation in your mind, “consequence(A, believes(me, goal))”, is not what the world actually uses to compute the results if you eat the pill. It’s easy to forget this, because so often we write simulated worlds where we use one and the same rule set both for our agents to reason with, and also for the simulator to compute the next world state. So it’s fine to use this representation.
Definitely makes some sense.
But I didn’t understand what you meant in the paragraph starting with “What stops us from just saying...” What does stop us from just saying this, and how come some desires successfully result in action and others result in wishful thinking? Can you predict when wishful thinking would be more likely to occur?
On a similar note, if “the final goal of a plan is a belief”, would you expect me to be indifferent between saving the world and taking a pill that caused me to believe that the world was saved, or is that confusing levels?
The algorithm you use to build your plan won’t let you believe a step in the plan is successful until you can satisfy its preconditions. The problem is that “satisfy its preconditions” can be done in a one-sided, non-Bayesian manner, which doesn’t work as well for inference as for action.
Re. the pill—that’s a good question. To avoid taking the pill, you’d need to have a representation that distinguishes between causing X and causing believes(X), from the viewpoint of an outside observer. What I said in the post needs to be revised or clarified to account for this.
Your goal is X, a truth in the external world. When constructing a plan, you operate in a belief space representing the external world’s viewpoint. You predict a plan will be successful if, in simulating it, you find it leads to the assertion X within that simulated belief space; not if it leads to finding believes(you, X) there. believes(you, X) in that belief space maps to X in your “root” belief space (which I’ll call your mind); X in that belief space maps to X being true in the external world.
Successfully executing that plan would result in finding X in your mind. To an external observer, the X in your mind means believes(you, X), not X. That’s because, to that observer, your mind is a belief space, just like the belief space you use when simulating a plan.
To represent the pill-taking action this way:
A = action(eat(me, pill)), precondition(A, have(me, pill)) , consequence(A, goal).
is not right, because that represents that you believe that eating the pill makes X true in the external world.
At first, it appears that it would also be wrong to represent it as
consequence(A, believes(me, goal))
because it appears that eating the pill would cause you to add believes(me, X) instead of X to your knowledge base, whereas you actually will add X.
However! Your inference engine is not the world. The representation in your mind, “consequence(A, believes(me, goal))”, is not what the world actually uses to compute the results if you eat the pill. It’s easy to forget this, because so often we write simulated worlds where we use one and the same rule set both for our agents to reason with, and also for the simulator to compute the next world state. So it’s fine to use this representation.