Yeah, I’ve known since my first post on formalizing UDT that counterfactuals are about what the agent will prove, not what’s true. I think of them as a complicated but finite tree of tendrils, representing the theorems that the agent will prove before making a decision, living inside the infinite tree of all theorems provable from the agent’s axioms. All the interesting questions are about the shape of the tree, given that the agent can affect its shape by exploring more in one direction or the other. I’d love to understand it better.
I’ve known since my first post on formalizing UDT that counterfactuals are about what the agent will prove, not what’s true.
What’s “true”? If it’s things that can be knowledge, proofs are affected. Even observations may affect proof strategies, as actions in a world where you get particular observations may be monstrously more efficient to find if you take observations into account. You’d just want to coordinate with other versions of yourself that observed differently and used different proof strategies.
Yeah, I’ve spent the last two weeks making slowly making my way through the posts on Ambient Decision Theory. I found that absolutely fascinating, but rather difficult for me to get my head around. I guess my question was more about whether there are any other simple scenarios that demonstrate that logical counterfactuals are more about your state of knowledge than the physical state of the universe. I think that would help me understand what exactly is going on better. This one took me much longer than you’d expect to construct.
What in particular would you love to understand better?
My current guess is that logical counterfactuals are a wrong thing to focus on in decision making. They have their place in figuring out imperfect reasoning about yourself, but that’s specific to coordination with imperfect reasoners of a particular kind, not decision making itself (so should probably be done on meta level, as a symbol pushing game not working on objects of knowledge for the agent).
A trick that works well for that (I’m not sure it was much discussed) is to aim to prove that you win, given provability (validity in form of box modality) of a decision you can make, instead of aiming to discover consequences of a decision (discovering and exploiting dependencies). This way, you don’t deal with problematic consequences of non-actual decisions, only with the enacted requirements for an actual proof of winning. This also abstracts away from utility maximization, so there is just winning and not winning (and the issue of how to get back to maximization is more difficult than in the dependency-based approaches, it becomes more like bargaining).
I’m having difficulty following this comment. I don’t know what you mean by a “symbol pushing game”. Also, does box modality simply refer to modal logic?
Anyway, re: problems like Perfect Parfit’s Hitchhiker, proving you win by paying still requires you to define what the predictor actually predicts. Otherwise we can’t say that a never paying agent doesn’t end up in town and hence win. So I don’t understand how this avoids the “problematic consequences of non-actual decisions”.
No one “knows” this. You can come up with a DT that optimises metaphysical neutrality, if youre interested. but it won’t optimise utility. To optimise utility, you have to find the DT that is the best fit for your universe. But deciding that you value X over Y is not discovering a fact.
Yeah, I’ve known since my first post on formalizing UDT that counterfactuals are about what the agent will prove, not what’s true. I think of them as a complicated but finite tree of tendrils, representing the theorems that the agent will prove before making a decision, living inside the infinite tree of all theorems provable from the agent’s axioms. All the interesting questions are about the shape of the tree, given that the agent can affect its shape by exploring more in one direction or the other. I’d love to understand it better.
What’s “true”? If it’s things that can be knowledge, proofs are affected. Even observations may affect proof strategies, as actions in a world where you get particular observations may be monstrously more efficient to find if you take observations into account. You’d just want to coordinate with other versions of yourself that observed differently and used different proof strategies.
Yeah, I’ve spent the last two weeks making slowly making my way through the posts on Ambient Decision Theory. I found that absolutely fascinating, but rather difficult for me to get my head around. I guess my question was more about whether there are any other simple scenarios that demonstrate that logical counterfactuals are more about your state of knowledge than the physical state of the universe. I think that would help me understand what exactly is going on better. This one took me much longer than you’d expect to construct.
What in particular would you love to understand better?
My current guess is that logical counterfactuals are a wrong thing to focus on in decision making. They have their place in figuring out imperfect reasoning about yourself, but that’s specific to coordination with imperfect reasoners of a particular kind, not decision making itself (so should probably be done on meta level, as a symbol pushing game not working on objects of knowledge for the agent).
A trick that works well for that (I’m not sure it was much discussed) is to aim to prove that you win, given provability (validity in form of box modality) of a decision you can make, instead of aiming to discover consequences of a decision (discovering and exploiting dependencies). This way, you don’t deal with problematic consequences of non-actual decisions, only with the enacted requirements for an actual proof of winning. This also abstracts away from utility maximization, so there is just winning and not winning (and the issue of how to get back to maximization is more difficult than in the dependency-based approaches, it becomes more like bargaining).
I’m having difficulty following this comment. I don’t know what you mean by a “symbol pushing game”. Also, does box modality simply refer to modal logic?
Anyway, re: problems like Perfect Parfit’s Hitchhiker, proving you win by paying still requires you to define what the predictor actually predicts. Otherwise we can’t say that a never paying agent doesn’t end up in town and hence win. So I don’t understand how this avoids the “problematic consequences of non-actual decisions”.
No one “knows” this. You can come up with a DT that optimises metaphysical neutrality, if youre interested. but it won’t optimise utility. To optimise utility, you have to find the DT that is the best fit for your universe. But deciding that you value X over Y is not discovering a fact.
What do you mean by “metaphysical neutrality”?
That requires the minimum number of assumptions about the world.