Maybe this is analagous to Newcomb’s problem. Maybe not.
It’s different. If the reward isn’t determined by the decision the agent makes, but instead by how the agent made that decision, it isn’t a “decision-determined problem” anymore. That’s why I’ve been using that phrase. TDT is only generally good for decision-determined problems. Newcomb’s problem is a decision-determined problem, which is important because it doesn’t set out to expressly reward some type of agent; it’s fair.
It’s that the strategy itself is to find the optimal solution, for some value of optimal.
But all these “local optimal solutions” can be measured on the same scale, e.g. dollars. And so if the decision theory is just an intermediary—if what we really want is a dollar-maximizing agent, or a game-winning agent, we can compare different decision theories along a common yardstick. The best decision theories will be the ones that dominate all the others within a certain class of problems—they do as well or better than all other decision theories on every single problem of that class. This quickly becomes impossible for larger classes of problems, but can be made possible again by Occamian constraints like symmetry.
It’s different. If the reward isn’t determined by the decision the agent makes, but instead by how the agent made that decision, it isn’t a “decision-determined problem” anymore. That’s why I’ve been using that phrase. TDT is only generally good for decision-determined problems. Newcomb’s problem is a decision-determined problem, which is important because it doesn’t set out to expressly reward some type of agent; it’s fair.
But all these “local optimal solutions” can be measured on the same scale, e.g. dollars. And so if the decision theory is just an intermediary—if what we really want is a dollar-maximizing agent, or a game-winning agent, we can compare different decision theories along a common yardstick. The best decision theories will be the ones that dominate all the others within a certain class of problems—they do as well or better than all other decision theories on every single problem of that class. This quickly becomes impossible for larger classes of problems, but can be made possible again by Occamian constraints like symmetry.