In Newcomb’s paradox, the predictor only cares about what you do when presented with the boxes. It doesn’t care about whether that’s because you use ADT, BDT, or anything else. Whereas Dick Kick’em has to actually look at your source code and base his decision on that. He might as well be deciding based on any arbitrary property of the code, like whether it uses tabs or spaces, rather than what decision theory it implements. (Actually, the tabs and spaces thing sounds more plausible! Determining what kind of decision theory a given piece of code implements seems like it could be equivalent to solving the halting problem.) Agents that care about what you’ll do in various situations seem much more common and relevant than agents that arbitrarily reward you for being designed a certain way.
Newcomb’s problem is analogous to many real world situations. (For example: I want to trade with a person, but not if they’re going to rip me off. But as soon as I agree to trade with them, that grants them the opportunity to rip me off with no consequences. Thus, unless they’re predictably not going to do that, I had better not trade with them.) It’s not clear what real world situations (if any) Dick Kick’em is analogous to. Religious wars?
Why can’t I use the strategy “pick one or two boxes in a way affected by what decision theories the predictor approves of”? Or more likely, something which is mathematically equivalent to doing that, but doesn’t do an explicit comparison?
I believe that the AI does care about your beliefs, just not specific beliefs. The AI only cares about if your decision theory falls into the class of decision theories that will pick two boxes, and if it does then it punishes you. Sure, unlike Dick Kick’em the AI isn’t looking for specific theories just any theory within a specific class, but it is still the same thing. The AI is punishing the agent by putting in less money based soley on your beliefs. In Newcomb’s paradox, the AI scans your brain BEFORE you take any action whatsoever, the punishment cannot be based on your actions, the punishment from the AI is based only on your beliefs. This is exactly the same as the Dick Kick’em Paradox; Dick will punish you purely on your beliefs, not any action. The only difference is that in Newcomb’s paradox you get to play a little game after the AI has punished you.
Eh, Omega only cares about your beliefs insofar as they affect your actions (past, present, or future, it’s all just a different t coordinate). I still think that seems way more natural and common than caring about beliefs in general.
Example: Agent A goes around making death threats, saying to people: “Give me $200 or I’m going to kill you.” Agent B goes around handing out brochures that criticize the government. If the police arrest agent A, that’s probably a reasonable decision. If the police arrest agent B, that’s bad and authoritarian, since it goes against freedom of expression. This is true even though all either agent has done is say things. Agent A hasn’t actually killed anyone yet. But the police still arrest agent A because they care about agent A’s future actions.
“How dare you infer my future actions from what I merely say,” cries agent A as they’re being handcuffed, “you’re arbitrarily punishing me for what I believe. This is a crass violation of my right to freedom of speech.” The door of the police car slams shut and further commentary is inaudible.
Omega only cares about your beliefs insofar as they affect your actions
So does Dick Kick’em, since he only cares about distinct decision theories that a particular agent believes in, and that in turn decides the agent’s actions.
In Newcomb’s paradox, the predictor only cares about what you do when presented with the boxes. It doesn’t care about whether that’s because you use ADT, BDT, or anything else. Whereas Dick Kick’em has to actually look at your source code and base his decision on that. He might as well be deciding based on any arbitrary property of the code, like whether it uses tabs or spaces, rather than what decision theory it implements. (Actually, the tabs and spaces thing sounds more plausible! Determining what kind of decision theory a given piece of code implements seems like it could be equivalent to solving the halting problem.) Agents that care about what you’ll do in various situations seem much more common and relevant than agents that arbitrarily reward you for being designed a certain way.
Newcomb’s problem is analogous to many real world situations. (For example: I want to trade with a person, but not if they’re going to rip me off. But as soon as I agree to trade with them, that grants them the opportunity to rip me off with no consequences. Thus, unless they’re predictably not going to do that, I had better not trade with them.) It’s not clear what real world situations (if any) Dick Kick’em is analogous to. Religious wars?
Why can’t I use the strategy “pick one or two boxes in a way affected by what decision theories the predictor approves of”? Or more likely, something which is mathematically equivalent to doing that, but doesn’t do an explicit comparison?
I believe that the AI does care about your beliefs, just not specific beliefs. The AI only cares about if your decision theory falls into the class of decision theories that will pick two boxes, and if it does then it punishes you. Sure, unlike Dick Kick’em the AI isn’t looking for specific theories just any theory within a specific class, but it is still the same thing. The AI is punishing the agent by putting in less money based soley on your beliefs. In Newcomb’s paradox, the AI scans your brain BEFORE you take any action whatsoever, the punishment cannot be based on your actions, the punishment from the AI is based only on your beliefs. This is exactly the same as the Dick Kick’em Paradox; Dick will punish you purely on your beliefs, not any action. The only difference is that in Newcomb’s paradox you get to play a little game after the AI has punished you.
Eh, Omega only cares about your beliefs insofar as they affect your actions (past, present, or future, it’s all just a different t coordinate). I still think that seems way more natural and common than caring about beliefs in general.
Example: Agent A goes around making death threats, saying to people: “Give me $200 or I’m going to kill you.” Agent B goes around handing out brochures that criticize the government. If the police arrest agent A, that’s probably a reasonable decision. If the police arrest agent B, that’s bad and authoritarian, since it goes against freedom of expression. This is true even though all either agent has done is say things. Agent A hasn’t actually killed anyone yet. But the police still arrest agent A because they care about agent A’s future actions.
“How dare you infer my future actions from what I merely say,” cries agent A as they’re being handcuffed, “you’re arbitrarily punishing me for what I believe. This is a crass violation of my right to freedom of speech.” The door of the police car slams shut and further commentary is inaudible.
So does Dick Kick’em, since he only cares about distinct decision theories that a particular agent believes in, and that in turn decides the agent’s actions.