it’s only when you have the circularity of a program whose output might depend on its output that you need to beware this kind of thing.
Well, the substitutions are specifically to turn a circularity into a case of having x on both sides of some equation. We might be talking about different things. The failure mode is benign; you arrive at x=x .
edit: ahh, another thing. If you have source of randomness, you need to consider the solution with, and without, the substitution, as you can make substitution invalid by employing the random number generator. The substitution of the nonrandom part of strategy can still be useful though. Maybe that’s what you had in mind?
If you have source of randomness, you need to consider the solution with, and without, the substitution, as you can make substitution invalid by employing the random number generator.
Err, I’m not sure what you mean here. In the CDT algorithm, if it deduces that Y employs a particular mixed strategy, then it can calculate the expected value of each action against that mixed strategy.
(For complete simplicity, though, starting next post I’m going to assume that there’s at least one pure Nash equilibrium option in G. If it doesn’t start with one, we can treat a mixed equilibrium as x{n+1} and y{m+1}, and fill in the new row and column of the matrix with the right expected values.)
The calculator computes “What is 2 + 3?”, not “What does this calculator compute as the result of 2 + 3?” The answer to the former question is 5, but if the calculator were to ask the latter question instead, the result could self-consistently be anything at all! If the calculator returned 42, then indeed, “What does this calculator compute as the result of 2 + 3?” would in fact be 42.
That’s why you don’t let your calculator be sentient. FAI might give a number that makes you most happy, which might well be 42 if you are not relying on this number for anything useful. (E.g. it might tell 42 as a joke, knowing that you know what 2+3 is)
Edit: you can, however, have some requirements on the calculator’s output, and then there will be the number that satisfies those criteria; the x substitution will work to solve for this value, and in principle even to solve for protective measures to take against cosmic rays, and so on.
edit: and on the NDT, it doesn’t one-way substitute at start. It assumes equivalence.
Sure, if it happened to be in a situation where the most valuable thing to do by my standards was make me happy. Agreed. You seem to be implying that I should prefer to avoid this result… do you in fact believe that? If so, can you clarify why?
A somewhat analogous real-world situation: one of Siri’s possible responses to “open the pod bay doors” as a command is “We’re never going to live that down, are we?” This delights me enormously, and costs nothing of consequence. Should I prefer that this result be eliminated?
Actually I misunderstood his point with calculator. He was speaking of NDT with issues resulting from equivalence, i thought he was speaking of issues resulting from substitution. I did not mean to imply that you should avoid this result, simply that if you want your calculator to work by the decision theory I thought he was referring to, it got to have some utilities associated with outputs. And this doesn’t really help make a calculating device.
Who said anything about sentience? NDT, as described, is a perfectly comprehensible program that (in certain games that you or I would regard as fair tests) generates spurious counterfactuals and thus makes terrible decisions, thanks to a particular kind of circularity.
In this sequence, I’m not talking about FAI or anything beyond my current understanding, and I’m not intentionally drawing metaphors. I’m simply outlining programs which (if I could write a good automated theorem-prover) I could write myself, and comparing how they do in a straightforward tournament setting, with the twist of allowing read-access to source codes. We should be able to agree on that base level.
Well, the substitutions are specifically to turn a circularity into a case of having x on both sides of some equation. We might be talking about different things. The failure mode is benign; you arrive at x=x .
edit: ahh, another thing. If you have source of randomness, you need to consider the solution with, and without, the substitution, as you can make substitution invalid by employing the random number generator. The substitution of the nonrandom part of strategy can still be useful though. Maybe that’s what you had in mind?
Err, I’m not sure what you mean here. In the CDT algorithm, if it deduces that Y employs a particular mixed strategy, then it can calculate the expected value of each action against that mixed strategy.
(For complete simplicity, though, starting next post I’m going to assume that there’s at least one pure Nash equilibrium option in G. If it doesn’t start with one, we can treat a mixed equilibrium as x{n+1} and y{m+1}, and fill in the new row and column of the matrix with the right expected values.)
I mean this sort of circularity:
I agree that some forms are benign. The Naive Decision Theory post and cousin_it’s followup illustrate a malignant form.
That’s why you don’t let your calculator be sentient. FAI might give a number that makes you most happy, which might well be 42 if you are not relying on this number for anything useful. (E.g. it might tell 42 as a joke, knowing that you know what 2+3 is)
Edit: you can, however, have some requirements on the calculator’s output, and then there will be the number that satisfies those criteria; the x substitution will work to solve for this value, and in principle even to solve for protective measures to take against cosmic rays, and so on.
edit: and on the NDT, it doesn’t one-way substitute at start. It assumes equivalence.
Sure, if it happened to be in a situation where the most valuable thing to do by my standards was make me happy. Agreed.
You seem to be implying that I should prefer to avoid this result… do you in fact believe that?
If so, can you clarify why?
A somewhat analogous real-world situation: one of Siri’s possible responses to “open the pod bay doors” as a command is “We’re never going to live that down, are we?” This delights me enormously, and costs nothing of consequence. Should I prefer that this result be eliminated?
Actually I misunderstood his point with calculator. He was speaking of NDT with issues resulting from equivalence, i thought he was speaking of issues resulting from substitution. I did not mean to imply that you should avoid this result, simply that if you want your calculator to work by the decision theory I thought he was referring to, it got to have some utilities associated with outputs. And this doesn’t really help make a calculating device.
Gotcha. Thanks for clarifying.
Who said anything about sentience? NDT, as described, is a perfectly comprehensible program that (in certain games that you or I would regard as fair tests) generates spurious counterfactuals and thus makes terrible decisions, thanks to a particular kind of circularity.
In this sequence, I’m not talking about FAI or anything beyond my current understanding, and I’m not intentionally drawing metaphors. I’m simply outlining programs which (if I could write a good automated theorem-prover) I could write myself, and comparing how they do in a straightforward tournament setting, with the twist of allowing read-access to source codes. We should be able to agree on that base level.
Yea, NDT is no good, agreed about that. That doesn’t so much results from substitution as from full blown two way equivalence.