I guess I didn’t give you enough credit. I had redrawn AnnaSalamon’s graph to account for your concerns, and I still think it does (even more so than the one in your top-level post), but in doing so, I made it no longer (consistent with) TDT. There is a relevant difference between the algorithm you’re actually running, and the algorithm you think you’re running, and this uncertainty about the difference affects how you choose your algorithm.
What’s more perplexing is, as I’ve pointed out, Eliezer_Yudkowsky seems to recognize this problem in other contexts, yet you also seem to be correct that TDT assumes it away. Interesting.
(Btw, could you put a link to my graph in your top-level post to make it easier to find?)
Sure. Do you want to give a specific algorithm summary for me to add with your graph to explain your version of how to solve this problem, or do you just want me to add a link to the graph? (added just the link for now, lemme know if there’s either a specific comment you want me to also point out to explain your version or anything)
(if nothing else, incidentally, I’d suggest even in your version, the platonic space of algorithms (set of all algorithms) ought have a direct link to “actual think that ends up being implemented”)
Do you want to give a specific algorithm summary for me to add …
No need: The graph isn’t intended to represent a solution algorithm, and I’m not yet proposing one. I just think it better represents what’s going on in the problem than Eliezer_Yudkowsky’s representation. Which, like you suggest, throws into doubt how well his solution works.
Still, the EY post I linked is an excellent example of how to handle the uncertainty that results from the effect of your innards on what you try to implement, in a way consistent with my graph.
I’d suggest even in your version, the platonic space of algorithms (set of all algorithms) ought have a direct link to “actual think that ends up being implemented”
I’m having a hard time thinking about that issue. What does it mean for the PSoA (Platonic Space of Algorithms) to have a causal effect on something? Arguably, it points to all the nodes on the graph, because every physical process is constrained by this space, but what would that mean in terms of finding a solution? What might I miss if I fail to include a link from PSoA to something where it really does connect?
As far as linking “space of algorithms” to “Actual thing effectively implemented”, the idea is more to maintain one of the key ideas of TDT… ie, that what you’re ‘controlling’ (for lack of a better word) is effectively the output of all instances of the algorithm you actually implement, right?
Yes, well, the goal should be to find an accurate representation of the situation. If the causal model implied by TDT doesn’t fit, well, all the worse for TDT!
And incidentally, this idea:
that what you’re ‘controlling’ (for lack of a better word) is effectively the output of all instances of the algorithm you actually implement
is very troubling to me in that it is effectively saying that my internal thoughts have causal power (or something indistinguishable therefrom) over other people: that by coming to one conclusion means other people must be coming to the same conclusion with enough frequency to matter for my predictions.
Yes, the fact that I have general thoughts, feeling, emotions, etc. is evidence that other people have them … but it’s very weak evidence, and more so for my decisions. Whatever the truth behind the TDT assumption, it’s not likely to be applicable often.
But… two instances of the exact same (deterministic) algorithm fed the exact same parameters will return the exact same output. So when choosing what the output ought to be, one ought to act as if one is determining the output of all the instances.
Perfectly true, and perfectly useless. You’ll never find the exact same deterministic algorithm as the one you’re implementing, especially when you consider the post-decision interference of innards. Any identical sub-algorithm will be lost in a sea of different sub-algorithms.
Sure, but then we can extend to talking about classes of related algorithms that produce related outputs, so that there would still statistical dependence even if they’re not absolutely identical in all cases. (And summing over ‘which algorithm is being run’ would be part of dealing with that.)
But statistical dependence does not imply large dependence nor does it imply useful dependence. The variety of psychology among people means that my decision is weak evidence of others’ decision, even if it is evidence. It doesn’t do much to alter my prior of how other observations have molded my expectations of people. (And this point applies doubly so in the case of autistic spectrum people like me who are e.g. surprised at other’s unwillingness to point out my easily correctible difficulties.)
Now, if we were talking about a situation confined to that one sub-algorithm, your point would still have validity. But the problem involves the interplay of other algorithms with even more uncertainty.
Plus, the inherent implausibility of the position (implied by TDT) that my decision to be more charitable must mean that other people just decided to become more charitable.
Well, one would actually take into account the degree of dependence when doing the relevant computation.
And your decision to be more charitable would correlate to others being so to the extent that they’re using related methods to come to their own decision.
Well, one would actually take into account the degree of dependence when doing the relevant computation.
Yes, and here’s what it would look like: I anticipate a 1⁄2 + e probability of the other person doing the same thing as me in the true PD. I’ll use the payoff matrix of
C D
C (3,3) (0,5)
D (5,0) (1,1)
where the first value is my utility. The expected payoff is then (after a little algebra):
If I cooperate: 3⁄2 + 3e; if I defect: 3 − 4e
Defection has a higher payoff as long as e is less than 3⁄14 (total probability of other person doing what I do = 10⁄14). So you should cooperate as long as you have over 0.137 bits of evidence that they will do what you do. Does the assumption that other people’s algorithm has a minor resemblance to mine get me that?
And your decision to be more charitable would correlate to others being so to the extent that they’re using related methods to come to their own decision.
Yes, and that’s the tough bullet to bite: me being more charitable, irrespective the impact of my charitable action, causes (me to observe) other people being more charitable.
But if you are literally talking about the same computation, that computation must be unable to know which instance it is operating from. Once the question of getting identified with “other” instances is raised, the computations are different, and can lead to different outcomes, if these outcomes nontrivially depend on the different contexts. How is this progress compared to the case of two identical copies in PD that know their actions to be necessarily identical, and thus choosing between (C,C) and (D,D)?
The interesting part is making the sense of dependence between different decision processes precise.
I guess I didn’t give you enough credit. I had redrawn AnnaSalamon’s graph to account for your concerns, and I still think it does (even more so than the one in your top-level post), but in doing so, I made it no longer (consistent with) TDT. There is a relevant difference between the algorithm you’re actually running, and the algorithm you think you’re running, and this uncertainty about the difference affects how you choose your algorithm.
What’s more perplexing is, as I’ve pointed out, Eliezer_Yudkowsky seems to recognize this problem in other contexts, yet you also seem to be correct that TDT assumes it away. Interesting.
(Btw, could you put a link to my graph in your top-level post to make it easier to find?)
Sure. Do you want to give a specific algorithm summary for me to add with your graph to explain your version of how to solve this problem, or do you just want me to add a link to the graph? (added just the link for now, lemme know if there’s either a specific comment you want me to also point out to explain your version or anything)
(if nothing else, incidentally, I’d suggest even in your version, the platonic space of algorithms (set of all algorithms) ought have a direct link to “actual think that ends up being implemented”)
Thanks for adding the link.
No need: The graph isn’t intended to represent a solution algorithm, and I’m not yet proposing one. I just think it better represents what’s going on in the problem than Eliezer_Yudkowsky’s representation. Which, like you suggest, throws into doubt how well his solution works.
Still, the EY post I linked is an excellent example of how to handle the uncertainty that results from the effect of your innards on what you try to implement, in a way consistent with my graph.
I’m having a hard time thinking about that issue. What does it mean for the PSoA (Platonic Space of Algorithms) to have a causal effect on something? Arguably, it points to all the nodes on the graph, because every physical process is constrained by this space, but what would that mean in terms of finding a solution? What might I miss if I fail to include a link from PSoA to something where it really does connect?
No problem.
As far as linking “space of algorithms” to “Actual thing effectively implemented”, the idea is more to maintain one of the key ideas of TDT… ie, that what you’re ‘controlling’ (for lack of a better word) is effectively the output of all instances of the algorithm you actually implement, right?
Yes, well, the goal should be to find an accurate representation of the situation. If the causal model implied by TDT doesn’t fit, well, all the worse for TDT!
And incidentally, this idea:
is very troubling to me in that it is effectively saying that my internal thoughts have causal power (or something indistinguishable therefrom) over other people: that by coming to one conclusion means other people must be coming to the same conclusion with enough frequency to matter for my predictions.
Yes, the fact that I have general thoughts, feeling, emotions, etc. is evidence that other people have them … but it’s very weak evidence, and more so for my decisions. Whatever the truth behind the TDT assumption, it’s not likely to be applicable often.
“controlling” is the wrong word.
But… two instances of the exact same (deterministic) algorithm fed the exact same parameters will return the exact same output. So when choosing what the output ought to be, one ought to act as if one is determining the output of all the instances.
Perfectly true, and perfectly useless. You’ll never find the exact same deterministic algorithm as the one you’re implementing, especially when you consider the post-decision interference of innards. Any identical sub-algorithm will be lost in a sea of different sub-algorithms.
Sure, but then we can extend to talking about classes of related algorithms that produce related outputs, so that there would still statistical dependence even if they’re not absolutely identical in all cases. (And summing over ‘which algorithm is being run’ would be part of dealing with that.)
But statistical dependence does not imply large dependence nor does it imply useful dependence. The variety of psychology among people means that my decision is weak evidence of others’ decision, even if it is evidence. It doesn’t do much to alter my prior of how other observations have molded my expectations of people. (And this point applies doubly so in the case of autistic spectrum people like me who are e.g. surprised at other’s unwillingness to point out my easily correctible difficulties.)
Now, if we were talking about a situation confined to that one sub-algorithm, your point would still have validity. But the problem involves the interplay of other algorithms with even more uncertainty.
Plus, the inherent implausibility of the position (implied by TDT) that my decision to be more charitable must mean that other people just decided to become more charitable.
Well, one would actually take into account the degree of dependence when doing the relevant computation.
And your decision to be more charitable would correlate to others being so to the extent that they’re using related methods to come to their own decision.
Yes, and here’s what it would look like: I anticipate a 1⁄2 + e probability of the other person doing the same thing as me in the true PD. I’ll use the payoff matrix of
C D
C (3,3) (0,5)
D (5,0) (1,1)
where the first value is my utility. The expected payoff is then (after a little algebra):
If I cooperate: 3⁄2 + 3e; if I defect: 3 − 4e
Defection has a higher payoff as long as e is less than 3⁄14 (total probability of other person doing what I do = 10⁄14). So you should cooperate as long as you have over 0.137 bits of evidence that they will do what you do. Does the assumption that other people’s algorithm has a minor resemblance to mine get me that?
Yes, and that’s the tough bullet to bite: me being more charitable, irrespective the impact of my charitable action, causes (me to observe) other people being more charitable.
But if you are literally talking about the same computation, that computation must be unable to know which instance it is operating from. Once the question of getting identified with “other” instances is raised, the computations are different, and can lead to different outcomes, if these outcomes nontrivially depend on the different contexts. How is this progress compared to the case of two identical copies in PD that know their actions to be necessarily identical, and thus choosing between (C,C) and (D,D)?
The interesting part is making the sense of dependence between different decision processes precise.