I don’t get it. Is this supposed to be some weird form of evidential or maybe timeless decision theory? It hardly matters; whatever decision theory you’re using, you already know you exist; conditioning on the possibility that you don’t is nonsensical. Hell, even if you’re an AI using UDT you gain nothing from not assuming you exist; you were built to not update in the normal sense because whoever built you cared about all possible worlds you might end up in, but regardless, if you’re standing there making the decision, you exist (i.e. this can be assumed at the start and taken into account).
Edit: Just for the purpose of explicitness, I should probably state that the conclusion here is that you should two-box in this case.
In this scenario two-boxers get 200$ and exist, while one-boxers get 100$ and exist.
Two-boxers will be numerically fewer, because Prometheus is biased in favour of irrationality, but nonetheless it’ll be two-boxers that’ll be winning. That’s the opposite of two-boxers in the Newcomb problem.
Nice icon, though my reasoning is the exact opposite than that of Quantum Suicide. I have no shared identity with the people who would one-box here, so I don’t need to one-box in order to increase their chances at having existed—if anything such an action would increase the stupidity levels in the multiverse.
Even a one-boxer would have to be particularly weird to want to increase the amplitude of his universe’s configuration, as if that would affect his own life at all.
Quantum Suicide on the other hand assumes a shared identity between the people who’ll die and the people who’ll suffer permanent brain damage with a bullet lodged on their brain, and the people who’ll have their consciousness magically copied by magical aliens before they kill themselves. I don’t assume shared identity, and that’s why I two-box here, quantum suiciders on the other hand assume it and that’s why they fail.
I’m not wondering whether interacting with 4chan would poison the mind of a specific software construct. I’m wondering whether the long term political consequences would be good or bad if the 4chan community picked up the generic technique of adding photo-shopped text to MS Clippy images as a joke generating engine that involved re-purposing of LW’s themes content (probably sometimes in troll-like or deprecating ways).
Would it raise interesting emotional critiques of moral arguments? Would it poison the discourse with jokes and confusion? Would it bring new people here with worthwhile insights? Would it reduce/increase the seriousness with which the wider world took AGI research… and which of those outcomes is even preferred?
I still don’t really have a good theory of what kinds of mass opinion on the subject of FAI is possible or desirable and when I see something novel like the clippy image it sometimes makes me try to re-calculate the public relations angle of singularity stuff.
It seems to me that if you find yourself having a choice, you should two-box. If the premise is true then you probably won’t feel like you have a choice, and your choice will be to one-box.
I guess you were selected by Prometheus :).
edit: this is related to the idea about going back in time and killing your grandfather. Either this is possible, or it’s not. Either way you can’t erase yourself and end up with the universe in an inconsistent state.
edit2: In other words, either the premise is impossible, or most people will one-box regardless of any recommendations or stratagems devised here or elsewhere.
edit3: I think this is different from the traditional Newcomb’s problem in that by the time you know there’s a problem, it’s certainly too late to change anything. With Newcomb’s you can pre-commit to one-boxing if you’ve heard about the problem beforehand.
If time travel to your own past (rather than creating an extra time line) is possible hypothetical people with access to time travel who are determined to kill their grandfathers (before their parent’s conception) have (in the sense of actions in inconsistent hypothetical time lines that influence which possible stable time line comes about) all eventually created a stable time loop where they don’t exist as people who are determined to kill their grandfathers.
(e. g. they succeed and influence the time line in such a way that their other parent has a different child with someone else instead who goes back in time and accidentally kills the would be grandfather of the first person. Or they die in a freak accident that influences which children their would be grandfather has, which means a different grandchild that time travels with different actions and influences what grandchildren the grandfather ends up with until a grand child comes into existence who coincidentally influences the time line in just exactly the right way to bring their own existence about. Or something more complicated.)
Since I prefer to exist I will not time travel in any way that seems likely to make my existence inconsistent and take actions to make it consistent when it seems to be inconsistent without such actions. For example if I learned that my grandmother’s fiancé was murdered by someone who claimed to be his grandchild and I had access to time travel I would try to stage that murder and take the fiancé back to the future with me.
My point is that you can’t step outside the system and say that you’re making a choice. Killing your own (true) grandfather in the past is simply impossible, so you won’t be able to do it, for one reason or another. The details don’t matter.
edit: I guess my position on Newcomb’s is that you should precommit to one-boxing if you can, but if someone is put into that situation with no pre-knowledge, it is too late to bother talking about what they “should” do—their fate is already sealed.
I think this is different from the traditional Newcomb’s problem in that by the time you know there’s a problem, it’s certainly too late to change anything. With Newcomb’s you can pre-commit to one-boxing if you’ve heard about the problem beforehand.
Newcomb’s with precommitments? Next can we do Tic-tac-toe? ;)
If you haven’t heard about the problem beforehand then asking what decision you “should” make is incoherent. You will get the result you were selected to get. There is no use talking as if you have some meta-choice.
edit: ie if you are selected on your decision process without having heard of such problems, then it is already too late to change your past decision process even if you fully understand the situation you are in. If you’re capable of understanding the situation though, you presumably already had the right decision process on some level and will successfully one-box.
edit2: The probabilistic method of dealing with Newcomb’s problem is to observe that one-boxers win, therefore you should one-box. This doesn’t apply to the Prometheus problem; we can’t observe that two-boxers probably never existed.
The probabilistic method of dealing with Newcomb’s problem is to observe that one-boxers win, therefore you should one-box. This doesn’t apply to the Prometheus problem; we can’t observe that two-boxers probably never existed.
Including observations of other people who have encountered Omega’s game in the description of Newcomb’s problem is sometimes helpful because it engages the intuitions of those who aren’t familiar with the relevant kinds of reasoning. It is not, however, an important part of the problem or the critical part of the solution.
I don’t find it helpful that you just keep asserting that you’re right without explaining your reasoning. Please explain why you think one-boxing is correct in the Prometheus case.
That you do not understand the explanations does not mean I have not given any. I refer you to the original post. From that link a search for ‘wedrifid’ will give you at least three explanations.
In this case of the grandparent you may (or may not) note that my reply speaks to the relevance of that comment’s parent to the same comment’s grandparent.
I also observe that when replying to a rebuttal (pre-edit) that consists of asserting an incorrect premise used to support reasoning that isn’t quite relevant there is only so much you can do. The second edit contained what we could call a ‘high quality mistake’ so I attempted to explain to you why that line of reasoning does not influence the decision making here.
I suspect you will find it more enjoyable to engage with one of the other people who have also explained the reasoning behind one-boxing here (complete with pictures!) If you keep making replies to me that don’t seem (to me) make any sense in the context it is natural that you will be unsatisfied with the response.
Sorry if my original posts were unclear—I was drunk at the time :). However I’ve read the rest of this thread and I agree with the positions of the Prometheus two-boxers for the problem as given. If Prometheus’s strategy involves simulating you to adulthood and giving you a sim-test before the “real” test, then things may be different.
Okay: originally I was leaning toward two-boxing, but now I’m not sure. Conceivably, for example, I am doomed to have a sudden cardiac arrest and die before actually getting to make my selection; this would kind of trivially satisfy Prometheus’ criteria (depending, I suppose, on precisely how they’re formulated). My death, in that case, would not be a consequence of my choosing both boxes, as I never actually get to make that decision.
Better not to tangle with the gods, I think. I’d take one box.
“I think this is different from the traditional Newcomb’s problem in that by the time you know there’s a problem, it’s certainly too late to change anything. With Newcomb’s you can pre-commit to one-boxing if you’ve heard about the problem beforehand.”
Agreed. It would be like opening the first box, finding the million dollars, and then having someone explain Newcomb’s problem to you as you consider whether or not to open the second. My thought would be, “Ha! Omega was WRONG!!!! ” laughing as I dove into the second box.
edit: Because there was no contract made between TDT agents before the first box was opened, there seems to be no reason to honor that contract, which was drawn afterwards.
I don’t get it. Is this supposed to be some weird form of evidential or maybe timeless decision theory? It hardly matters; whatever decision theory you’re using, you already know you exist; conditioning on the possibility that you don’t is nonsensical. Hell, even if you’re an AI using UDT you gain nothing from not assuming you exist; you were built to not update in the normal sense because whoever built you cared about all possible worlds you might end up in, but regardless, if you’re standing there making the decision, you exist (i.e. this can be assumed at the start and taken into account).
Edit: Just for the purpose of explicitness, I should probably state that the conclusion here is that you should two-box in this case.
And so as to demonstrate that the first part of the post is controversial enough to be interesting: Sniffnoy is wrong—you are better off one boxing.
Rationalists should win.
In this scenario two-boxers get 200$ and exist, while one-boxers get 100$ and exist.
Two-boxers will be numerically fewer, because Prometheus is biased in favour of irrationality, but nonetheless it’ll be two-boxers that’ll be winning. That’s the opposite of two-boxers in the Newcomb problem.
Nice icon, though my reasoning is the exact opposite than that of Quantum Suicide. I have no shared identity with the people who would one-box here, so I don’t need to one-box in order to increase their chances at having existed—if anything such an action would increase the stupidity levels in the multiverse.
Even a one-boxer would have to be particularly weird to want to increase the amplitude of his universe’s configuration, as if that would affect his own life at all.
Quantum Suicide on the other hand assumes a shared identity between the people who’ll die and the people who’ll suffer permanent brain damage with a bullet lodged on their brain, and the people who’ll have their consciousness magically copied by magical aliens before they kill themselves. I don’t assume shared identity, and that’s why I two-box here, quantum suiciders on the other hand assume it and that’s why they fail.
! ! ! !
Tangential Question: Would it be good or bad for the world if 4chan picked this up as a meme?
The Friendly AI must be kept away from 4chan at all costs.
FAI’s don’t run away from hard problems.
I should have been more specific.
I’m not wondering whether interacting with 4chan would poison the mind of a specific software construct. I’m wondering whether the long term political consequences would be good or bad if the 4chan community picked up the generic technique of adding photo-shopped text to MS Clippy images as a joke generating engine that involved re-purposing of LW’s themes content (probably sometimes in troll-like or deprecating ways).
Would it raise interesting emotional critiques of moral arguments? Would it poison the discourse with jokes and confusion? Would it bring new people here with worthwhile insights? Would it reduce/increase the seriousness with which the wider world took AGI research… and which of those outcomes is even preferred?
I still don’t really have a good theory of what kinds of mass opinion on the subject of FAI is possible or desirable and when I see something novel like the clippy image it sometimes makes me try to re-calculate the public relations angle of singularity stuff.
Which meme, MS Clippy jokes or quantum suicide?
I’m fine with 4channers picking up quantum suicide, especially since to me it will almost always look like regular suicide.
That is brilliant. Did you create it manually?
Thanks, I did. I’m sure there are generators for it, though.
It seems to me that if you find yourself having a choice, you should two-box. If the premise is true then you probably won’t feel like you have a choice, and your choice will be to one-box.
I guess you were selected by Prometheus :).
edit: this is related to the idea about going back in time and killing your grandfather. Either this is possible, or it’s not. Either way you can’t erase yourself and end up with the universe in an inconsistent state.
edit2: In other words, either the premise is impossible, or most people will one-box regardless of any recommendations or stratagems devised here or elsewhere.
edit3: I think this is different from the traditional Newcomb’s problem in that by the time you know there’s a problem, it’s certainly too late to change anything. With Newcomb’s you can pre-commit to one-boxing if you’ve heard about the problem beforehand.
If time travel to your own past (rather than creating an extra time line) is possible hypothetical people with access to time travel who are determined to kill their grandfathers (before their parent’s conception) have (in the sense of actions in inconsistent hypothetical time lines that influence which possible stable time line comes about) all eventually created a stable time loop where they don’t exist as people who are determined to kill their grandfathers.
(e. g. they succeed and influence the time line in such a way that their other parent has a different child with someone else instead who goes back in time and accidentally kills the would be grandfather of the first person. Or they die in a freak accident that influences which children their would be grandfather has, which means a different grandchild that time travels with different actions and influences what grandchildren the grandfather ends up with until a grand child comes into existence who coincidentally influences the time line in just exactly the right way to bring their own existence about. Or something more complicated.)
Since I prefer to exist I will not time travel in any way that seems likely to make my existence inconsistent and take actions to make it consistent when it seems to be inconsistent without such actions. For example if I learned that my grandmother’s fiancé was murdered by someone who claimed to be his grandchild and I had access to time travel I would try to stage that murder and take the fiancé back to the future with me.
My point is that you can’t step outside the system and say that you’re making a choice. Killing your own (true) grandfather in the past is simply impossible, so you won’t be able to do it, for one reason or another. The details don’t matter.
edit: I guess my position on Newcomb’s is that you should precommit to one-boxing if you can, but if someone is put into that situation with no pre-knowledge, it is too late to bother talking about what they “should” do—their fate is already sealed.
Newcomb’s with precommitments? Next can we do Tic-tac-toe? ;)
If you haven’t heard about the problem beforehand then asking what decision you “should” make is incoherent. You will get the result you were selected to get. There is no use talking as if you have some meta-choice.
edit: ie if you are selected on your decision process without having heard of such problems, then it is already too late to change your past decision process even if you fully understand the situation you are in. If you’re capable of understanding the situation though, you presumably already had the right decision process on some level and will successfully one-box.
edit2: The probabilistic method of dealing with Newcomb’s problem is to observe that one-boxers win, therefore you should one-box. This doesn’t apply to the Prometheus problem; we can’t observe that two-boxers probably never existed.
Including observations of other people who have encountered Omega’s game in the description of Newcomb’s problem is sometimes helpful because it engages the intuitions of those who aren’t familiar with the relevant kinds of reasoning. It is not, however, an important part of the problem or the critical part of the solution.
I didn’t claim it was—I was just pointing out another way that these two problems are different.
You claimed that The claimed “what decision you “should” make is incoherent”. (This claim is false.)
I don’t find it helpful that you just keep asserting that you’re right without explaining your reasoning. Please explain why you think one-boxing is correct in the Prometheus case.
That you do not understand the explanations does not mean I have not given any. I refer you to the original post. From that link a search for ‘wedrifid’ will give you at least three explanations.
In this case of the grandparent you may (or may not) note that my reply speaks to the relevance of that comment’s parent to the same comment’s grandparent.
I also observe that when replying to a rebuttal (pre-edit) that consists of asserting an incorrect premise used to support reasoning that isn’t quite relevant there is only so much you can do. The second edit contained what we could call a ‘high quality mistake’ so I attempted to explain to you why that line of reasoning does not influence the decision making here.
I suspect you will find it more enjoyable to engage with one of the other people who have also explained the reasoning behind one-boxing here (complete with pictures!) If you keep making replies to me that don’t seem (to me) make any sense in the context it is natural that you will be unsatisfied with the response.
Sorry if my original posts were unclear—I was drunk at the time :). However I’ve read the rest of this thread and I agree with the positions of the Prometheus two-boxers for the problem as given. If Prometheus’s strategy involves simulating you to adulthood and giving you a sim-test before the “real” test, then things may be different.
Taking drunken boxing to a whole new level! ;)
I don’t agree with any of this.
Good for you? I guess we’ll have to call up Omega and Prometheus and test it all..
Okay: originally I was leaning toward two-boxing, but now I’m not sure. Conceivably, for example, I am doomed to have a sudden cardiac arrest and die before actually getting to make my selection; this would kind of trivially satisfy Prometheus’ criteria (depending, I suppose, on precisely how they’re formulated). My death, in that case, would not be a consequence of my choosing both boxes, as I never actually get to make that decision.
Better not to tangle with the gods, I think. I’d take one box.
“I think this is different from the traditional Newcomb’s problem in that by the time you know there’s a problem, it’s certainly too late to change anything. With Newcomb’s you can pre-commit to one-boxing if you’ve heard about the problem beforehand.”
Agreed. It would be like opening the first box, finding the million dollars, and then having someone explain Newcomb’s problem to you as you consider whether or not to open the second. My thought would be, “Ha! Omega was WRONG!!!! ” laughing as I dove into the second box.
edit: Because there was no contract made between TDT agents before the first box was opened, there seems to be no reason to honor that contract, which was drawn afterwards.