I guess so, although looking at it now Elcenia seems to be pretty massive. It will take me a couple of weeks to catch up at least (unless it’s exceptionally compelling, it which case damn you in advance for taking up all my time), and we also have to allow for the possibility that it’s not just my kind of thing, in which case trying to finish it will make me miserable and I won’t be much use to you anyway. But sure, I’ll give it a shot.
APMason
Karma: 578
Sent.
Okay, I wrote up my thoughts, but it’s pretty long and I’m not sure it’s fair to post it here (also it’s too long for a PM). Do you have an email I can send it to?
What happens if you’re using this method and you’re offered a gamble where you have a 49% chance of gaining 1000000utils and a 51% chance of losing 5utils (if you don’t take the deal you gain and lose nothing). Isn’t the “typical outcome” here a loss, even though we might really really want to take the gamble? Or have I misunderstood what you propose?
I might be interested in giving a fuller critique of this at some point (but then who the hell am I), but for now I’ll confine myself to just one point:
It was, of course, a highly ceremonial occasion...
The reader knows that the narrator knows more about this world than they do. The reader is okay with that. Trying to impart information by pretending that the reader already knows it seems clumsy and distracting to me. Compare with:
It was a highly ceremonial occasion, excruciatingly ritualized, and he was bored.
I think this is fine. No need to pretend you’re the reader’s chum.
I think the clearest and simplest version of Problem 1 is where Omega chooses to simulate a CDT agent with .5 probability and a TDT agent with .5 probability. Let’s say that Value-B is $1000000, as is traditional, and Value-A is $1000. TDT will one-box for an expected value of $500500 (as opposed to $1000 if it two-boxes), and CDT will always two-box, and receive an expected $501000. Both TDT and CDT have an equal chance of playing against each other in this version, and an equal chance of playing against themselves, and yet CDT still outperforms. It seems TDT suffers for CDT’s irrationality, and CDT benefits from TDT’s rationality. Very troubling.
EDIT: (I will note, though, that a TDT agent still can’t do any better by two-boxing—only make CDT do worse).
Hmm, if I’ve understood this correctly, it’s the way I’ve always thought about decision theory for as long as I’ve had a concept of expected utility maximisation. Which makes me think I must have missed some important aspect of the ex post version.
I’m not sure whether it is the case that primitive cultures have a category of things they think of as “supernatural”—pagan religions were certainly quite literal: they lived on Olympus, they mated with humans, they were birthed. I wonder whether the distinction between “natural” and “supernatural” only comes about when it becomes clear that gods don’t belong in the former category.
And that’s without even getting into my experiences, or those close to me.
Well, don’t be coy. There’s no point in withholding your strongest piece of evidence. Please, get into it.
Why is it important for a decision theory to pass fair tests but not unfair tests?
Well, on unfair tests a decision theory still needs to do as well as possible. If we had a version of the original Newcomb’s problem, with the one difference that a CDT agent gets $1billion just for showing up, it’s still incumbent upon a TDT agent to walk away with $1000000 rather than $1000. The “unfair” class of problems is that class where “winning as much as possible” is distinct from “winning the most out of all possible agents”.
Why are we not counting philosophers? Isn’t that like saying, “Not counting physicists, where’s this supposed interest in gravity?”
Well, I’ve had a think about it, and I’ve concluded that it would matter how great the difference between TDT and TDT-prime is. If TDT-prime is almost the same as TDT, but has an extra stage in its algorithm in which it converts all dollar amounts to yen, it should still be able to prove that it is isomorphic to Omega’s simulation, and therefore will not be able to take advantage of “logical separation”.
But if TDT-prime is different in a way that makes it non-isomorphic, i.e. it sometimes gives a different output given the same inputs, that may still not be enough to “separate” them. If TDT-prime acts the same as TDT, except when there is a walrus in the vicinity, in which case it tries to train the walrus to fight crime, it is still the case in this walrus-free problem that it makes exactly the same choice as the simulation (?). It’s as if you need the ability to prove that two agents necessarily give the same output for the particular problem you’re faced with, without proving what output those agents actually give, and that sure looks crazy-hard.
EDIT: I mean crazy-hard for the general case, but much, much easier for all the cases where the two agents are actually the same.
EDIT 2: On the subject of fairness, my first thoughts: A fair problem is one in which if you had arrived at your decision by a coin flip (which is as transparently predictable as your actual decision process—i.e. Omega can predict whether it’s going to come down heads or tails with perfect accuracy), you would be rewarded or punished no more or less than you would be using your actual decision algorithm (and this applies to every available option).
EDIT 3: Sorry to go on like this, but I’ve just realised that won’t work in situations where some other agent bases their decision on whether you’re predicting what their decision will be, i.e. Prisoner’s Dilemma.
Hmm, so TDT-prime would reason something like, “The TDT simulation will one-box because, not knowing that it’s the simulation, but also knowing that the simulation will use exactly the same decision theory as itself, it will conclude that the simulation will do the same thing as itself and so one-boxing is the best option. However, I’m different to the TDT-simulation, and therefore I can safely two-box without affecting its decision.” In which case, does it matter how inconsequential the difference is? Yep, I’m confused.
You can see that something funny has hapened by postulating TDT-prime, which is identical to TDT except that Omega doesn’t recognize it as a duplicate (eg, it differs in some way that should be irrelevant). TDT-prime would two-box, and win.
I don’t think so. If TDT-prime two boxes, the TDT simulation two-boxes, so only one box is full, so TDT-prime walks away with $1000. Omega doesn’t check what decision theory you’re using at all—it just simulates TDT and bases its decision on that. I do think that this ought to fall outside a rigorously defined class of “fair” problems, but it doesn’t matter whether Omega can recognise you as a TDT-agent or not.
this problem is the reason why decision theories have to be non-deterministic. It comes up all the time in real life: I try and guess what safe combination you chose, try that combination, and if it works I take all your money.
Of course, you can just set up the thought experiment with the proviso that “be unpredictable” is not a possible move—in fact that’s the whole point of Omega in these sorts of problems. If Omega’s trying to break into your safe, he takes your money. In Nesov’s problem, if you can’t make yourself unpredictable, then you win nothing—it’s not even worth your time to open the box. In both cases, a TDT agent does strictly as well as it possibly could—the fact that there’s $100 somewhere in the vicinity doesn’t change that.
Okay, so there’s no such thing as jackalopes. Now I know.
At a certain point the psychological quality of life of living individuals that comes from living in a society with a certain structure and values may trump the right of individuals who thought they were dead to live once more.
This is vague. Can you pinpoint exactly why you think this would damage people’s psychological quality of life?
If information cannot travel back more than six hours
This does seem to be a constraint that exclusively affects the time-turners. Otherwise prophesies wouldn’t be possible. It also seems like it’s an artificial rule rather than a deep law of magic because after the Stanford Prison experiment, Bones tells Dumbledore that she has information from four hours in the future and asks whether he’d like to know it. That there is relevant information from four hours in the future is information from the future—she would not have said that if it were otherwise, so it seems there must be exemptions of that kind.
Alternative hypothesis: prophesies are jive, and Eliezer didn’t think of the other thing.
Edit: In other words, I think Torture v. Specks is just a restatement of the Repugnant Conclusion.
The Repugnant Conclusion can be rejected by average-utilitarianism, whereas in Torture vs. Dustspecks average-utilitarianism still tells you to torture, because the disutility of 50 years of torture divided among 3^^^3 people is less than the disutility of 3^^^3 dustspecks divided among 3^^^3 people. That’s an important structural difference to the thought experiment.
Actually you only cooperate if the other player would defect if you didn’t cooperate. If they cooperate no matter what, defect.