Eliezer finally published TDT a few days ago, I think it’s up at the singinst.org site by now.
Excellent, that’ll be a fun read.
Cryonics isn’t an agent we have to deal with. Pascal’s Mugger we can deal with because both options lead to negative expected utility, and so we find ways to avoid the choice entirely by appealing to the motivations of the agent to not waste resources. But in the case of cryonics no one has a gun to our head, and there’s no one to argue with: either cryonics works, or it doesn’t. We just have to figure it out. The invalidity of paying Pascal’s mugger doesn’t have anything to do with the infinity in the calculation; that gets sidestepped entirely by refusing to engage in negative sum actions of any kind, negative sum or not, large or small.
I’m still not sure if I follow this—I’ll have to do some more reading on it. I still don’t see how the two situations are different—for example, if I was talking to someone selling cryonics, wouldn’t that be qualitatively the same as Pascal’s Mugging? I’m not sure.
Might it have been here? That’s where I was first introduced to LW and Eliezer.
Unfortunately no, it was here. I didn’t look at that article until recently.
Any ideas/heuristics you’re suspicious of specifically?
That opens a whole new can of worms that it’s far too late at night for me to address, but I’m thinking of writing a post on this soon, perhaps tomorrow.
I still don’t see how the two situations are different—for example, if I was talking to someone selling cryonics, wouldn’t that be qualitatively the same as Pascal’s Mugging?
Nah, the cryonics agent isn’t trying to mug you! (Er, hopefully.) He’s just giving you two options and letting you calculate.
In this case of Pascal’s Mugging both choices lead to negative expected utility as defined by the problem. Hence you look for a third option, and in this case, you find one: ignore all blackmailers; tell them to go ahead and torture all those people, you don’t care. Unless they find joy in torturing people (then you’re screwed) they have no incentive to actually use up the resources to go through with it. So they leave you alone, ’cuz you won’t budge.
Cryonics is a lot simpler in its nature, but a lot harder to calculate. You have two options, and the options are given to you by reality, not an agent you can outwit. (Throwing in a cryonics agent doesn’t change anything.) When you have to choose between the binary cryonics versus no cryonics, it’s just a matter of seeing which decision is better (or worse). It could be that both are bad, like in the Pascal’s mugger scenario, but in this case you’re just screwed: reality likes to make you suffer, and you have to take the best possible world. Telling reality that it can go ahead and give you tons of disutility doesn’t take away its incentive to give you tons of disutility. There’s no way out of the problem.
That opens a whole new can of worms that it’s far too late at night for me to address, but I’m thinking of writing a post on this soon, perhaps tomorrow.
Cool! Be careful not to generalize too much, though: there might bad general trends, but no one likes to be yelled at for things they didn’t do. Try to frame it as humbly as possible, maybe. Sounding unsure of your position when arguing against LW norms gets you disproportionately large amounts of karma. Game the system!
In this case of Pascal’s Mugging both choices lead to negative expected utility as defined by the problem. Hence you look for a third option, and in this case, you find one: ignore all blackmailers; tell them to go ahead and torture all those people, you don’t care.
That works for the LW version of the problem (and I understand why it does), but not for Bostrom’s original formulation. In that version the mugger claims to have magic powers and will give Pascal quadrillions of utility if he hands over his wallet. This means that the mugger avoids the rule “ignore all threats of blackmail but accept postiive-sum trades.” That’s why it looks so much like cryonics to me, and therein lies the problem.
Sounding unsure of your position when arguing against LW norms gets you disproportionately large amounts of karma. Game the system!
Will do! I obviously don’t want to sound obnoxious; there’s no reason to be rude about rationality.
In that version the mugger claims to have magic powers and will give Pascal quadrillions of utility if he hands over his wallet.
Oh, sorry! In that case all my talk was egregious. That sounds like a much better problem whose answer isn’t immediately obvious to me. I shall think about it.
That sounds like a much better problem whose answer isn’t immediately obvious to me.
Yep, that’s the problem I’ve been struggling with. Like I said, it would help if Pascal’s disbelief in the mugger’s powers scaled with the utility the mugger promises him, but there’s not always a reason for that to be so. In any case, it might help to look at Bostrom’s version. And do let me know if you come up with anything, since this one really bothers me.
In any case, it might help to look at Bostrom’s version. And do let me know if you come up with anything, since this one really bothers me.
Thanks for pointing this out, I’m shocked I hadn’t heard of it. I’ll let you know if I think up something. If I can’t, I’ll ask a decision theory veteran, they’re sure to know.
Excellent, that’ll be a fun read.
I’m still not sure if I follow this—I’ll have to do some more reading on it. I still don’t see how the two situations are different—for example, if I was talking to someone selling cryonics, wouldn’t that be qualitatively the same as Pascal’s Mugging? I’m not sure.
Unfortunately no, it was here. I didn’t look at that article until recently.
That opens a whole new can of worms that it’s far too late at night for me to address, but I’m thinking of writing a post on this soon, perhaps tomorrow.
Nah, the cryonics agent isn’t trying to mug you! (Er, hopefully.) He’s just giving you two options and letting you calculate.
In this case of Pascal’s Mugging both choices lead to negative expected utility as defined by the problem. Hence you look for a third option, and in this case, you find one: ignore all blackmailers; tell them to go ahead and torture all those people, you don’t care. Unless they find joy in torturing people (then you’re screwed) they have no incentive to actually use up the resources to go through with it. So they leave you alone, ’cuz you won’t budge.
Cryonics is a lot simpler in its nature, but a lot harder to calculate. You have two options, and the options are given to you by reality, not an agent you can outwit. (Throwing in a cryonics agent doesn’t change anything.) When you have to choose between the binary cryonics versus no cryonics, it’s just a matter of seeing which decision is better (or worse). It could be that both are bad, like in the Pascal’s mugger scenario, but in this case you’re just screwed: reality likes to make you suffer, and you have to take the best possible world. Telling reality that it can go ahead and give you tons of disutility doesn’t take away its incentive to give you tons of disutility. There’s no way out of the problem.
Cool! Be careful not to generalize too much, though: there might bad general trends, but no one likes to be yelled at for things they didn’t do. Try to frame it as humbly as possible, maybe. Sounding unsure of your position when arguing against LW norms gets you disproportionately large amounts of karma. Game the system!
That works for the LW version of the problem (and I understand why it does), but not for Bostrom’s original formulation. In that version the mugger claims to have magic powers and will give Pascal quadrillions of utility if he hands over his wallet. This means that the mugger avoids the rule “ignore all threats of blackmail but accept postiive-sum trades.” That’s why it looks so much like cryonics to me, and therein lies the problem.
Will do! I obviously don’t want to sound obnoxious; there’s no reason to be rude about rationality.
Oh, sorry! In that case all my talk was egregious. That sounds like a much better problem whose answer isn’t immediately obvious to me. I shall think about it.
Yep, that’s the problem I’ve been struggling with. Like I said, it would help if Pascal’s disbelief in the mugger’s powers scaled with the utility the mugger promises him, but there’s not always a reason for that to be so. In any case, it might help to look at Bostrom’s version. And do let me know if you come up with anything, since this one really bothers me.
Thanks for pointing this out, I’m shocked I hadn’t heard of it. I’ll let you know if I think up something. If I can’t, I’ll ask a decision theory veteran, they’re sure to know.
I’m not so sure, but I certainly hope someone knows.