That’s not the point. None of those probabilities are as strong as 3^^^3. Maybe big, buy not THAT big.
The point is that no more than 1/3^^^3 people have sole control over the life or death of 3^^3 people. This improbability, that you would be one of those very special people, IS big enough.
(This answer fails unless your ethics and anthropics use the same measure. That’s how the pig example works.)
Even if the Matrix-claimant says that the 3^^^^3 minds created will be unlike you, with information that tells them they’re powerless, if you’re in a generalized scenario where anyone has and uses that kind of power, the vast majority of mind-instantiations are in leaves rather than roots.
The point is that no more than 1/3^^^3 people have sole control
I was about to express mild amusement about how cavalier we are with jumping to, from and between numbers like 3^^^^3 and 3^^^3. I had to squint to tell the difference. Then it occurred to me that:
The point is that no more than 1/3^^^3 people have sole control over the life or death of 3^^3 people. This improbability, that you would be one of those very special people, IS big enough.
3^^3 is not even unimaginably big, Knuth arrows or no. It’s about 1/5th the number of people that can fit in the MCG.
Being cavalier with proofreading =/= being cavalier with number size.
Well, I didn’t want to declare a proofreading error because 3^^^3 does technically fit correctly in the context, even if you may not have meant it. ;)
I was thinking the fact that we are so cavalier makes it easier to slip between them if not paying close attention. Especially since 3^^^3 is more commonly used than 3^^^^3. I don’t actually recall Eliezer going beyond pentation elsewhere.
I know if I go that high I tend to use 4^^^^4. It appeals more aesthetically and is more clearly distinct. Mind you it isn’t nearly as neat as 3^^^3 given that 3^^^3 can also be written and visualized conceptually as 3 → 3 → 3 while 4^^^^4 is just 4 → 4 → 4 not 4 → 4 → 4 → 4.
The point is that no more than 1/3^^^3 people have sole control over the life or death of 3^^3 people. This improbability, that you would be one of those very special people, IS big enough.
(This answer fails unless your ethics and anthropics use the same measure. That’s how the pig example works.)
So can we solve the problem by putting some sort of upper bound on the degree to which ethics and anthropics can differ, along the lines of “creation of 3^^^^3 people is at most N times less probable than creation of 3^^^^3 pigs, so across the ensemble of possible worlds the prior against your being in a position to influence that many pigs still cuts down the expected utility from something vaguely like 3^^^^3 to something vaguely like N”?
So you’re saying that the implausibility is that I’d run into a person that just happened to have that level of “power” ?
Is that different in kind to what I was saying?
If I find it implausible that the person I’m speaking to can actually do what they’re claiming, is that not the same as it being implausible that I happen to have met a person that can do what this person is claiming/ (leaving aside the resource-question which is probably just my rationalisation as to why I think he couldn’t pull it off).
Basically I’m trying to taboo the actual BigNum… and trying to fit the concepts around in my head.
It’s implausible that you’re the person with that power. We could easily imagine a world in which everyone runs into a single absurdly powerful person. We could not imagine a world in which everyone was absurdly powerful (in their ability to control other people), because then multiple people would have control over the same thing.
If you knew that he had the power, but that his action wasn’t going to depend on yours, then you wouldn’t give him the money. So you’re only concerned with the situation where you have the power.
Ok, sure thing. I get what you’re saying.
I managed to encompass that implausibility also into the arguments I made in my restatement anyway, but yeah, I agree that these are different kinds of “unlikely thing”
In fact… let me restate what I think I was trying to say.
The mugger is making an extraordinary claim. One for which he has provided no evidence.
The amount of evidence required to make me believe that his claim is possible, grows at the same proportion as the size of his claim.
Think about it at the lower levels of potential claims.
1)
If he claimed to be able to kill one person—I’d believe that he was capable of killing one person. I’d then weigh that against the likelihood that he’d pick me to blackmail, and the low blackmail amount that he’d picked… and consider it more likely that he’s lying to make a fast buck, than that he actually has a hostage somewhere ready to kill.
2)
If he claimed to be able to kill 3^3 people, I’d consider it plausible… with a greatly diminished likelihood. I’d have to weigh the evidence that he was a small-time terrorist, willing to take the strong risk of being caught while preparing to blow up a buildings-worth of people… or to value his life so low as to actually do it and die in the process. It’s not very high, but we’ve all seen people like this in our lifetime both exist and carry out this threat. So it’s “plausible but extremely unlikely”.
The likelihood that I’ve: a) happened to run into one of these rare people and b) that he’d pick me (pretty much a nobody) to blackmail combine to be extremely unlikely… and I’d reckon that those two, balanced against the much higher prior likelihood that he’s just a con-artist, would fairly well cancel out against the actual value of a buildings-worth of people.
Especially when you consider that the resources to do this would far outweigh the money he’s asked for. As far as I know about people wiling to kill large numbers of people—most of them do it for a reason, and that reason is almost never a paltry amount of cash. It’s still possible… after all the school-killers have done crazy stunts to kill people for a tiny reason… but usually there’s fame or revenge involved… not blackmail of a nobody.
3)
So now we move to 3^^3 people.
Now, I personally have never seen that many die in one sitting (or even as the result of a single person)… but my Grandfather did, and using technology from 65 years ago.
It is plausible, though even less likely than before, that the person I’ve just run into happens to be willing and able to use a nuke on a large city, or to have the leadership capabilities (and luck) required to take over a country and divert it’s resources to killing that number of people.
I would consider it exponentially less likely that he’d pick me to blackmail about this… and certainly not for such a pitiful amount of cash. People that threaten this kind of thing are either after phenomenal amounts of money, recognition or some kind of political or religious statement.… they are extremely unlikely to find a random citizen to blackmail for a tiny amount of cash. The likelihood that this is a con seems about as high as the number of people to potentially die.
4)
Now we hit the first real BigNum.
AFAIK, the world has never seen 3^^^3 sentient intelligences ever die in one sitting. We don’t have that many people on the Earth right now. Maybe the universe has seen it somewhere… some planetary system wiped out in a supernova. It’s plausible… but now think of the claims the guy is making:
a) that he can create (or knows of) a civilisation that contains that number of sentient beings.
b) that he (and he alone) has the ability to destroy that civilisation, and can do so at whim and that
c) it’s worthwhile him doing so for the mere pittance he’s demanding from a complete, unrelated nobody… (or potentially the whim of watching said nobody squirm).
I actually think that the required (and missing) evidence for his outrageous claims stack fairly evenly against the potential downside of his claims actually being true.
So, to get back to the original point:
In my mind, as each step grows exponentially more extreme, so does the evidence required to support such a ludicrous claim. These two cancel out roughly evenly, leaving the leftovers of “is he likely to have picked me?” and other smaller probabilities to actually sway the balance.
Those, added with the large disutility of “encouraging the guy to do it again” would sway me to choose not to give him £5, but to walk away, then immediately find the nearest police officer...
3) So now we move to 3^^3 people. Now, I personally have never seen that many die in one sitting (or even as the result of a single person)… but my Grandfather did, and using technology from 65 years ago.
3^^3 is a thousand times larger than the number of people currently alive.
Yes, I should not have used the word exponential… but I don’t know the word for “grows at a rate that is a tower of exponentials”… “hyperexponential” perhaps?
However—I consider that my argument still holds. That the evidence required grows at the same rate as the size of the claim.
My point in explaining the lower levels is that is that we don’t demand evidence from most claimants of small amounts of damage because we’ve already seen evidence that these threats are plausible. But if we start getting to the “hyperexponential” threats, we hit a point where we suddenly realise that there is no evidence supporting the plausibility of the claim… so we automatically assume that the person is a crank.
That’s not the point. None of those probabilities are as strong as 3^^^3. Maybe big, buy not THAT big.
The point is that no more than 1/3^^^3 people have sole control over the life or death of 3^^3 people. This improbability, that you would be one of those very special people, IS big enough.
(This answer fails unless your ethics and anthropics use the same measure. That’s how the pig example works.)
I was about to express mild amusement about how cavalier we are with jumping to, from and between numbers like 3^^^^3 and 3^^^3. I had to squint to tell the difference. Then it occurred to me that:
3^^3 is not even unimaginably big, Knuth arrows or no. It’s about 1/5th the number of people that can fit in the MCG.
Being cavalier with proofreading =/= being cavalier with number size.
But that is indeed amusing.
Well, I didn’t want to declare a proofreading error because 3^^^3 does technically fit correctly in the context, even if you may not have meant it. ;)
I was thinking the fact that we are so cavalier makes it easier to slip between them if not paying close attention. Especially since 3^^^3 is more commonly used than 3^^^^3. I don’t actually recall Eliezer going beyond pentation elsewhere.
I know if I go that high I tend to use 4^^^^4. It appeals more aesthetically and is more clearly distinct. Mind you it isn’t nearly as neat as 3^^^3 given that 3^^^3 can also be written and visualized conceptually as 3 → 3 → 3 while 4^^^^4 is just 4 → 4 → 4 not 4 → 4 → 4 → 4.
So can we solve the problem by putting some sort of upper bound on the degree to which ethics and anthropics can differ, along the lines of “creation of 3^^^^3 people is at most N times less probable than creation of 3^^^^3 pigs, so across the ensemble of possible worlds the prior against your being in a position to influence that many pigs still cuts down the expected utility from something vaguely like 3^^^^3 to something vaguely like N”?
So you’re saying that the implausibility is that I’d run into a person that just happened to have that level of “power” ?
Is that different in kind to what I was saying?
If I find it implausible that the person I’m speaking to can actually do what they’re claiming, is that not the same as it being implausible that I happen to have met a person that can do what this person is claiming/ (leaving aside the resource-question which is probably just my rationalisation as to why I think he couldn’t pull it off).
Basically I’m trying to taboo the actual BigNum… and trying to fit the concepts around in my head.
It’s implausible that you’re the person with that power. We could easily imagine a world in which everyone runs into a single absurdly powerful person. We could not imagine a world in which everyone was absurdly powerful (in their ability to control other people), because then multiple people would have control over the same thing.
If you knew that he had the power, but that his action wasn’t going to depend on yours, then you wouldn’t give him the money. So you’re only concerned with the situation where you have the power.
Ok, sure thing. I get what you’re saying. I managed to encompass that implausibility also into the arguments I made in my restatement anyway, but yeah, I agree that these are different kinds of “unlikely thing”
In fact… let me restate what I think I was trying to say.
The mugger is making an extraordinary claim. One for which he has provided no evidence.
The amount of evidence required to make me believe that his claim is possible, grows at the same proportion as the size of his claim.
Think about it at the lower levels of potential claims.
1) If he claimed to be able to kill one person—I’d believe that he was capable of killing one person. I’d then weigh that against the likelihood that he’d pick me to blackmail, and the low blackmail amount that he’d picked… and consider it more likely that he’s lying to make a fast buck, than that he actually has a hostage somewhere ready to kill.
2) If he claimed to be able to kill 3^3 people, I’d consider it plausible… with a greatly diminished likelihood. I’d have to weigh the evidence that he was a small-time terrorist, willing to take the strong risk of being caught while preparing to blow up a buildings-worth of people… or to value his life so low as to actually do it and die in the process. It’s not very high, but we’ve all seen people like this in our lifetime both exist and carry out this threat. So it’s “plausible but extremely unlikely”.
The likelihood that I’ve: a) happened to run into one of these rare people and b) that he’d pick me (pretty much a nobody) to blackmail combine to be extremely unlikely… and I’d reckon that those two, balanced against the much higher prior likelihood that he’s just a con-artist, would fairly well cancel out against the actual value of a buildings-worth of people.
Especially when you consider that the resources to do this would far outweigh the money he’s asked for. As far as I know about people wiling to kill large numbers of people—most of them do it for a reason, and that reason is almost never a paltry amount of cash. It’s still possible… after all the school-killers have done crazy stunts to kill people for a tiny reason… but usually there’s fame or revenge involved… not blackmail of a nobody.
3) So now we move to 3^^3 people. Now, I personally have never seen that many die in one sitting (or even as the result of a single person)… but my Grandfather did, and using technology from 65 years ago.
It is plausible, though even less likely than before, that the person I’ve just run into happens to be willing and able to use a nuke on a large city, or to have the leadership capabilities (and luck) required to take over a country and divert it’s resources to killing that number of people.
I would consider it exponentially less likely that he’d pick me to blackmail about this… and certainly not for such a pitiful amount of cash. People that threaten this kind of thing are either after phenomenal amounts of money, recognition or some kind of political or religious statement.… they are extremely unlikely to find a random citizen to blackmail for a tiny amount of cash. The likelihood that this is a con seems about as high as the number of people to potentially die.
4) Now we hit the first real BigNum. AFAIK, the world has never seen 3^^^3 sentient intelligences ever die in one sitting. We don’t have that many people on the Earth right now. Maybe the universe has seen it somewhere… some planetary system wiped out in a supernova. It’s plausible… but now think of the claims the guy is making:
a) that he can create (or knows of) a civilisation that contains that number of sentient beings.
b) that he (and he alone) has the ability to destroy that civilisation, and can do so at whim and that
c) it’s worthwhile him doing so for the mere pittance he’s demanding from a complete, unrelated nobody… (or potentially the whim of watching said nobody squirm).
I actually think that the required (and missing) evidence for his outrageous claims stack fairly evenly against the potential downside of his claims actually being true.
So, to get back to the original point: In my mind, as each step grows exponentially more extreme, so does the evidence required to support such a ludicrous claim. These two cancel out roughly evenly, leaving the leftovers of “is he likely to have picked me?” and other smaller probabilities to actually sway the balance.
Those, added with the large disutility of “encouraging the guy to do it again” would sway me to choose not to give him £5, but to walk away, then immediately find the nearest police officer...
3^^3 is a thousand times larger than the number of people currently alive.
oops, yes I mixed up 3^^3 with 3^^^3
Ok, so skip step 3 and move straight on to 4 ;)
3+3, 3*3, 3^3, 3^^3, 3^^^3, etc. grows much faster than exponentially. a^b, for any halfway reasonable a and b, can’t touch 3^^^3
3^^^3=3^^(3^^3)=3^^(7625597484987)=3^(3^^(7625597484986))
It’s not an exponential, it’s a huge, huge tower of exponentials. It is simply too big for that argument to work.
Yes, I should not have used the word exponential… but I don’t know the word for “grows at a rate that is a tower of exponentials”… “hyperexponential” perhaps?
However—I consider that my argument still holds. That the evidence required grows at the same rate as the size of the claim.
The evidence must be of equal value to the claim.
(from “extraordinary claims require extraordinary evidence”)
My point in explaining the lower levels is that is that we don’t demand evidence from most claimants of small amounts of damage because we’ve already seen evidence that these threats are plausible. But if we start getting to the “hyperexponential” threats, we hit a point where we suddenly realise that there is no evidence supporting the plausibility of the claim… so we automatically assume that the person is a crank.