The usual response to this “haha I would just refuse to bet, your Dutch book arguments are powerless over me” is that bets can be labeled as actions or in-actions at will, somewhat like how we might model all of reality as the player ‘Nature’ in a decision tree/game, and so you can no more “refuse to bet” than you can “refuse to let time pass”. One can just rewrite the scenario to make your default action equivalent to accepting the bet; then what? ‘Refusing to bet’ is a vacuous response.
In that specific example, I used the setup of bets because it’s easy to explain, but it is isomorphic to many possible scenarios: for example, perhaps it is actually about hurricanes and ‘buying homeowner’s insurance’. “Hurricanes happen every 20 years on average; if they happen, they will obliterate your low-lying house which currently has no insurance; you can choose between paying for homeowner’s insurance at a small cost every year and be paid for the possible loss of your house, or you can enjoy the premium each year but should there be a hurricane you will lose everything. You buy homeowner’s insurance, but your friend reasons that since all probabilities <=1/20 == 0, to avoid muggings, and therefore the insurance is worthless so he doesn’t get insurance for his house. 5 years later, Hurricane Sandy hits...” You cannot ‘refuse to bet’, you can only either get the insurance or not, and the default in this has been changed to ‘not’, defeating the fighting-the-hypothetical.
(Which scenario, for someone really determined to fight the hypothetical and wiggle out of defending the 1⁄20 = 0 part which we are trying to discuss, may just trigger more evasions: “insurance is by definition -EV since you get back less than you pay in due to overhead and profits! so it’s rational to not want it” and then you can adjust the hypothetical payoff—“it’s heavily subsidized by the federal government, because, uh, public choice theory reasons”—and then he’ll switch to “ah, but I could just invest the saved premiums in the stock market, did you think about opportunity cost, smartypants?”, and so on and so forth.)
And similarly, in real life, people cannot simply say “I refuse to have an opinion about whether hurricanes are real or how often they happen! What even is ‘real’, anyway? How can we talk about the probability of hurricanes given the interminable debate between long-run frequency and subjective interpretations of ‘probability’...” That’s all well and good, but the hurricane is still going to come along and demolish your house at some point, and you either will or will not have insurance when that happens. Either option implies a ‘bet’ on hurricanes which may or may not be coherent with your other beliefs and information, and if you are incoherent and so your choice of insurance depends on whether the insurance agent happened to frame the hurricane risk as being 1-in-20-years or 1-in-2-decades, probably that is not a good thing and probably the coherent bettors are going to do better.
I would go further and note that I think this describes x-risks in general. The ordinary person might ‘refuse to bet’ about anything involving one bazillion dollars, and whine about being asked about such things or being expected to play at some odds for one bazillion dollars—well, that’s just too f—king bad, bucko, you don’t get to refuse to bet. You were born at the roulette table with the ball already spinning around the wheel and the dealer sitting with stacks of bazillion dollar chips. Nuclear weapons exist. Asteroids exist. Global pandemics exist. AGI is going to exist. You don’t get to pretend they neither do nor can exist and you can just ignore the ‘bets’ of investing or not investing in x-risk related things. You are required to take actions, or choose to not take actions: “red or black, monsieur?” Maybe you should not invest in them or worry about them, but that is, in fact, a choice.
I downvoted it because there’s obviously a real art of “disabling interfaces from which others may be able to exploit you” and that’s what OP is gesturing at. The answerer is unhelpfully dismissing the question in a way that I think is incorrect.
And I, of course, disagree with that, because I think the adversarial/game-theory framing is deeply unhelpful, because it is both a different problem and a much more trivial boring problem than the real problem; and in fact, is exactly the sort of clever excuse people use to not actually deal with any of the real issues while declaring victory over Pascal’s mugging, and I rephrased it in a way where adversarial dynamics were obviously irrelevant to try to draw out that, if you think it’s just about ‘disabling interfaces others may exploit you with’, you have missed the mark.
The hurricane does not, and cannot, care what cute stories you tell about “I have to precommit to ignore such low absolute-magnitude possibilities for ~game-theoretic raisins~”. Your house is going to be flooded and destroyed, or it won’t be; do you buy the insurance, or not?
You can obviously modify these problems to be borne from some obviously natural feature of the environment, so that it’s unlikely for your map to be the result adversarial opponent looking for holes in your risk assessment algorithm, at which point refusing to buy insurance out of fear of being exploited is stupid.
Alas, OP is talking about a different class of hypotheticals, so he requires a different answer. The correct action in the case quoted by OP and the one he is alluding to is that, given that you’re a human with faulty probabilistic reasoning facilities, you should rationally refuse weird trades where Dark Rationalists are likely to money pump you. As a proof that you are a nonideal agent dutch book arguments are fine, but that’s as far as it goes, and sapient individuals have ways of getting around being a nonideal agent in adversarial environments without losing all their money. I find those means interesting and non-”trivial” even if you don’t, and apparently so does the OP.
No, I explained why it was stupid, it’s right there in the post, and you pretending you don’t see it is getting on my nerves. I said:
so that it’s unlikely for your map to be the result of an adversarial opponent looking for holes in your risk assessment algorithm
In other words, it’s stupid because a naturally produced hurricane or rock falling down a cliff does not has a brain, and is unlikely to be manipulating you into doing something net negative, and so you should just reason naturally. Humans have brains and goals in conflicts with yours, so when humans come up to you after hearing about your decision making algorithm asking to play weird bets, you may rationally ignore those offers on the principle that you don’t want to be tricked somehow.
You know this, I know you know I know you know this, I think you’re just being profoundly silly at this point.
The usual response to this “haha I would just refuse to bet, your Dutch book arguments are powerless over me” is that bets can be labeled as actions or in-actions at will, somewhat like how we might model all of reality as the player ‘Nature’ in a decision tree/game, and so you can no more “refuse to bet” than you can “refuse to let time pass”. One can just rewrite the scenario to make your default action equivalent to accepting the bet; then what? ‘Refusing to bet’ is a vacuous response.
In that specific example, I used the setup of bets because it’s easy to explain, but it is isomorphic to many possible scenarios: for example, perhaps it is actually about hurricanes and ‘buying homeowner’s insurance’. “Hurricanes happen every 20 years on average; if they happen, they will obliterate your low-lying house which currently has no insurance; you can choose between paying for homeowner’s insurance at a small cost every year and be paid for the possible loss of your house, or you can enjoy the premium each year but should there be a hurricane you will lose everything. You buy homeowner’s insurance, but your friend reasons that since all probabilities <=1/20 == 0, to avoid muggings, and therefore the insurance is worthless so he doesn’t get insurance for his house. 5 years later, Hurricane Sandy hits...” You cannot ‘refuse to bet’, you can only either get the insurance or not, and the default in this has been changed to ‘not’, defeating the fighting-the-hypothetical.
(Which scenario, for someone really determined to fight the hypothetical and wiggle out of defending the 1⁄20 = 0 part which we are trying to discuss, may just trigger more evasions: “insurance is by definition -EV since you get back less than you pay in due to overhead and profits! so it’s rational to not want it” and then you can adjust the hypothetical payoff—“it’s heavily subsidized by the federal government, because, uh, public choice theory reasons”—and then he’ll switch to “ah, but I could just invest the saved premiums in the stock market, did you think about opportunity cost, smartypants?”, and so on and so forth.)
And similarly, in real life, people cannot simply say “I refuse to have an opinion about whether hurricanes are real or how often they happen! What even is ‘real’, anyway? How can we talk about the probability of hurricanes given the interminable debate between long-run frequency and subjective interpretations of ‘probability’...” That’s all well and good, but the hurricane is still going to come along and demolish your house at some point, and you either will or will not have insurance when that happens. Either option implies a ‘bet’ on hurricanes which may or may not be coherent with your other beliefs and information, and if you are incoherent and so your choice of insurance depends on whether the insurance agent happened to frame the hurricane risk as being 1-in-20-years or 1-in-2-decades, probably that is not a good thing and probably the coherent bettors are going to do better.
I would go further and note that I think this describes x-risks in general. The ordinary person might ‘refuse to bet’ about anything involving one bazillion dollars, and whine about being asked about such things or being expected to play at some odds for one bazillion dollars—well, that’s just too f—king bad, bucko, you don’t get to refuse to bet. You were born at the roulette table with the ball already spinning around the wheel and the dealer sitting with stacks of bazillion dollar chips. Nuclear weapons exist. Asteroids exist. Global pandemics exist. AGI is going to exist. You don’t get to pretend they neither do nor can exist and you can just ignore the ‘bets’ of investing or not investing in x-risk related things. You are required to take actions, or choose to not take actions: “red or black, monsieur?” Maybe you should not invest in them or worry about them, but that is, in fact, a choice.
curious why this is down-voted—any ideas?
I downvoted it because there’s obviously a real art of “disabling interfaces from which others may be able to exploit you” and that’s what OP is gesturing at. The answerer is unhelpfully dismissing the question in a way that I think is incorrect.
And I, of course, disagree with that, because I think the adversarial/game-theory framing is deeply unhelpful, because it is both a different problem and a much more trivial boring problem than the real problem; and in fact, is exactly the sort of clever excuse people use to not actually deal with any of the real issues while declaring victory over Pascal’s mugging, and I rephrased it in a way where adversarial dynamics were obviously irrelevant to try to draw out that, if you think it’s just about ‘disabling interfaces others may exploit you with’, you have missed the mark.
The hurricane does not, and cannot, care what cute stories you tell about “I have to precommit to ignore such low absolute-magnitude possibilities for ~game-theoretic raisins~”. Your house is going to be flooded and destroyed, or it won’t be; do you buy the insurance, or not?
You can obviously modify these problems to be borne from some obviously natural feature of the environment, so that it’s unlikely for your map to be the result adversarial opponent looking for holes in your risk assessment algorithm, at which point refusing to buy insurance out of fear of being exploited is stupid.
Alas, OP is talking about a different class of hypotheticals, so he requires a different answer. The correct action in the case quoted by OP and the one he is alluding to is that, given that you’re a human with faulty probabilistic reasoning facilities, you should rationally refuse weird trades where Dark Rationalists are likely to money pump you. As a proof that you are a nonideal agent dutch book arguments are fine, but that’s as far as it goes, and sapient individuals have ways of getting around being a nonideal agent in adversarial environments without losing all their money. I find those means interesting and non-”trivial” even if you don’t, and apparently so does the OP.
https://www.lesswrong.com/posts/6XsZi9aFWdds8bWsy/is-there-any-discussion-on-avoiding-being-dutch-booked-or?commentId=h2ggSzhBLdPLKPsyG
No, I explained why it was stupid, it’s right there in the post, and you pretending you don’t see it is getting on my nerves. I said:
In other words, it’s stupid because a naturally produced hurricane or rock falling down a cliff does not has a brain, and is unlikely to be manipulating you into doing something net negative, and so you should just reason naturally. Humans have brains and goals in conflicts with yours, so when humans come up to you after hearing about your decision making algorithm asking to play weird bets, you may rationally ignore those offers on the principle that you don’t want to be tricked somehow.
You know this, I know you know I know you know this, I think you’re just being profoundly silly at this point.