I am kinda surprised that you are in such a muddle about this—and are willing to patronise me over the issue!
“Don’t negotiate with terrorists” and “don’t give into extortion” are well-known maxims. As this thread illustrates, you don’t seem to understand why they exist. I do understand. It isn’t terribly complicated. I expect I can explain it to you.
If a government gives in to terrorist demands during a hijacking, it sends a signal to all the other terrorists in the world that the government is vulnerable to extortion. Subsequently the government is likely to face more hijackings.
So… in addition to the obvious cost associated with the immediate demands of the terrorists, there is a hidden cost associated with gaining a reputation for giving in to terrorists. That hidden cost is often huge. Thus the strategy of not giving in to terrorist demands—even if doing so looks attractive on the basis of a naive cost-benefit analysis.
Other forms of extortion exhibit similar dynamics...
So, in addition to the obvious cost associated with the immediate demands of the terrorists, there is a hidden cost associated with getting a reputation for giving in to terrorists. That hidden cost is often huge. Thus the strategy of not giving in to terrorists.
So if Thud cooperated with some less drastic version of Fred’s plan that left a future to care about, he would be causing humans to get a reputation for giving in to extortion, even if the particular extortion he was faced with would not have been prevented by the aliens knowing he probably would not have given in. This is a different argument from the backward causality UDT seems to use in this situation, and AIXI could get it right by simulating the behavior of the next extortionist.
I am kinda surprised that you are in such a muddle about this—and are willing to patronise me over the issue!
“Don’t negotiate with terrorists” and “don’t give into extortion” are well-known maxims. As this thread illustrates, you don’t seem to understand why they exist. I do understand. It isn’t terribly complicated. I expect I can explain it to you.
If a government gives in to terrorist demands during a hijacking, it sends a signal to all the other terrorists in the world that the government is vulnerable to extortion. Subsequently the government is likely to face more hijackings.
So… in addition to the obvious cost associated with the immediate demands of the terrorists, there is a hidden cost associated with gaining a reputation for giving in to terrorists. That hidden cost is often huge. Thus the strategy of not giving in to terrorist demands—even if doing so looks attractive on the basis of a naive cost-benefit analysis.
Other forms of extortion exhibit similar dynamics...
So if Thud cooperated with some less drastic version of Fred’s plan that left a future to care about, he would be causing humans to get a reputation for giving in to extortion, even if the particular extortion he was faced with would not have been prevented by the aliens knowing he probably would not have given in. This is a different argument from the backward causality UDT seems to use in this situation, and AIXI could get it right by simulating the behavior of the next extortionist.
Good idea. Thanks for posting.