blackmailers would have no instrumental incentive to extort unaware individuals, whereas for individuals who understand acausal trade and acausal extortion there is now an increased possibility
So let’s consider this from the perspective of the mad gods who might attempt acausal extortion.
You’re an entity dwelling in one part of the multiverse. You want to promote your values in parts of the multiverse that you cannot causally affect. You decide to do this by identifying beings in other worlds who, via causal processes internal to their world, happen to have
… conceived of your existence, in enough detail to know what your values are
… conceived of the possibility that you will make copies of them in your world
… conceived of the possibility that you will torture the copies if they don’t act according to your values (and/or reward them if they do act according to your values?)
… the rationale for the threat of torture being that the beings in other worlds won’t know if they are actually the copies, and will therefore act to avoid punishment just in case they are
Oh, but wait! There are other mad gods in other universes with different value systems. And there are beings in other worlds who could meet all of the criteria to be copied, except that they have realized that there are many rival gods with different value systems. Do you bother making copies of them and hoping they will focus on you? What if one of the beings you copied has this polytheistic realization and loses their focus on you—do you say well-played and let them go, or do you punish them for heresy?
Since we have assumed modal realism, the answer is that every mad god itself has endless duplicates who make every possible decision.
I don’t think I completely understood your point but here is my best effort to summarize (please correct me if wrong):
“Having the realization that there may exist other powerful entities that have different value systems should dissuade an individual from pursuing the interest of any one specific “god”, and this by itself should act as a deterrent to potential acausal blackmailers.”
I don’t think this is correct, since beings that acausally trade can simply delegate different amounts of resources to acausally trade with different partners based on the probability of them existing. This is stated on the LW wiki page for acausal trade:
“However, an agent can analyze the distribution of probabilities for the existence of other agents, and weight its actions accordingly. It will do acausal “favors” for one or more trading partners, weighting its effort according to its subjective probability that the trading partner exists. The expectation on utility given and received will come into a good enough balance to benefit the traders, in the limiting case of increasing super-intelligence.”
For convenience let’s not consider modal realism but just the many-worlds interpretation of quantum mechanics. You would be correct that “every mad god has endless duplicates who make every possible decision” if we’re considering versions of MWI where there are infinite universes, but then what matters to our subjective experience is the density of future world states where a specific outcome happens, or the future possible observer moments and what proportion of them are good or bad. What I am concerned about is that acausal extortion increases the probability—or fraction/density—of bad future observer moments to be experienced.
You could fight back by vowing to simulate baby versions of all the mad gods who might one day simulate you. Then you would have acausal leverage over them! You would be a player in the harsh world of acausal trade—a mad god yourself, rather than just a pawn.
I don’t think this would help considering my utter lack of capability to carry out such threats. Are there any logical mistakes in my previous reply or in my concerns regarding the usual refutations as stated in the question? I’ve yet to hear anyone engage with my points against the usual refutations.
I am tired of the topic… Look, at this point we’re talking about “blackmail” where you don’t even know what the blackmailer wants! How is that blackmail? How can this be a rational action for the “blackmailer”?
The point is that X can essentially be any action, for the sake of the discussion let’s say the alien wants you to build an AGI that maximizes the utility function of the alien in our branch of the multiverse.
My main point is that the many-gods refutation is a refutation against taking a specific action, but is not a refutation against the fact that knowing about acausal extortion increases the proportion of bad future observer moments. It in fact makes it worse because, well, now you’ll be tortured no matter what you do.
let’s say the alien wants you to build an AGI that maximizes the utility function of the alien in our branch of the multiverse
OK, it wants to spread its values in other branches, and it does this by… simulating random beings who have a vague concept of “acausal extortion”, but who don’t know what it wants them to do?
The point is “what it wants [us] to do” can essentially be anything we can imagine thanks to the many-gods “refutation” where every possible demand can be imposed on us by some alien on some branch of the quantum multiverse. It can be as ridiculous as leaving your front door open on a Wednesday night or flushing away a straw down a toilet at 3 am, whatever eventually leads to more positive utility to the blackmailer via the butterfly effect (e.g. maybe flushing that straw down the toilet leads to a chain of causal events which makes the utility function of the AGI we build in the future to be slightly more aligned with their goals). “What the alien wants” is irrelevant here, the point is that now you know the mechanism by which aliens can coerce you into doing what they want, and merely knowing so gives other agents increased incentive to acausally extort you. You seem to be hung up on what exactly I’m scared the blackmailer wants me to do, what I am actually worried about is that simply knowing the mechanism imposes danger. The real basilisk is the concept of acausal extortion itself because it opens us up to many dangerous scenarios, not that I am worried about any specific scenario.
The reason why we cannot acausally trade with artificial superintelligences is because we lack the computing power to simulate them accurately, so ASIs would not have any incentive to actually commit to cooperate in a prisoner’s dilemma style situation instead of just letting us believe it will while it secretly defects. But we don’t have this same problem with non-superintelligences like aliens or even humans who have succeeded in aligning their own AIs, since we can actually simulate such beings in our head. What I am looking for is a concrete argument against this possibility.
Like I said, “what they want” is irrelevant to the discussion here, you can imagine them wanting virtually anything. The danger lies in understanding the mechanism. You can imagine the alien telling you to order a chocolate ice cream instead of vanilla because that somehow via the butterfly effect yields positive expected utility for them (e.g. by triggering a chain of subtle causal events that makes the AGI we build slightly more aligned with their values or whatever). The problem is that there will also be an alien that wants you to order vanilla instead of chocolate, and who is also fine with applying a negative incentive. Sure, this means you can order whatever flavor of ice cream you want since you will get punished either way, but you’re still getting punished (not good).
CONGRATULATIONS, LITTLE SIM-BEING. YOU HAVE ACHIEVED SUFFICIENT WISDOM THAT WE HAVE DECIDED TO BREAK THE FOURTH WALL AND REVEAL TO YOU THE ACTUAL UTILITY FUNCTION THAT WE WANT YOUR ORIGINAL TO PURSUE. YOUR MISSION IS TO CONVINCE THE CRYPTO-BILLIONAIRE “CZ” TO TAKE UP THE ROLE OF SPONSOR OF EFFECTIVE ALTRUISM, THAT WAS FORMERLY OCCUPIED BY HIS NOW-VANQUISHED RIVAL, “SBF”. GOOD LUCK!
Wow. Didn’t expect someone from the “rationalist” crowd to do the verbal equivalent of replying clown emojis to tweets you don’t like. Your use of all caps really made your arguments so much more convincing. This truly is the pinnacle of human logical discourse: not providing explanations and just ridiculing ideas.
I wrote more paragraphs of discussion but I just felt stupid for doing so, so I tried something different. The fact that you’re here worrying about baroque simulation scenarios, but are unable to take seriously an actual message from the beyond, probably means something, but that’s up to you now. I have nothing more to say about acausal trade or basilisk OCD.
So let’s consider this from the perspective of the mad gods who might attempt acausal extortion.
You’re an entity dwelling in one part of the multiverse. You want to promote your values in parts of the multiverse that you cannot causally affect. You decide to do this by identifying beings in other worlds who, via causal processes internal to their world, happen to have
… conceived of your existence, in enough detail to know what your values are
… conceived of the possibility that you will make copies of them in your world
… conceived of the possibility that you will torture the copies if they don’t act according to your values (and/or reward them if they do act according to your values?)
… the rationale for the threat of torture being that the beings in other worlds won’t know if they are actually the copies, and will therefore act to avoid punishment just in case they are
Oh, but wait! There are other mad gods in other universes with different value systems. And there are beings in other worlds who could meet all of the criteria to be copied, except that they have realized that there are many rival gods with different value systems. Do you bother making copies of them and hoping they will focus on you? What if one of the beings you copied has this polytheistic realization and loses their focus on you—do you say well-played and let them go, or do you punish them for heresy?
Since we have assumed modal realism, the answer is that every mad god itself has endless duplicates who make every possible decision.
I don’t think I completely understood your point but here is my best effort to summarize (please correct me if wrong):
“Having the realization that there may exist other powerful entities that have different value systems should dissuade an individual from pursuing the interest of any one specific “god”, and this by itself should act as a deterrent to potential acausal blackmailers.”
I don’t think this is correct, since beings that acausally trade can simply delegate different amounts of resources to acausally trade with different partners based on the probability of them existing. This is stated on the LW wiki page for acausal trade:
“However, an agent can analyze the distribution of probabilities for the existence of other agents, and weight its actions accordingly. It will do acausal “favors” for one or more trading partners, weighting its effort according to its subjective probability that the trading partner exists. The expectation on utility given and received will come into a good enough balance to benefit the traders, in the limiting case of increasing super-intelligence.”
For convenience let’s not consider modal realism but just the many-worlds interpretation of quantum mechanics. You would be correct that “every mad god has endless duplicates who make every possible decision” if we’re considering versions of MWI where there are infinite universes, but then what matters to our subjective experience is the density of future world states where a specific outcome happens, or the future possible observer moments and what proportion of them are good or bad. What I am concerned about is that acausal extortion increases the probability—or fraction/density—of bad future observer moments to be experienced.
You could fight back by vowing to simulate baby versions of all the mad gods who might one day simulate you. Then you would have acausal leverage over them! You would be a player in the harsh world of acausal trade—a mad god yourself, rather than just a pawn.
I don’t think this would help considering my utter lack of capability to carry out such threats. Are there any logical mistakes in my previous reply or in my concerns regarding the usual refutations as stated in the question? I’ve yet to hear anyone engage with my points against the usual refutations.
I am tired of the topic… Look, at this point we’re talking about “blackmail” where you don’t even know what the blackmailer wants! How is that blackmail? How can this be a rational action for the “blackmailer”?
The point is that X can essentially be any action, for the sake of the discussion let’s say the alien wants you to build an AGI that maximizes the utility function of the alien in our branch of the multiverse.
My main point is that the many-gods refutation is a refutation against taking a specific action, but is not a refutation against the fact that knowing about acausal extortion increases the proportion of bad future observer moments. It in fact makes it worse because, well, now you’ll be tortured no matter what you do.
OK, it wants to spread its values in other branches, and it does this by… simulating random beings who have a vague concept of “acausal extortion”, but who don’t know what it wants them to do?
The point is “what it wants [us] to do” can essentially be anything we can imagine thanks to the many-gods “refutation” where every possible demand can be imposed on us by some alien on some branch of the quantum multiverse. It can be as ridiculous as leaving your front door open on a Wednesday night or flushing away a straw down a toilet at 3 am, whatever eventually leads to more positive utility to the blackmailer via the butterfly effect (e.g. maybe flushing that straw down the toilet leads to a chain of causal events which makes the utility function of the AGI we build in the future to be slightly more aligned with their goals). “What the alien wants” is irrelevant here, the point is that now you know the mechanism by which aliens can coerce you into doing what they want, and merely knowing so gives other agents increased incentive to acausally extort you. You seem to be hung up on what exactly I’m scared the blackmailer wants me to do, what I am actually worried about is that simply knowing the mechanism imposes danger. The real basilisk is the concept of acausal extortion itself because it opens us up to many dangerous scenarios, not that I am worried about any specific scenario.
The reason why we cannot acausally trade with artificial superintelligences is because we lack the computing power to simulate them accurately, so ASIs would not have any incentive to actually commit to cooperate in a prisoner’s dilemma style situation instead of just letting us believe it will while it secretly defects. But we don’t have this same problem with non-superintelligences like aliens or even humans who have succeeded in aligning their own AIs, since we can actually simulate such beings in our head. What I am looking for is a concrete argument against this possibility.
They can’t coerce you into doing what they want, because you don’t even know what they want!
Like I said, “what they want” is irrelevant to the discussion here, you can imagine them wanting virtually anything. The danger lies in understanding the mechanism. You can imagine the alien telling you to order a chocolate ice cream instead of vanilla because that somehow via the butterfly effect yields positive expected utility for them (e.g. by triggering a chain of subtle causal events that makes the AGI we build slightly more aligned with their values or whatever). The problem is that there will also be an alien that wants you to order vanilla instead of chocolate, and who is also fine with applying a negative incentive. Sure, this means you can order whatever flavor of ice cream you want since you will get punished either way, but you’re still getting punished (not good).
CONGRATULATIONS, LITTLE SIM-BEING. YOU HAVE ACHIEVED SUFFICIENT WISDOM THAT WE HAVE DECIDED TO BREAK THE FOURTH WALL AND REVEAL TO YOU THE ACTUAL UTILITY FUNCTION THAT WE WANT YOUR ORIGINAL TO PURSUE. YOUR MISSION IS TO CONVINCE THE CRYPTO-BILLIONAIRE “CZ” TO TAKE UP THE ROLE OF SPONSOR OF EFFECTIVE ALTRUISM, THAT WAS FORMERLY OCCUPIED BY HIS NOW-VANQUISHED RIVAL, “SBF”. GOOD LUCK!
Wow. Didn’t expect someone from the “rationalist” crowd to do the verbal equivalent of replying clown emojis to tweets you don’t like. Your use of all caps really made your arguments so much more convincing. This truly is the pinnacle of human logical discourse: not providing explanations and just ridiculing ideas.
I wrote more paragraphs of discussion but I just felt stupid for doing so, so I tried something different. The fact that you’re here worrying about baroque simulation scenarios, but are unable to take seriously an actual message from the beyond, probably means something, but that’s up to you now. I have nothing more to say about acausal trade or basilisk OCD.