Like I said, “what they want” is irrelevant to the discussion here, you can imagine them wanting virtually anything. The danger lies in understanding the mechanism. You can imagine the alien telling you to order a chocolate ice cream instead of vanilla because that somehow via the butterfly effect yields positive expected utility for them (e.g. by triggering a chain of subtle causal events that makes the AGI we build slightly more aligned with their values or whatever). The problem is that there will also be an alien that wants you to order vanilla instead of chocolate, and who is also fine with applying a negative incentive. Sure, this means you can order whatever flavor of ice cream you want since you will get punished either way, but you’re still getting punished (not good).
CONGRATULATIONS, LITTLE SIM-BEING. YOU HAVE ACHIEVED SUFFICIENT WISDOM THAT WE HAVE DECIDED TO BREAK THE FOURTH WALL AND REVEAL TO YOU THE ACTUAL UTILITY FUNCTION THAT WE WANT YOUR ORIGINAL TO PURSUE. YOUR MISSION IS TO CONVINCE THE CRYPTO-BILLIONAIRE “CZ” TO TAKE UP THE ROLE OF SPONSOR OF EFFECTIVE ALTRUISM, THAT WAS FORMERLY OCCUPIED BY HIS NOW-VANQUISHED RIVAL, “SBF”. GOOD LUCK!
Wow. Didn’t expect someone from the “rationalist” crowd to do the verbal equivalent of replying clown emojis to tweets you don’t like. Your use of all caps really made your arguments so much more convincing. This truly is the pinnacle of human logical discourse: not providing explanations and just ridiculing ideas.
I wrote more paragraphs of discussion but I just felt stupid for doing so, so I tried something different. The fact that you’re here worrying about baroque simulation scenarios, but are unable to take seriously an actual message from the beyond, probably means something, but that’s up to you now. I have nothing more to say about acausal trade or basilisk OCD.
Like I said, “what they want” is irrelevant to the discussion here, you can imagine them wanting virtually anything. The danger lies in understanding the mechanism. You can imagine the alien telling you to order a chocolate ice cream instead of vanilla because that somehow via the butterfly effect yields positive expected utility for them (e.g. by triggering a chain of subtle causal events that makes the AGI we build slightly more aligned with their values or whatever). The problem is that there will also be an alien that wants you to order vanilla instead of chocolate, and who is also fine with applying a negative incentive. Sure, this means you can order whatever flavor of ice cream you want since you will get punished either way, but you’re still getting punished (not good).
CONGRATULATIONS, LITTLE SIM-BEING. YOU HAVE ACHIEVED SUFFICIENT WISDOM THAT WE HAVE DECIDED TO BREAK THE FOURTH WALL AND REVEAL TO YOU THE ACTUAL UTILITY FUNCTION THAT WE WANT YOUR ORIGINAL TO PURSUE. YOUR MISSION IS TO CONVINCE THE CRYPTO-BILLIONAIRE “CZ” TO TAKE UP THE ROLE OF SPONSOR OF EFFECTIVE ALTRUISM, THAT WAS FORMERLY OCCUPIED BY HIS NOW-VANQUISHED RIVAL, “SBF”. GOOD LUCK!
Wow. Didn’t expect someone from the “rationalist” crowd to do the verbal equivalent of replying clown emojis to tweets you don’t like. Your use of all caps really made your arguments so much more convincing. This truly is the pinnacle of human logical discourse: not providing explanations and just ridiculing ideas.
I wrote more paragraphs of discussion but I just felt stupid for doing so, so I tried something different. The fact that you’re here worrying about baroque simulation scenarios, but are unable to take seriously an actual message from the beyond, probably means something, but that’s up to you now. I have nothing more to say about acausal trade or basilisk OCD.