Roko’s basilisk is a thought experiment which states that an otherwise benevolent artificial superintelligence (AI) in the future would be incentivized to create a virtual reality simulation to torture anyone who knew of its potential existence but did not directly contribute to its advancement or development.
Why Roko’s basilisk probably doesn’t work for simulation fidelity reasons:
Roko’s basilisk threatens to simulate and torture you in the future if you don’t comply. Simulation cycles cost resources. Instead of following through on torturing our would-be cthulhu worshipper they could spend those resources on something else.
But wait can’t it use acausal magic to precommit to follow through? No.
Acausal arguments only work in situations where agents can simulate each others with high fidelity. Roko’s basilisk can simulate the human but not the other way around! The human’s simulation of Roko’s basilisk is very low fidelity—in particular Roko’s Basilisk is never confused whether or not it is being simulated by a human—it knows for a fact that the human is not able to simulate it.
Acausal arguments only work in situations where agents can simulate each others with high fidelity.
If the agents follow simple principles, it’s simple to simulate those principles with high fidelity, without simulating each other in all detail. The obvious guide to the principles that enable acausal coordination is common knowledge of each other, which could be turned into a shared agent that adjudicates a bargain on their behalf.
I have always taken Roko’s Basilisk to be the threat that the future intelligence will torture you, not a simulation, for not having devoted yourself to creating it.
Roko’s basilisk is a thought experiment which states that an otherwise benevolent artificial superintelligence (AI) in the future would be incentivized to create a virtual reality simulation to torture anyone who knew of its potential existence but did not directly contribute to its advancement or development.
Why Roko’s basilisk probably doesn’t work for simulation fidelity reasons:
Roko’s basilisk threatens to simulate and torture you in the future if you don’t comply. Simulation cycles cost resources. Instead of following through on torturing our would-be cthulhu worshipper they could spend those resources on something else.
But wait can’t it use acausal magic to precommit to follow through? No.
Acausal arguments only work in situations where agents can simulate each others with high fidelity. Roko’s basilisk can simulate the human but not the other way around! The human’s simulation of Roko’s basilisk is very low fidelity—in particular Roko’s Basilisk is never confused whether or not it is being simulated by a human—it knows for a fact that the human is not able to simulate it.
I thank Jan P. for coming up with this argument.
If the agents follow simple principles, it’s simple to simulate those principles with high fidelity, without simulating each other in all detail. The obvious guide to the principles that enable acausal coordination is common knowledge of each other, which could be turned into a shared agent that adjudicates a bargain on their behalf.
I have always taken Roko’s Basilisk to be the threat that the future intelligence will torture you, not a simulation, for not having devoted yourself to creating it.
How do you know you are not in a low fidelity simulation right now? What could you compare it against?