there’s no reason to think we couldn’t have rational agents implemented in much more predictable form, as python routines for example, so that any being with superior computation power could simply inspect the source and determine what the output would be.
Such a being would be different from a human in fundamental ways. Imagine knowing with certainty that your actions can be predicted perfectly by the guy next door, even taking into account that you are trying to be hard to predict?
A (quasi)rational agent with access to genuine randomness (such as a human) is a different matter. A superintelligence could almost perfectly predict the probability distribution over my actions, but by quantum entanglement it would not be able to predict my actual actions.
A (quasi)rational agent with access to genuine randomness (such as a human)
Whaddaya mean humans are rational agents with access to genuine randomness? That’s what we’re arguing about in the first place!
A superintelligence could almost perfectly predict the probability distribution over my actions, but by quantum entanglement it would not be able to predict my actual actions.
Perhaps Omega is entangled with your brain such that in all the worlds in which you would choose to one-box, he would predict that you one-box, and all the worlds in which you would choose to two-box, he would predict that you two-box?
Imagine knowing with certainty that your actions can be predicted perfectly by the guy next door, even taking into account that you are trying to be hard to predict?
You wouldn’t know this with certainty* because it wouldn’t be true.
(*unless you were delusional)
The guy next door is on roughly your mental level. Thus, the guy next door can’t predict your actions perfectly, because he can’t run a perfect simulation of your mind that’s faster than you. He doesn’t have the capacity.
And he certainly doesn’t have the capacity to simulate the environment, including other people, while doing so.
A (quasi)rational agent with access to genuine randomness (such as a human) is a different matter.
Humans may or may not generally have access to genuine randomness.
It’s as yet unknown whether we even have run on quantum randomness; and its also unprovable that quantum randomness is actually genuine randomness, and not just based on effects we don’t yet understand, as so many other types of randomness have been.
You wouldn’t know this with certainty* because it wouldn’t be true.
You’re not taking this in the least convenient possible world. Surely it’s not impossible in principle that your neighbor can simulate you and your environment. Perhaps your neighbor is superintelligent?
It’s ALSO not impossible in principle in the real world. A superintelligent entity could, in principle, perfectly predict my actions.
Remember, in the Least Convenient Possible World quantum “randomness” isn’t random.
As such, this ISN’T a fundamental difference between humans and “such beings”.
Which was all I set out to demonstrate.
I was using the “most plausible world” on the basis that it seemed pretty clear that that was the one Roko intended. (Where your neighbour isn’t in fact Yahweh in disguise).
EDIT: Probably should specify worlds for things in this kind of environment. Thanks, the critical environment here is helping me think about how I think/argue.
It’s as yet unknown whether we even have run on quantum randomness; and its also unprovable that quantum randomness is actually genuine randomness, and not just based on effects we don’t yet understand, as so many other types of randomness have been.
If you believe the Many Worlds Interpretation, then quantum randomness just creates copies in a deterministic way.
Such a being would be different from a human in fundamental ways. Imagine knowing with certainty that your actions can be predicted perfectly by the guy next door, even taking into account that you are trying to be hard to predict?
A (quasi)rational agent with access to genuine randomness (such as a human) is a different matter. A superintelligence could almost perfectly predict the probability distribution over my actions, but by quantum entanglement it would not be able to predict my actual actions.
Whaddaya mean humans are rational agents with access to genuine randomness? That’s what we’re arguing about in the first place!
Perhaps Omega is entangled with your brain such that in all the worlds in which you would choose to one-box, he would predict that you one-box, and all the worlds in which you would choose to two-box, he would predict that you two-box?
In the original formulation, if Omega expects you to flip a coin, he leaves box B empty.
You wouldn’t know this with certainty* because it wouldn’t be true.
(*unless you were delusional)
The guy next door is on roughly your mental level. Thus, the guy next door can’t predict your actions perfectly, because he can’t run a perfect simulation of your mind that’s faster than you. He doesn’t have the capacity.
And he certainly doesn’t have the capacity to simulate the environment, including other people, while doing so.
Humans may or may not generally have access to genuine randomness.
It’s as yet unknown whether we even have run on quantum randomness; and its also unprovable that quantum randomness is actually genuine randomness, and not just based on effects we don’t yet understand, as so many other types of randomness have been.
You’re not taking this in the least convenient possible world. Surely it’s not impossible in principle that your neighbor can simulate you and your environment. Perhaps your neighbor is superintelligent?
It’s ALSO not impossible in principle in the real world. A superintelligent entity could, in principle, perfectly predict my actions. Remember, in the Least Convenient Possible World quantum “randomness” isn’t random.
As such, this ISN’T a fundamental difference between humans and “such beings”. Which was all I set out to demonstrate.
I was using the “most plausible world” on the basis that it seemed pretty clear that that was the one Roko intended. (Where your neighbour isn’t in fact Yahweh in disguise). EDIT: Probably should specify worlds for things in this kind of environment. Thanks, the critical environment here is helping me think about how I think/argue.
If you believe the Many Worlds Interpretation, then quantum randomness just creates copies in a deterministic way.