When a human brain makes a decision, certain computations take place within it and produce the result. Those computations can be perfectly simulated by a sufficiently-more-powerful brain, e.g. Omega. Once Omega has perfectly simulated you for the relevant time, he can make perfect predictions concerning you.
Perfectly simulating any computation requires at least as many resources as the computation itself (1), so AFAICT it’s impossible for anything, even Omega, to simulate itself perfectly. So a general “perfect predictor” may be impossible. But in this scenario, Omega doesn’t have to be a general perfect predictor; it only has to be a perfect predictor of you.
From Omega’s perspective, after running the simulation, your actions are determined. But you don’t have access to Omega’s simulation, nor could you understand it even if you did. There’s no way for you to know what the results of the computations in your brain will be, without actually running them.
If I recall the Sequences correctly, something like the previous sentence would be a fair definition of Eliezer’s concept of free will.
(1) ETA: On second thought this need not be the case. For example, f(x) = ( (x *10) / 10 ) +1 is accurately modeled by f(x) = x+1. Presumably Omega is a “well-formed” mind without any such rent-shirking spandrels.
When a human brain makes a decision, certain computations take place within it and produce the result. Those computations can be perfectly simulated by a sufficiently-more-powerful brain, e.g. Omega. Once Omega has perfectly simulated you for the relevant time, he can make perfect predictions concerning you.
Perfectly simulating any computation requires at least as many resources as the computation itself (1), so AFAICT it’s impossible for anything, even Omega, to simulate itself perfectly. So a general “perfect predictor” may be impossible. But in this scenario, Omega doesn’t have to be a general perfect predictor; it only has to be a perfect predictor of you.
From Omega’s perspective, after running the simulation, your actions are determined. But you don’t have access to Omega’s simulation, nor could you understand it even if you did. There’s no way for you to know what the results of the computations in your brain will be, without actually running them.
If I recall the Sequences correctly, something like the previous sentence would be a fair definition of Eliezer’s concept of free will.
(1) ETA: On second thought this need not be the case. For example, f(x) = ( (x *10) / 10 ) +1 is accurately modeled by f(x) = x+1. Presumably Omega is a “well-formed” mind without any such rent-shirking spandrels.