Suppose I disallow Mu from examining the source code of either p or o. It can examine the behaviors in advance of their execution, but not the source code. And now suppose that if Mu is allowed to know this much, then I am allowed to know the same about Mu. Then what?
If we are going to allow you a chance to figure out the behavior of Mu, Mu should be given the chance to find out the behavior of Eliezer(what programs you are likely to produce etc). Only then would information parity be preserved.
Mu is standing in for the entire world, your system is a small bit of it, it is entirely reasonable to expect the world to know more about your system than you do about the behavior of the would. I’m not sure where you are taking this idea, but it is unrealistic in my view.
I design a system p that has high optimising power, but which o is not smart enough to prove does so. Actually, I might get away with obfuscating p enough to confuse not o, but Mu’s simulation of o, but my simulation of Mu simulating o might be imperfect.
Mu wrongly concludes that p is a poor optimiser, and performs the optimisation it thinks p incapable of. Meanwhile, I have bet against the verdict of o and win.
Suppose I disallow Mu from examining the source code of either p or o. It can examine the behaviors in advance of their execution, but not the source code. And now suppose that if Mu is allowed to know this much, then I am allowed to know the same about Mu. Then what?
Optimality through obscurity?
If we are going to allow you a chance to figure out the behavior of Mu, Mu should be given the chance to find out the behavior of Eliezer(what programs you are likely to produce etc). Only then would information parity be preserved.
Mu is standing in for the entire world, your system is a small bit of it, it is entirely reasonable to expect the world to know more about your system than you do about the behavior of the would. I’m not sure where you are taking this idea, but it is unrealistic in my view.
I design a system p that has high optimising power, but which o is not smart enough to prove does so. Actually, I might get away with obfuscating p enough to confuse not o, but Mu’s simulation of o, but my simulation of Mu simulating o might be imperfect.
Mu wrongly concludes that p is a poor optimiser, and performs the optimisation it thinks p incapable of. Meanwhile, I have bet against the verdict of o and win.
But I agree with Psychohistorian’s comment and am not clear what the point of all this is.