I think this is a nice model, thank you for writing it up!
I’m afraid that not many people will be able to wrap their head around it in this format. It seems like there is a more computer sciencey presentation which has a similar upside, which Scott Aaronson and I discussed in March 2013 and Jessica Taylor and I worked out that April after a MIRI workshop. (Sorry I didn’t mention this earlier, I vaguely recalled it when I saw this post and found it while rooting around in my emails.)
I think there exists an oracle O, which takes as input the description of a probabilistic oracle turing machine M, and has the following property. O(<M>, n) outputs a (possibly stochastic) rational number within 1/n of the interval [P(M[O] = 1), 1 - P(M[O] = 0)], where M[O] is M run with the oracle O. If M[O] outputs something in {0, 1} with probability 1, then this gives us an approximation to the distribution of its outputs. Not halting or outputting something outside of {0, 1} can be interpreted arbitrarily. If we want to make things total, we can also use the lim inf of O(<M>, n) to define a machine’s output distribution, and I think this lets us reproduce the desired results on Nash equilibria.
This can then be used to define rational agents in a way that might be more comprehensible to computer scientists. Do you think there are big advantages of the more mathematical approach?
So you’re saying that there is an oracle that can predict the outputs of any oracle you give it through a probability distribution of the possible outputs?
I’d love to see what this approach looked like worked out in more detail. I think there would be a lot of benefit to making this kind of result accessible to computer scientists without strong math backgrounds.
I think this is a nice model, thank you for writing it up!
I’m afraid that not many people will be able to wrap their head around it in this format. It seems like there is a more computer sciencey presentation which has a similar upside, which Scott Aaronson and I discussed in March 2013 and Jessica Taylor and I worked out that April after a MIRI workshop. (Sorry I didn’t mention this earlier, I vaguely recalled it when I saw this post and found it while rooting around in my emails.)
I think there exists an oracle O, which takes as input the description of a probabilistic oracle turing machine M, and has the following property. O(<M>, n) outputs a (possibly stochastic) rational number within 1/n of the interval [P(M[O] = 1), 1 - P(M[O] = 0)], where M[O] is M run with the oracle O. If M[O] outputs something in {0, 1} with probability 1, then this gives us an approximation to the distribution of its outputs. Not halting or outputting something outside of {0, 1} can be interpreted arbitrarily. If we want to make things total, we can also use the lim inf of O(<M>, n) to define a machine’s output distribution, and I think this lets us reproduce the desired results on Nash equilibria.
This can then be used to define rational agents in a way that might be more comprehensible to computer scientists. Do you think there are big advantages of the more mathematical approach?
So you’re saying that there is an oracle that can predict the outputs of any oracle you give it through a probability distribution of the possible outputs?
Thanks! Replied in a post.
I’d love to see what this approach looked like worked out in more detail. I think there would be a lot of benefit to making this kind of result accessible to computer scientists without strong math backgrounds.