While obviously there are caveats, they are limited. AIXI rewires its inputs if (a) it’s possible, and (b) it increases r(x). It’s not super-complicated.
Maybe I’m missing something about the translation from implementation to the language used in the paper. But nobody is saying “you’re missing something.” It’s more like you’re saying “surely it must be complicated!” Well, no.
AIXI is a noncomputable thing that always picks the option that maximizes the total expected reward r(x(k)). So everything I’ve been saying has been about functions, not about turing machines. If rewiring your inputs is possible, and it increases r(x), then AIXI will prefer to do it. Not hard.
Yep. Seems to apply to the limited time versions as well. At least they don’t specify any difference between “doing innovative stuff that you want them to do for sake of the AI risk argument” and “sitting in a corner masturbating quietly”, and the latter looks like way simpler solution to the problem they are really given (in math) [but not of our human-language loose and fuzzy description of that problem]
What I think is the case, is that this whole will to really live and really do stuff is very hard to implement, and implementing it doesn’t really add anything to the engineering powers of the AI so even when it’s implemented, it’ll not result in something that’s out engineering everyone. I’d become concerned if we had engineering tools that are very powerful but are wireheading (or masturbating) left and right to the point that we can’t get much use out of them. Then i’d be properly freaked out that if someone fixes this problem somehow, something undesired might happen and it would be impossible to deal with it.
While obviously there are caveats, they are limited. AIXI rewires its inputs if (a) it’s possible, and (b) it increases r(x). It’s not super-complicated.
Maybe I’m missing something about the translation from implementation to the language used in the paper. But nobody is saying “you’re missing something.” It’s more like you’re saying “surely it must be complicated!” Well, no.
Can you actually formalize what that means in terms of Turing machines? It isn’t obvious to me how to do so.
AIXI is a noncomputable thing that always picks the option that maximizes the total expected reward r(x(k)). So everything I’ve been saying has been about functions, not about turing machines. If rewiring your inputs is possible, and it increases r(x), then AIXI will prefer to do it. Not hard.
Yep. Seems to apply to the limited time versions as well. At least they don’t specify any difference between “doing innovative stuff that you want them to do for sake of the AI risk argument” and “sitting in a corner masturbating quietly”, and the latter looks like way simpler solution to the problem they are really given (in math) [but not of our human-language loose and fuzzy description of that problem]
What I think is the case, is that this whole will to really live and really do stuff is very hard to implement, and implementing it doesn’t really add anything to the engineering powers of the AI so even when it’s implemented, it’ll not result in something that’s out engineering everyone. I’d become concerned if we had engineering tools that are very powerful but are wireheading (or masturbating) left and right to the point that we can’t get much use out of them. Then i’d be properly freaked out that if someone fixes this problem somehow, something undesired might happen and it would be impossible to deal with it.