Check out Moshe’s expounding of Steve’s objection so Schmidhuber’s main point, which I think makes the same argument that you do. (One could easily counter that such a wireheading AI would never get off the ground, but I think that debate can be cordoned off.)
ETA: Maybe a counterargument could be made involving omega or super-omega promising more compression than any artificial pseudo-random generator… but AFAIK Schmidhuber hasn’t gone that route.
moshez’s first argument sounds like it’s the same thing as my point about it not being optimal for a utility-maximizer, in considerably different terms.
His second hyperbolic argument seems to me to be wrong or irrelevant: I would argue that people are in practice extremely capable of engaging in hyperbolic discounting with regard to the best and most absorbing artworks while over-consuming ‘junk food’ art (and this actually forms part of my essay arguing that new art should not be subsidized).
Maybe a counterargument could be made involving omega or super-omega promising more compression than any artificial pseudo-random generator...
I don’t really follow. Is this Omega as in the predictor, or Omega as in Chaitin’s Omega? The latter doesn’t allow any compressor any progress beyond the first few bits due to resource constraints, and if bits of Chaitin’s Omega are doled out, they will have to be at least as cheap to crack as brute-force running the equivalent Turing machine or else the agent will prefer the brute-forcing and ignore the Omega-bait. So the agent will do no worse than before and possibly better (eg. if the bits are offered as-is with no tricky traps or proof of work-style schemes).
His second hyperbolic argument seems to me to be wrong or irrelevant
Agreed. (I like your essay about junk food art. By the way, did you ever actually do the utilitarian calculations re Nazi Germany’s health policies? Might you share the results?)
I don’t really follow.
Me neither, I just intuit that there might be interesting non-obvious arguments in roughly that argumentspace.
Omega as in the predictor, or Omega as in Chaitin’s Omega?
I like to think of the former as the physical manifestation of the latter, and I like to think of both of them as representations of God. But anyway, the latter.
beyond the first few bits due to resource constraints
You mean because it’s hard to find/verify bits of omega? But Schmidhuber argues that certain generalized computers can enumerate bits of omega very easily, which is why he developed the idea of a super-omega. I’m not sure what that would imply or if it’s relevant… maybe I should look at this again after the next time I re-familiarize myself with the generalized Turing machine literature.
By the way, did you ever actually do the utilitarian calculations re Nazi Germany’s health policies? Might you share the results?
I was going off a library copy, and thought of it only afterwards; I keep hoping someone else will do it for me.
But Schmidhuber argues that certain generalized computers can enumerate bits of omega very easily, which is why he developed the idea of a super-omega.
His jargon is a little much for me. I agree one can approximate Omega by enumerating digits, but what is ‘very easily’ here?
Check out Moshe’s expounding of Steve’s objection so Schmidhuber’s main point, which I think makes the same argument that you do. (One could easily counter that such a wireheading AI would never get off the ground, but I think that debate can be cordoned off.)
ETA: Maybe a counterargument could be made involving omega or super-omega promising more compression than any artificial pseudo-random generator… but AFAIK Schmidhuber hasn’t gone that route.
moshez’s first argument sounds like it’s the same thing as my point about it not being optimal for a utility-maximizer, in considerably different terms.
His second hyperbolic argument seems to me to be wrong or irrelevant: I would argue that people are in practice extremely capable of engaging in hyperbolic discounting with regard to the best and most absorbing artworks while over-consuming ‘junk food’ art (and this actually forms part of my essay arguing that new art should not be subsidized).
I don’t really follow. Is this Omega as in the predictor, or Omega as in Chaitin’s Omega? The latter doesn’t allow any compressor any progress beyond the first few bits due to resource constraints, and if bits of Chaitin’s Omega are doled out, they will have to be at least as cheap to crack as brute-force running the equivalent Turing machine or else the agent will prefer the brute-forcing and ignore the Omega-bait. So the agent will do no worse than before and possibly better (eg. if the bits are offered as-is with no tricky traps or proof of work-style schemes).
Agreed. (I like your essay about junk food art. By the way, did you ever actually do the utilitarian calculations re Nazi Germany’s health policies? Might you share the results?)
Me neither, I just intuit that there might be interesting non-obvious arguments in roughly that argumentspace.
I like to think of the former as the physical manifestation of the latter, and I like to think of both of them as representations of God. But anyway, the latter.
You mean because it’s hard to find/verify bits of omega? But Schmidhuber argues that certain generalized computers can enumerate bits of omega very easily, which is why he developed the idea of a super-omega. I’m not sure what that would imply or if it’s relevant… maybe I should look at this again after the next time I re-familiarize myself with the generalized Turing machine literature.
I was going off a library copy, and thought of it only afterwards; I keep hoping someone else will do it for me.
His jargon is a little much for me. I agree one can approximate Omega by enumerating digits, but what is ‘very easily’ here?