if Schmidhuber’s usefulness means “This modification has expected utility greater than or equal to all other modifications including not modifying”
Schmidhuber’s Goedel machine can handle decision rules other than the expected value, but apart from that, this is correct.
That is a huge computation, but I don’t see how it’s logically impossible.
It is required to compute its own expected actions. The naive way of doing this collapses into infinite recursion. We can imagine more sophisticated ways that usually work, but usually is not a proof; the Goedel machine only takes expectations over possible states of the environment, not over possible truth-values of mathematical statements. It is unclear how one could preform estimates over the latter; if you have an algorithm that can estimate the probability of, say, Goldbach’s conjecture, you have made quite a significant amount of progress in AGI.
Where does consist(A2) ever enter into a calculation? What do we gain by it?
Without either a proof of consist(A2) or some way of estimating the probability of consist(A2), we will not be able to calculate expected utilities.
Schmidhuber’s Goedel machine can handle decision rules other than the expected value, but apart from that, this is correct.
It is required to compute its own expected actions. The naive way of doing this collapses into infinite recursion. We can imagine more sophisticated ways that usually work, but usually is not a proof; the Goedel machine only takes expectations over possible states of the environment, not over possible truth-values of mathematical statements. It is unclear how one could preform estimates over the latter; if you have an algorithm that can estimate the probability of, say, Goldbach’s conjecture, you have made quite a significant amount of progress in AGI.
Without either a proof of consist(A2) or some way of estimating the probability of consist(A2), we will not be able to calculate expected utilities.