My recursive suggestion won’t work. One can devise a UTM that gives the shortest code to itself, by the usual reflexivity constructions. The computability theory textbook method looks better. But what theoretical justification can be given for it? Why are we confident that bad explanations are not lurking within it?
Actually, perhaps we shouldn’t be. It has already been remarked by Eliezer that Solomonoff induction gives what looks like undue weight to hypotheses involving gigantic numbers with short descriptions, e.g. 3^^^3, despite the fact that, looking at the world, such numbers have never been useful for anything but talking about gigantic numbers, and proving what are generally expected to be very generous upper bounds for some combinatorial theorems.
My recursive suggestion won’t work. One can devise a UTM that gives the shortest code to itself, by the usual reflexivity constructions. The computability theory textbook method looks better. But what theoretical justification can be given for it? Why are we confident that bad explanations are not lurking within it?
Actually, perhaps we shouldn’t be. It has already been remarked by Eliezer that Solomonoff induction gives what looks like undue weight to hypotheses involving gigantic numbers with short descriptions, e.g. 3^^^3, despite the fact that, looking at the world, such numbers have never been useful for anything but talking about gigantic numbers, and proving what are generally expected to be very generous upper bounds for some combinatorial theorems.