First, could you review the previous comment to see if you agree with the logic, and if not, what do you disagree in particular.
It feels like we are going around in circles at this point. I’m not sure where the disconnect is.
-if the total possible observation data is infinite, what is prior probability that it is exactly reproduced by finite hypothesis? I argue that it is infinitesimal
The set of all natural numbers is infinite, yet can be enumerated by a finite computer program (when run on an infinite computer, AKA, a Turing machine). There are many many other examples of infinite patterns enumerable by finite programs. And some of them, like “compute the digits of pi” seem pretty chaotic, yet their Kolmogorov complexity is small.
One wrinkle, which you might be alluding to, is that no program with infinite output ever halts. This is true, but there are halting programs that can compute any finite prefix of pi. And like I said before, at no point is your observation infinite. It’s always finite so far. The infinity is never completed.
So the hypothesis “these are the digits of pi” is considered by Solomonoff induction, but maybe it looks like a weighted sum of a class of programs that say “compute pi up to the nth digit” for some n. These still compress quite well, (especially for compressible n’s) so their Kolmogorov complexity is small. I don’t think this is an obstacle for Solomonoff induction.
Has Solomonoff induction got it wrong? Close but not quite? I would argue no. I don’t believe uncomputable sets can physically exist. There are no perfect circles. The abstraction called pi is the approximation, for whatever algorithm physics is actually running, which Solomonoff induction would eventually find.
It feels like we are going around in circles at this point. I’m not sure where the disconnect is.
The set of all natural numbers is infinite, yet can be enumerated by a finite computer program (when run on an infinite computer, AKA, a Turing machine). There are many many other examples of infinite patterns enumerable by finite programs. And some of them, like “compute the digits of pi” seem pretty chaotic, yet their Kolmogorov complexity is small.
One wrinkle, which you might be alluding to, is that no program with infinite output ever halts. This is true, but there are halting programs that can compute any finite prefix of pi. And like I said before, at no point is your observation infinite. It’s always finite so far. The infinity is never completed.
So the hypothesis “these are the digits of pi” is considered by Solomonoff induction, but maybe it looks like a weighted sum of a class of programs that say “compute pi up to the nth digit” for some n. These still compress quite well, (especially for compressible n’s) so their Kolmogorov complexity is small. I don’t think this is an obstacle for Solomonoff induction.
Has Solomonoff induction got it wrong? Close but not quite? I would argue no. I don’t believe uncomputable sets can physically exist. There are no perfect circles. The abstraction called pi is the approximation, for whatever algorithm physics is actually running, which Solomonoff induction would eventually find.