You can’t “speed up” an uncomputable non-algorithm.
Okay, we’re going in circles. You had just mentioned possible computable algorithms that approximate Solomonoff induction.
it’s not obvious that there are no algorithms that approximate Solomonoff induction well enough in practical cases. [emphasis added—SB]
So, we were talking about approximating algorithms. The point I was making, in response to this argument that “well, we can have working algorithms that are close enough to S/I”, was that to do so, you have to bring in knowledge of the distribution gained some other way, at which point it is no longer universal. (And, in which case talk of “speeding up” is meaningful.)
Demonstrating my point that universal intelligence has its limits and must combine with intelligence in a different sense of the term.
You introduce operations on the approximate algorithms (changing the algorithm by adding data), something absent from the original problem. What doesn’t make sense is to compare “speed” of non-algorithmic specification with the speed of algorithmic approximations. And absent any approximate algorithms, it’s also futile to compare their speed, much less propose mechanisms for their improvement that assume specific structure of these absent algorithms (if you are not serious about exploring the design space in this manner to obtain actual results).
You introduce operations on the approximate algorithms (changing the algorithm by adding data), something absent from the original problem.
What you call “the original problem” (pure Solomonoff induction) isn’t. It’s not a problem. It can’t be done, so it’s a moot point.
What doesn’t make sense is to compare “speed” of non-algorithmic specification with the speed of algorithmic approximations
Sure it does. The uncomputable Solomonoff induction has a speed of zero. Non-halting approximations have a speed greater than zero. Sounds comparable to me for the purposes of this discussion.
And absent any approximate algorithms, it’s also futile to compare their speed, much less propose mechanisms for their improvement that assume specific structure of these absent algorithms (if you are not serious about exploring the design space in this manner to obtain actual results).
There are approximate algorithms. Even Bayesian inference counts. And my point is that any time you add something to modify Solomonoff induction to make it useful is, directly or indirectly, introducing a prior unique to the search space—cleary showing the distinctness of type 2 intelligence.
I don’t understand why you’d continue arguing definitions about speed of Solomonoff induction or it being “the original problem”. It’s clear what we both mean.
I believe you are wrong about general statements about what needs to be done to implement approximate Solomonoff induction. Since we don’t technically define in what sense this approximation has to be general, there remain possibilities for a good technical definition that preserves “generality” in an approximate implementation.
don’t understand why you’d continue arguing definitions about speed of Solomonoff induction or it being “the original problem”. It’s clear what we both mean.
A better question would be why you brought up the issue. We both knew what the other meant before that, but you kept bringing it up.
I believe you are wrong … there remain possibilities for a good technical definition that preserves “generality” in an approximate implementation.
Okay, well, I’ll believe it when I see it. In the mean time, I suspect it will be far more productive to exploit whatever regularity we already know about the environment, and work on building that into the inference program’s prior. (Arguably, even the Occamian prior does that by using our hard-won belief in the universe’s preference for simplicity!)
It introduces a prior, period. Not a prior about “skipping over”. Universal induction doesn’t have to “run” anything in a trivial manner.
You can’t “speed up” an uncomputable non-algorithm.
Okay, we’re going in circles. You had just mentioned possible computable algorithms that approximate Solomonoff induction.
So, we were talking about approximating algorithms. The point I was making, in response to this argument that “well, we can have working algorithms that are close enough to S/I”, was that to do so, you have to bring in knowledge of the distribution gained some other way, at which point it is no longer universal. (And, in which case talk of “speeding up” is meaningful.)
Demonstrating my point that universal intelligence has its limits and must combine with intelligence in a different sense of the term.
You introduce operations on the approximate algorithms (changing the algorithm by adding data), something absent from the original problem. What doesn’t make sense is to compare “speed” of non-algorithmic specification with the speed of algorithmic approximations. And absent any approximate algorithms, it’s also futile to compare their speed, much less propose mechanisms for their improvement that assume specific structure of these absent algorithms (if you are not serious about exploring the design space in this manner to obtain actual results).
What you call “the original problem” (pure Solomonoff induction) isn’t. It’s not a problem. It can’t be done, so it’s a moot point.
Sure it does. The uncomputable Solomonoff induction has a speed of zero. Non-halting approximations have a speed greater than zero. Sounds comparable to me for the purposes of this discussion.
There are approximate algorithms. Even Bayesian inference counts. And my point is that any time you add something to modify Solomonoff induction to make it useful is, directly or indirectly, introducing a prior unique to the search space—cleary showing the distinctness of type 2 intelligence.
To wrap up (as an alternative to not replying):
I don’t understand why you’d continue arguing definitions about speed of Solomonoff induction or it being “the original problem”. It’s clear what we both mean.
I believe you are wrong about general statements about what needs to be done to implement approximate Solomonoff induction. Since we don’t technically define in what sense this approximation has to be general, there remain possibilities for a good technical definition that preserves “generality” in an approximate implementation.
A better question would be why you brought up the issue. We both knew what the other meant before that, but you kept bringing it up.
Okay, well, I’ll believe it when I see it. In the mean time, I suspect it will be far more productive to exploit whatever regularity we already know about the environment, and work on building that into the inference program’s prior. (Arguably, even the Occamian prior does that by using our hard-won belief in the universe’s preference for simplicity!)