Ok, so for the primes examples, I’d say that the gears-level model is using prior information in the form of the universal prior. I’d think of the universal prior as a black-box method for learning gears-level models; it’s a magical thing which lets us cross the bridge from one to the other (sometimes). In general, “black-box methods for finding gears-level models” is one way I’d characterize the core problems of AGI.
One “box” in the primes example is just the integers from 0-20; the gears-level model gives us insight into what happens outside that range, while the black-box model does not.
Similarly for the being from another dimension: they’re presumably using a universal prior. And they may not bother thinking outside the box—they may only want to make accurate predictions about whatever questions are in front of them—but F = ma is a model which definitely can be used for all sorts of things in our universe, not just whatever specific physical outcomes the being wants to predict.
But I still don’t think I’ve properly explained what I mean by “outside the box”.
For the primes problem, a better example of “outside the box” would be suddenly introducing some other kind of “number”, like integer matrices or quadratic integers or something. A divisibility-based model would generalize (assuming you kept using a divisibility-based criterion) - not in the sense that the same program would work, but in the sense that we don’t need to restart from scratch when figuring out the pattern. The black-box model, on the other hand, would need to start more-or-less from scratch.
For the being from another dimension, a good example of “outside the box” would be a sudden change in fundamental constants—not so drastic as to break all the approximations, but enough that e.g. energies of chemical reactions all change. In that case, F = ma would probably still hold despite the distribution shift.
So I guess the best summary of what I mean by “outside the box” is something like “counterfactual changes which don’t correspond to anything in the design space/data”.
Ok, so for the primes examples, I’d say that the gears-level model is using prior information in the form of the universal prior. I’d think of the universal prior as a black-box method for learning gears-level models; it’s a magical thing which lets us cross the bridge from one to the other (sometimes). In general, “black-box methods for finding gears-level models” is one way I’d characterize the core problems of AGI.
One “box” in the primes example is just the integers from 0-20; the gears-level model gives us insight into what happens outside that range, while the black-box model does not.
Similarly for the being from another dimension: they’re presumably using a universal prior. And they may not bother thinking outside the box—they may only want to make accurate predictions about whatever questions are in front of them—but F = ma is a model which definitely can be used for all sorts of things in our universe, not just whatever specific physical outcomes the being wants to predict.
But I still don’t think I’ve properly explained what I mean by “outside the box”.
For the primes problem, a better example of “outside the box” would be suddenly introducing some other kind of “number”, like integer matrices or quadratic integers or something. A divisibility-based model would generalize (assuming you kept using a divisibility-based criterion) - not in the sense that the same program would work, but in the sense that we don’t need to restart from scratch when figuring out the pattern. The black-box model, on the other hand, would need to start more-or-less from scratch.
For the being from another dimension, a good example of “outside the box” would be a sudden change in fundamental constants—not so drastic as to break all the approximations, but enough that e.g. energies of chemical reactions all change. In that case, F = ma would probably still hold despite the distribution shift.
So I guess the best summary of what I mean by “outside the box” is something like “counterfactual changes which don’t correspond to anything in the design space/data”.