I don’t think that’s the same as “thinking outside the box you’re given”. That’s about power of extrapolation, which is a separate entangled dimension.
Anyway, suppose I’m thinking of a criterion. Of the integers 1-20, the ones which meet my criterion are 2, 3, 5, 7, 11, 13, 17, 19. I challenge you to write a program that determines whether a number meets my criterion or not. A “black-box” program might check to see if the number is on the list I gave. A “gears-level” program might check to see if the number is divisible by any integer besides itself and 1. The “gears-level” program is “within the box” in the sense that it is a program which returns True or False depending on whether my criterion is supposedly met—the same box the “black-box” program is in. And in principle it doesn’t have to be constructed using prior knowledge. Maybe you could find it by brute forcing all short programs and returning the shortest one which matches available data with minimal hardcoded integers, or some other method for searching program space.
Similarly, a being from another dimension could be transported to our dimension, observe some physical objects, try to make predictions about them, and deduce that F=ma. They aren’t using prior knowledge since their dimension works differently than ours. And they aren’t “thinking outside the box they’re given”, they’re trying to make accurate predictions, just as one could do with a black box model.
Ok, so for the primes examples, I’d say that the gears-level model is using prior information in the form of the universal prior. I’d think of the universal prior as a black-box method for learning gears-level models; it’s a magical thing which lets us cross the bridge from one to the other (sometimes). In general, “black-box methods for finding gears-level models” is one way I’d characterize the core problems of AGI.
One “box” in the primes example is just the integers from 0-20; the gears-level model gives us insight into what happens outside that range, while the black-box model does not.
Similarly for the being from another dimension: they’re presumably using a universal prior. And they may not bother thinking outside the box—they may only want to make accurate predictions about whatever questions are in front of them—but F = ma is a model which definitely can be used for all sorts of things in our universe, not just whatever specific physical outcomes the being wants to predict.
But I still don’t think I’ve properly explained what I mean by “outside the box”.
For the primes problem, a better example of “outside the box” would be suddenly introducing some other kind of “number”, like integer matrices or quadratic integers or something. A divisibility-based model would generalize (assuming you kept using a divisibility-based criterion) - not in the sense that the same program would work, but in the sense that we don’t need to restart from scratch when figuring out the pattern. The black-box model, on the other hand, would need to start more-or-less from scratch.
For the being from another dimension, a good example of “outside the box” would be a sudden change in fundamental constants—not so drastic as to break all the approximations, but enough that e.g. energies of chemical reactions all change. In that case, F = ma would probably still hold despite the distribution shift.
So I guess the best summary of what I mean by “outside the box” is something like “counterfactual changes which don’t correspond to anything in the design space/data”.
I don’t think that’s the same as “thinking outside the box you’re given”. That’s about power of extrapolation, which is a separate entangled dimension.
Anyway, suppose I’m thinking of a criterion. Of the integers 1-20, the ones which meet my criterion are 2, 3, 5, 7, 11, 13, 17, 19. I challenge you to write a program that determines whether a number meets my criterion or not. A “black-box” program might check to see if the number is on the list I gave. A “gears-level” program might check to see if the number is divisible by any integer besides itself and 1. The “gears-level” program is “within the box” in the sense that it is a program which returns True or False depending on whether my criterion is supposedly met—the same box the “black-box” program is in. And in principle it doesn’t have to be constructed using prior knowledge. Maybe you could find it by brute forcing all short programs and returning the shortest one which matches available data with minimal hardcoded integers, or some other method for searching program space.
Similarly, a being from another dimension could be transported to our dimension, observe some physical objects, try to make predictions about them, and deduce that F=ma. They aren’t using prior knowledge since their dimension works differently than ours. And they aren’t “thinking outside the box they’re given”, they’re trying to make accurate predictions, just as one could do with a black box model.
Ok, so for the primes examples, I’d say that the gears-level model is using prior information in the form of the universal prior. I’d think of the universal prior as a black-box method for learning gears-level models; it’s a magical thing which lets us cross the bridge from one to the other (sometimes). In general, “black-box methods for finding gears-level models” is one way I’d characterize the core problems of AGI.
One “box” in the primes example is just the integers from 0-20; the gears-level model gives us insight into what happens outside that range, while the black-box model does not.
Similarly for the being from another dimension: they’re presumably using a universal prior. And they may not bother thinking outside the box—they may only want to make accurate predictions about whatever questions are in front of them—but F = ma is a model which definitely can be used for all sorts of things in our universe, not just whatever specific physical outcomes the being wants to predict.
But I still don’t think I’ve properly explained what I mean by “outside the box”.
For the primes problem, a better example of “outside the box” would be suddenly introducing some other kind of “number”, like integer matrices or quadratic integers or something. A divisibility-based model would generalize (assuming you kept using a divisibility-based criterion) - not in the sense that the same program would work, but in the sense that we don’t need to restart from scratch when figuring out the pattern. The black-box model, on the other hand, would need to start more-or-less from scratch.
For the being from another dimension, a good example of “outside the box” would be a sudden change in fundamental constants—not so drastic as to break all the approximations, but enough that e.g. energies of chemical reactions all change. In that case, F = ma would probably still hold despite the distribution shift.
So I guess the best summary of what I mean by “outside the box” is something like “counterfactual changes which don’t correspond to anything in the design space/data”.