Gears-level insights can highlight ideas we wouldn’t even have thought to try, whereas black-box just tests the things we think to test… it can’t find unknown unknowns
It seems to me that black-box methods can also highlight things we wouldn’t have thought to try, e.g. genetic algorithms can be pretty creative.
We have some black box with a bunch of dials on the outside. A black-box optimizer is one which fiddles the dials to make the box perform well, without any knowledge of the internals of the box. If the dials themselves are sufficiently expressive, it may find creative and interesting solutions—as in the genetic algorithm example you mention. But it cannot find dials which we had not noticed before. A black-box method can explore previously-unexplored areas of the design space, but it cannot expand our concept of what the “design space” is beyond the space we’ve told it to search.
A black-box algorithm doesn’t think outside the box it’s given.
I can imagine modeling strategies which feel relatively “gears-level” yet don’t make use of prior knowledge or “think outside the box they’re given”. I think there are a few entangled dimensions here which could be disentangled in principle.
I think that there’s a sliding scale between a black-box and a gears-level model; any gears-level model has black box components, and a mostly black-box model may include gears.
E.g. if you experimentally arrive at a physics equation that correctly describes how the wheel-with-weights behaves under a wide variety of parameters, this is more gearsy than just knowing the right settings for one set of parameters. But the deeper laws of physics which generated that equation are still a black box. While you might know how to adjust the weights if the slope changes, you won’t know how you should adjust them if fundamental physical constants were to change.
(Setting aside the point that fundamental physics constants changing would break your body so you couldn’t adjust the weights because you would be dead anyway.)
To put it in different terms, in a black box model you take some things as axiomatic. Any kind of reasoning requires you to eventually fall back on axioms that are not justified further, so all models are at least somewhat black boxy. The difference is in whether you settle on axioms that are useful for a narrow set of circumstances, or on ones which allow for broader generalization.
I think you’re pointing to a true and useful thing, but “sliding scale” isn’t quite the right way to characterize it. Rather, I’d say that we’re always operating at some level(s) of abstraction, and there’s always a lowest abstraction level in our model—a ground-level abstraction, in which the pieces are atomic. For a black-box method, the ground-level abstraction just has the one monolithic black box in it.
A gearsy method has more than just one object in its ground-level abstraction. There’s some freedom in how deep the abstraction goes—we could say a gear is atomic, or we could go all the way down to atoms—and the objects at the bottom will always be treated as black boxes. But I’d say it’s not quite right to think of the model as “partially black-box” just because the bottom-level objects are atomic; it’s usually the top-level breakdown that matters. E.g., in the maze example from the post, the top and bottom halves of the maze are still atomic black boxes, but our gearsy insight is still 100% gearsy—it is an insight which will not ever apply to some random black box in the wild.
Gears/no gears is a binary distinction; there’s a big qualitative jump between a black-box method which uses no information about internal system structure, and a gearsy model which uses any information about internal structure (even just very simple information). We can add more gears, reduce the black-box components in a gears level model. But as soon as we make the very first jump from one monolithic black box to two atomic gears, we’ve gone from a black-box method which applies to any random system, to a gears-level investment which will pay out on our particular system and systems related to it.
I don’t think that’s the same as “thinking outside the box you’re given”. That’s about power of extrapolation, which is a separate entangled dimension.
Anyway, suppose I’m thinking of a criterion. Of the integers 1-20, the ones which meet my criterion are 2, 3, 5, 7, 11, 13, 17, 19. I challenge you to write a program that determines whether a number meets my criterion or not. A “black-box” program might check to see if the number is on the list I gave. A “gears-level” program might check to see if the number is divisible by any integer besides itself and 1. The “gears-level” program is “within the box” in the sense that it is a program which returns True or False depending on whether my criterion is supposedly met—the same box the “black-box” program is in. And in principle it doesn’t have to be constructed using prior knowledge. Maybe you could find it by brute forcing all short programs and returning the shortest one which matches available data with minimal hardcoded integers, or some other method for searching program space.
Similarly, a being from another dimension could be transported to our dimension, observe some physical objects, try to make predictions about them, and deduce that F=ma. They aren’t using prior knowledge since their dimension works differently than ours. And they aren’t “thinking outside the box they’re given”, they’re trying to make accurate predictions, just as one could do with a black box model.
Ok, so for the primes examples, I’d say that the gears-level model is using prior information in the form of the universal prior. I’d think of the universal prior as a black-box method for learning gears-level models; it’s a magical thing which lets us cross the bridge from one to the other (sometimes). In general, “black-box methods for finding gears-level models” is one way I’d characterize the core problems of AGI.
One “box” in the primes example is just the integers from 0-20; the gears-level model gives us insight into what happens outside that range, while the black-box model does not.
Similarly for the being from another dimension: they’re presumably using a universal prior. And they may not bother thinking outside the box—they may only want to make accurate predictions about whatever questions are in front of them—but F = ma is a model which definitely can be used for all sorts of things in our universe, not just whatever specific physical outcomes the being wants to predict.
But I still don’t think I’ve properly explained what I mean by “outside the box”.
For the primes problem, a better example of “outside the box” would be suddenly introducing some other kind of “number”, like integer matrices or quadratic integers or something. A divisibility-based model would generalize (assuming you kept using a divisibility-based criterion) - not in the sense that the same program would work, but in the sense that we don’t need to restart from scratch when figuring out the pattern. The black-box model, on the other hand, would need to start more-or-less from scratch.
For the being from another dimension, a good example of “outside the box” would be a sudden change in fundamental constants—not so drastic as to break all the approximations, but enough that e.g. energies of chemical reactions all change. In that case, F = ma would probably still hold despite the distribution shift.
So I guess the best summary of what I mean by “outside the box” is something like “counterfactual changes which don’t correspond to anything in the design space/data”.
One key dimension is decomposition – I would say any gears model provides decomposition, but models can have it without gears.
For example, the error in any machine learning model can be broken down into bias + variance, which provides a useful model for debugging. But these don’t feel like gears in any meaningful sense, whereas, say, bootstrapping + weak learners feel like gears in understanding Random Forests.
It seems to me that black-box methods can also highlight things we wouldn’t have thought to try, e.g. genetic algorithms can be pretty creative.
I’ll clarify what I mean a bit.
We have some black box with a bunch of dials on the outside. A black-box optimizer is one which fiddles the dials to make the box perform well, without any knowledge of the internals of the box. If the dials themselves are sufficiently expressive, it may find creative and interesting solutions—as in the genetic algorithm example you mention. But it cannot find dials which we had not noticed before. A black-box method can explore previously-unexplored areas of the design space, but it cannot expand our concept of what the “design space” is beyond the space we’ve told it to search.
A black-box algorithm doesn’t think outside the box it’s given.
I can imagine modeling strategies which feel relatively “gears-level” yet don’t make use of prior knowledge or “think outside the box they’re given”. I think there are a few entangled dimensions here which could be disentangled in principle.
I think that there’s a sliding scale between a black-box and a gears-level model; any gears-level model has black box components, and a mostly black-box model may include gears.
E.g. if you experimentally arrive at a physics equation that correctly describes how the wheel-with-weights behaves under a wide variety of parameters, this is more gearsy than just knowing the right settings for one set of parameters. But the deeper laws of physics which generated that equation are still a black box. While you might know how to adjust the weights if the slope changes, you won’t know how you should adjust them if fundamental physical constants were to change.
(Setting aside the point that fundamental physics constants changing would break your body so you couldn’t adjust the weights because you would be dead anyway.)
To put it in different terms, in a black box model you take some things as axiomatic. Any kind of reasoning requires you to eventually fall back on axioms that are not justified further, so all models are at least somewhat black boxy. The difference is in whether you settle on axioms that are useful for a narrow set of circumstances, or on ones which allow for broader generalization.
I think you’re pointing to a true and useful thing, but “sliding scale” isn’t quite the right way to characterize it. Rather, I’d say that we’re always operating at some level(s) of abstraction, and there’s always a lowest abstraction level in our model—a ground-level abstraction, in which the pieces are atomic. For a black-box method, the ground-level abstraction just has the one monolithic black box in it.
A gearsy method has more than just one object in its ground-level abstraction. There’s some freedom in how deep the abstraction goes—we could say a gear is atomic, or we could go all the way down to atoms—and the objects at the bottom will always be treated as black boxes. But I’d say it’s not quite right to think of the model as “partially black-box” just because the bottom-level objects are atomic; it’s usually the top-level breakdown that matters. E.g., in the maze example from the post, the top and bottom halves of the maze are still atomic black boxes, but our gearsy insight is still 100% gearsy—it is an insight which will not ever apply to some random black box in the wild.
Gears/no gears is a binary distinction; there’s a big qualitative jump between a black-box method which uses no information about internal system structure, and a gearsy model which uses any information about internal structure (even just very simple information). We can add more gears, reduce the black-box components in a gears level model. But as soon as we make the very first jump from one monolithic black box to two atomic gears, we’ve gone from a black-box method which applies to any random system, to a gears-level investment which will pay out on our particular system and systems related to it.
Can you give 2-3 examples?
Of gears-level models that don’t make use of prior knowledge or entangled dimensions?
Gears-level models which don’t use prior knowledge or offer outside-the-box insights.
I don’t think that’s the same as “thinking outside the box you’re given”. That’s about power of extrapolation, which is a separate entangled dimension.
Anyway, suppose I’m thinking of a criterion. Of the integers 1-20, the ones which meet my criterion are 2, 3, 5, 7, 11, 13, 17, 19. I challenge you to write a program that determines whether a number meets my criterion or not. A “black-box” program might check to see if the number is on the list I gave. A “gears-level” program might check to see if the number is divisible by any integer besides itself and 1. The “gears-level” program is “within the box” in the sense that it is a program which returns True or False depending on whether my criterion is supposedly met—the same box the “black-box” program is in. And in principle it doesn’t have to be constructed using prior knowledge. Maybe you could find it by brute forcing all short programs and returning the shortest one which matches available data with minimal hardcoded integers, or some other method for searching program space.
Similarly, a being from another dimension could be transported to our dimension, observe some physical objects, try to make predictions about them, and deduce that F=ma. They aren’t using prior knowledge since their dimension works differently than ours. And they aren’t “thinking outside the box they’re given”, they’re trying to make accurate predictions, just as one could do with a black box model.
Ok, so for the primes examples, I’d say that the gears-level model is using prior information in the form of the universal prior. I’d think of the universal prior as a black-box method for learning gears-level models; it’s a magical thing which lets us cross the bridge from one to the other (sometimes). In general, “black-box methods for finding gears-level models” is one way I’d characterize the core problems of AGI.
One “box” in the primes example is just the integers from 0-20; the gears-level model gives us insight into what happens outside that range, while the black-box model does not.
Similarly for the being from another dimension: they’re presumably using a universal prior. And they may not bother thinking outside the box—they may only want to make accurate predictions about whatever questions are in front of them—but F = ma is a model which definitely can be used for all sorts of things in our universe, not just whatever specific physical outcomes the being wants to predict.
But I still don’t think I’ve properly explained what I mean by “outside the box”.
For the primes problem, a better example of “outside the box” would be suddenly introducing some other kind of “number”, like integer matrices or quadratic integers or something. A divisibility-based model would generalize (assuming you kept using a divisibility-based criterion) - not in the sense that the same program would work, but in the sense that we don’t need to restart from scratch when figuring out the pattern. The black-box model, on the other hand, would need to start more-or-less from scratch.
For the being from another dimension, a good example of “outside the box” would be a sudden change in fundamental constants—not so drastic as to break all the approximations, but enough that e.g. energies of chemical reactions all change. In that case, F = ma would probably still hold despite the distribution shift.
So I guess the best summary of what I mean by “outside the box” is something like “counterfactual changes which don’t correspond to anything in the design space/data”.
One key dimension is decomposition – I would say any gears model provides decomposition, but models can have it without gears.
For example, the error in any machine learning model can be broken down into bias + variance, which provides a useful model for debugging. But these don’t feel like gears in any meaningful sense, whereas, say, bootstrapping + weak learners feel like gears in understanding Random Forests.