Gears-level models are the opposite of black-box models.
I originally wrote “Gears vs Behavior” after I told someone (who was already familiar with the phrase “gears-level model”) that their model didn’t have any gears in it. I had expected this to be immediately obvious and self-explanatory, but it wasn’t; they weren’t sure what properties a “gearsy” model would have which theirs didn’t. They agreed that their model was fairly black-box-y, but couldn’t a black-box model be gearsy, in some respects?
This post says: black box models are the opposite of gearsy models. The various interesting/useful properties of gearsy models follow from the fact that they’re not black boxes.
One important corollary to this (from a related comment): gears/no gears is a binary distinction, not a sliding scale. There’s a qualitative difference between a pure black-box model which does not say anything at all about a system’s internals, versus a model with at least some internal structure. This has practical consequences: as soon as a model has some internal structure, the model is a capital investment. As soon as we divide a single gearbox into two sub-gearboxes, we need an investment to verify that the division is correct (i.e. there’s no hidden crosstalk), but then we can use the division to predict how e.g. one sub-gearbox behaves when the other is hit with a hammer (even if we never saw data on such a thing before).
We can still add more gears, distinguish between coarser or finer-grained gears, but it’s that first jump from black-box to not-black-box which gives a model its first gears.
To my eye, “Gears in Understanding” was written to point to the concept of gears-level models, and attach a name to it. It did that job well. But it didn’t really give a “definition” of a gears-level model. It didn’t say what the key properties are which make a model “gears-level”, or when/why we should care about those particular properties. It gave some heuristics, but didn’t say why those heuristics all point toward the same thing, or what that thing is, or where to draw the boundary around the concept so as to cut reality at the joints.
Thus, problems like the one above: while many people intuitively understood the concept (i.e. knew what cluster in concept-space “gearsiness” pointed to), it wasn’t quite clear where the boundaries were or why it mattered.
Gears vs Behavior presents something closer to a definition: the defining property of gears-level models is that they’re not black boxes. They have internal structure, and that structure itself makes predictions about the world. Because gears-level models have predictive internal structure:
they can make predictions about side-channel data or out-of-distribution behavior
we can guess the value of one variable given the rest
if the model turns out to be wrong, that tells us additional things about the world
etc
The heuristics from “Gears in Understanding” apply because gears-level models aren’t just black boxes.
How Does “Gears vs Behavior” Relate to Other Posts On Gears-Level Modelling?
“Gears vs Behavior” is most closely partnered with “Gears-Level Models Are Capital Investments”. “Gears vs Behavior” provides a rough definition of gears-level models; it draws a boundary around the concept. “Gears-Level Models Are Capital Investments” explains why gears-level models are useful—and when they aren’t. That, in turn, tells us why this particular definition was useful in the first place.
At this point, however, I think a better way to “define” gears-level models is the dimensionality and conditional independence framework, laid out in “Anatomy of a Gear” and “Everyday Lessons From High-DImensional Optimization”. I still see Gears vs Behavior as a basically-correct “dual” to those frames: “Gears vs Behavior” focuses on the space of queries that models address, rather than the structures of the models themselves—it is a black-box definition of gears-level models. Dimensionality and conditional independence, on the other hand, give a gears-level model of gears-level models.
Also worth noting: the conditional independence framework for understanding gears-level models is basically identical to abstraction as information at a distance. If you want a technical formulation of the ideas, then that’s the place to go.
What to Take Away From The Post
Gears-level models are the opposite of black-box models.
I originally wrote “Gears vs Behavior” after I told someone (who was already familiar with the phrase “gears-level model”) that their model didn’t have any gears in it. I had expected this to be immediately obvious and self-explanatory, but it wasn’t; they weren’t sure what properties a “gearsy” model would have which theirs didn’t. They agreed that their model was fairly black-box-y, but couldn’t a black-box model be gearsy, in some respects?
This post says: black box models are the opposite of gearsy models. The various interesting/useful properties of gearsy models follow from the fact that they’re not black boxes.
One important corollary to this (from a related comment): gears/no gears is a binary distinction, not a sliding scale. There’s a qualitative difference between a pure black-box model which does not say anything at all about a system’s internals, versus a model with at least some internal structure. This has practical consequences: as soon as a model has some internal structure, the model is a capital investment. As soon as we divide a single gearbox into two sub-gearboxes, we need an investment to verify that the division is correct (i.e. there’s no hidden crosstalk), but then we can use the division to predict how e.g. one sub-gearbox behaves when the other is hit with a hammer (even if we never saw data on such a thing before).
We can still add more gears, distinguish between coarser or finer-grained gears, but it’s that first jump from black-box to not-black-box which gives a model its first gears.
How Does “Gears vs Behavior” Relate To “Gears in Understanding”?
To my eye, “Gears in Understanding” was written to point to the concept of gears-level models, and attach a name to it. It did that job well. But it didn’t really give a “definition” of a gears-level model. It didn’t say what the key properties are which make a model “gears-level”, or when/why we should care about those particular properties. It gave some heuristics, but didn’t say why those heuristics all point toward the same thing, or what that thing is, or where to draw the boundary around the concept so as to cut reality at the joints.
Thus, problems like the one above: while many people intuitively understood the concept (i.e. knew what cluster in concept-space “gearsiness” pointed to), it wasn’t quite clear where the boundaries were or why it mattered.
Gears vs Behavior presents something closer to a definition: the defining property of gears-level models is that they’re not black boxes. They have internal structure, and that structure itself makes predictions about the world. Because gears-level models have predictive internal structure:
they can make predictions about side-channel data or out-of-distribution behavior
we can guess the value of one variable given the rest
if the model turns out to be wrong, that tells us additional things about the world
etc
The heuristics from “Gears in Understanding” apply because gears-level models aren’t just black boxes.
How Does “Gears vs Behavior” Relate to Other Posts On Gears-Level Modelling?
“Gears vs Behavior” is most closely partnered with “Gears-Level Models Are Capital Investments”. “Gears vs Behavior” provides a rough definition of gears-level models; it draws a boundary around the concept. “Gears-Level Models Are Capital Investments” explains why gears-level models are useful—and when they aren’t. That, in turn, tells us why this particular definition was useful in the first place.
At this point, however, I think a better way to “define” gears-level models is the dimensionality and conditional independence framework, laid out in “Anatomy of a Gear” and “Everyday Lessons From High-DImensional Optimization”. I still see Gears vs Behavior as a basically-correct “dual” to those frames: “Gears vs Behavior” focuses on the space of queries that models address, rather than the structures of the models themselves—it is a black-box definition of gears-level models. Dimensionality and conditional independence, on the other hand, give a gears-level model of gears-level models.
Here’s a visual of how the four posts relate:
Also worth noting: the conditional independence framework for understanding gears-level models is basically identical to abstraction as information at a distance. If you want a technical formulation of the ideas, then that’s the place to go.