[…]the initial opaque box is not a model at all since it doesn’t specify anything.
Er… I disagree, though I wonder whether we’re just using the word “model” differently and interpreting “specify” differently because of that.
We don’t expect the righthand gear to, say, turn green and explode. We generally expect it’ll rotate clockwise at some constant rate or counterclockwise at some constant rate. And we can deduce whether the number of hidden gears is odd or even based on that. Before Joe speaks, I think we have to put even odds on those two possibilities, but those are our most likely possibilities.
After Joe speaks, we’re in the position that we’d want to take some bets at even odds on which way the righthand gear rotates. This suggests we have a falsifiable model of what’s going on in the situation. We’d even be willing to take some bets on the number of hidden gears.
And the odds at which we’d accept bets changes as the number of people who look and come to agree with Joe grows. So we can become more confident in our model given evidence.
But something really important changes when we see the inside. Suddenly a bunch of that probability mass from Joe and his friends converges on a specific subset of claims that could be rounded to “My model doesn’t match the world”. And a bunch of that same probability mass moves onto “Joe and his friends are all wrong.” Which means that if you find strong Bayesian evidence in favor of your model being right, it quickly overwhelms the evidence from Joe and his friends even if there are thousands of them. (I mean, you need more evidence about your model being right to justify defying larger numbers of people, but the point about evidential leverage remains.)
So, I don’t think it’s just about whether you have a model. It looks to me like there’s something about the nature of the model that changes when you look inside the box.
Next you take off the cover and suddenly you have a lot more observables. Moreover, you think you see a particular causal structure, a chain of causes and effects. Your world got a lot more complicated.
I worry when people start bringing in the word “causal”. I don’t really know what that means. I have an intuition, but I think that intuition is less clear than is my intuition about what Gears are.
E.g., the student who thinks you carry the 1 “because the teacher said so” might have a causal model that looks something like “Following social rules from authority figures invokes laws of magic that make things work”. In what sense does this causal model not “count”? Well, somehow we don’t like the invocation of laws of magic as though the magic is an axiomatic thing in reality. It seems to violate reductionism. But, um, a lot of the laws of physics seem like they come out of the magic aether of math, and we seem to be okay with that? We can dig into the philosophy of how reductionism and causal modeling should interact and so on…
…but it’s curious that we don’t have to in order to immediately get the intuition that there’s something wrong with the type of justification of “because the teacher said so”. Which tells me that what we’re picking up on isn’t likely to be a deep thing about the nature of causality and how it interacts with reductionism. That or we’re picking up on a proxy for that, and I happen to be calling the proxy “Gears-ness”.
The “paying rent” thing is basically about whether I care. And sure, I don’t care about things I don’t care about, but that does not affect the correctness (or adequateness, etc.) of the models in question. I don’t see why it’s a criterion of the model and not of which things I (subjectively) find important.
Hmm. I think you missed something really important about test #1.
It’s not enough for a model to pay rent. Gears-ness is a stronger property than paying rent. (Well, with a possible caveat. I suspect it’s possible to have a squatter belief that’s made of Gears. But I think we really don’t care about Gears-like squatter models even if they’re possible, so I’m going to ignore that for now.) The claim is that if a Gears-like model makes a prediction and the prediction is falsified, then you can deduce something else from the falsification.
E.g., if I tell you “All blergs are fizzles” and you can somehow go look at blergs, and you find a blerg that isn’t a fizzle, then you can confidently say “Nope, you’re wrong.” But you basically can’t deduce anything other than that. The model is just a floating (false) fact.
But if I tell you “All blergs are fizzles because all blergs eat snargles and all things that eat snargles also become fizzles in addition to what they were”, and you go find a blerg that isn’t a fizzle, then you can additionally deduce that either (a) not all blergs eat snargles or (b) some things that eat snargles don’t also become fizzles. You can know this even if you can’t observe snargles let alone what eats them or what happens to things that eat them.
Which is to say, the second snargle claim is more Gears-like than the first, even though they both pay just as much rent.
The “what if a given variable could be different” I don’t understand. It it an issue of how generic or robust or fragile the model is? Is it whether a model is falsifiable?
Nope, not about falsifiability. I don’t know what the words “generic” or “robust” or “fragile” mean in this context, so I can’t speak to that.
In the second snargle claim above (the more Gears-like one), it’s totally implausible to have both (a) the claim being true and (b) some snargles not also be fizzles. So you can’t flip that variable conditioned on the model being right.
I guess you could trivially say the same thing about the first one… but I say “trivially” because it’s tautologically true that you can’t have both A and not-A being true at the same time. I guess here there aren’t variables in the model, so there isn’t really a way to run test #2.
(This is me bumping into a place where the concept of Gears is missing some Gears, so I’m resorting back to the intuition for guidance. This is now having me speculate that a necessary condition of a model being Gears-like is that it has variables that aren’t just the whole model. But I’m only just now speculating about this.)
And the third criterion looks like redundancy to me. Or consistency? Or is it about how this particular model fits into a wider, more generic model of how the world works?
Also, in practice, I sometimes find test #2 easier to run and sometimes find test #3 easier to run. So from a purely pragmatic standpoint, test #3 still seems useful.
We don’t expect the righthand gear to, say, turn green and explode.
Ah, OK, that I would call context. Context is important. If something that looks like a gear sticks from one side of a box that an alien ship dropped off and there is another looks-like-a-gear thing on the other side, my expectations are that it might well turn green and explode. On the other hand, if we are looking at a Victorian cast-iron contraption, turning green is way down on my list of possibilities.
Context, basically, provides boundaries for the hypotheses that we are willing to consider. Sometimes we take a too narrow view and nothing fits inside the context boundaries—then widening of the context (sometimes explosively) is in order. But some context is necessary, otherwise you’d be utterly lost.
But something really important changes when we see the inside.
Well, you got some evidence that you have a strong tendency to believe (though I think there were some quite discouraging psych experiments about the degree to which people are willing to believe the social consensus over their own lying eyes). And yes, there is a pretty major difference between hearsay and personal experience. But still, I’m not sure where is the boundary that you wish to draw—see stage magic, optical illusions, convincing conmen, and general trickery.
there’s something about the nature of the model that changes when you look inside the box.
There is a traditional division of models into explanatory models and forecasting models. The point of a forecasting model is to provide a forecast—and that’s how it is judged. If it provides good forecasts, it might well be a black box and that’s not important. But for explanatory models being a black box is forbidden. The point of an explanatory model is to provide insight and, potentially, show what possible interventions could achieve.
Is that something related to your change of perspective as you open the box?
the word “causal”. I don’t really know what that means
There is a fair amount of literature on it—see e.g. Pearl—but, basically, a causal model makes stronger claims then, say, a correlational model. A correlational model would say things like “any time you see X you should expect to see Y”—and it might well be a very robust and well supported by evidence claim. A causal model, on the other hand, would say that X causes Y and that, specifically, changing X (an “intervention”) would lead to an appropriate change in Y. A correlational model does not make such a claim.
Interpreting correlational models as causal is a very common mistake.
In what sense does this causal model not “count”?
You test causal models by interventions—does manipulating X lead to the changes you expect in Y? If you are limited to passive observation, establishing causal models is… difficult.
to immediately get the intuition that there’s something wrong with the type of justification of “because the teacher said so”
Isn’t that just the hearsay vs personal experience difference?
The claim is that if a Gears-like model makes a prediction and the prediction is falsified, then you can deduce something else from the falsification.
Hmmm. OK, let me try to get at it from another side. Let’s say that Gearness is the property of being tied into the wider understanding of how the world works.
Generally speaking, you have an interconnected network of various models of how the world is constructed. Some are implied by others, some are explicitly dependent on others, etc. This network is vaguely tree-like in the sense that some models are closer to the roots and changes in them have wide-ranging repercussions (e.g. a religious (de)conversion) and some models are leaves and changes in them affect little if anything else (e.g. learning that whales on dying usually sink to the ocean floor).
Gearness would then be the degree to which a model is implied and constrained by “surrounding” knowledge. Does that make any sense?
Then the second test would be basically about the implications of a particular model / result for the surrounding knowledge. Is it deeply enmeshed or does it stand by itself? And the third test is about the same thing as well—how well does the model fit into the overall picture.
Er… I disagree, though I wonder whether we’re just using the word “model” differently and interpreting “specify” differently because of that.
We don’t expect the righthand gear to, say, turn green and explode. We generally expect it’ll rotate clockwise at some constant rate or counterclockwise at some constant rate. And we can deduce whether the number of hidden gears is odd or even based on that. Before Joe speaks, I think we have to put even odds on those two possibilities, but those are our most likely possibilities.
After Joe speaks, we’re in the position that we’d want to take some bets at even odds on which way the righthand gear rotates. This suggests we have a falsifiable model of what’s going on in the situation. We’d even be willing to take some bets on the number of hidden gears.
And the odds at which we’d accept bets changes as the number of people who look and come to agree with Joe grows. So we can become more confident in our model given evidence.
But something really important changes when we see the inside. Suddenly a bunch of that probability mass from Joe and his friends converges on a specific subset of claims that could be rounded to “My model doesn’t match the world”. And a bunch of that same probability mass moves onto “Joe and his friends are all wrong.” Which means that if you find strong Bayesian evidence in favor of your model being right, it quickly overwhelms the evidence from Joe and his friends even if there are thousands of them. (I mean, you need more evidence about your model being right to justify defying larger numbers of people, but the point about evidential leverage remains.)
So, I don’t think it’s just about whether you have a model. It looks to me like there’s something about the nature of the model that changes when you look inside the box.
I worry when people start bringing in the word “causal”. I don’t really know what that means. I have an intuition, but I think that intuition is less clear than is my intuition about what Gears are.
E.g., the student who thinks you carry the 1 “because the teacher said so” might have a causal model that looks something like “Following social rules from authority figures invokes laws of magic that make things work”. In what sense does this causal model not “count”? Well, somehow we don’t like the invocation of laws of magic as though the magic is an axiomatic thing in reality. It seems to violate reductionism. But, um, a lot of the laws of physics seem like they come out of the magic aether of math, and we seem to be okay with that? We can dig into the philosophy of how reductionism and causal modeling should interact and so on…
…but it’s curious that we don’t have to in order to immediately get the intuition that there’s something wrong with the type of justification of “because the teacher said so”. Which tells me that what we’re picking up on isn’t likely to be a deep thing about the nature of causality and how it interacts with reductionism. That or we’re picking up on a proxy for that, and I happen to be calling the proxy “Gears-ness”.
Hmm. I think you missed something really important about test #1.
It’s not enough for a model to pay rent. Gears-ness is a stronger property than paying rent. (Well, with a possible caveat. I suspect it’s possible to have a squatter belief that’s made of Gears. But I think we really don’t care about Gears-like squatter models even if they’re possible, so I’m going to ignore that for now.) The claim is that if a Gears-like model makes a prediction and the prediction is falsified, then you can deduce something else from the falsification.
E.g., if I tell you “All blergs are fizzles” and you can somehow go look at blergs, and you find a blerg that isn’t a fizzle, then you can confidently say “Nope, you’re wrong.” But you basically can’t deduce anything other than that. The model is just a floating (false) fact.
But if I tell you “All blergs are fizzles because all blergs eat snargles and all things that eat snargles also become fizzles in addition to what they were”, and you go find a blerg that isn’t a fizzle, then you can additionally deduce that either (a) not all blergs eat snargles or (b) some things that eat snargles don’t also become fizzles. You can know this even if you can’t observe snargles let alone what eats them or what happens to things that eat them.
Which is to say, the second snargle claim is more Gears-like than the first, even though they both pay just as much rent.
Nope, not about falsifiability. I don’t know what the words “generic” or “robust” or “fragile” mean in this context, so I can’t speak to that.
In the second snargle claim above (the more Gears-like one), it’s totally implausible to have both (a) the claim being true and (b) some snargles not also be fizzles. So you can’t flip that variable conditioned on the model being right.
I guess you could trivially say the same thing about the first one… but I say “trivially” because it’s tautologically true that you can’t have both A and not-A being true at the same time. I guess here there aren’t variables in the model, so there isn’t really a way to run test #2.
(This is me bumping into a place where the concept of Gears is missing some Gears, so I’m resorting back to the intuition for guidance. This is now having me speculate that a necessary condition of a model being Gears-like is that it has variables that aren’t just the whole model. But I’m only just now speculating about this.)
Well, it might be redundant. I don’t know. It looks like it might be redundant. But I think that’s roughly equivalent to saying that having an understanding be truly a part of you is redundant with respect to being reliably confused by fiction in the area the understanding applies to.
Also, in practice, I sometimes find test #2 easier to run and sometimes find test #3 easier to run. So from a purely pragmatic standpoint, test #3 still seems useful.
Ah, OK, that I would call context. Context is important. If something that looks like a gear sticks from one side of a box that an alien ship dropped off and there is another looks-like-a-gear thing on the other side, my expectations are that it might well turn green and explode. On the other hand, if we are looking at a Victorian cast-iron contraption, turning green is way down on my list of possibilities.
Context, basically, provides boundaries for the hypotheses that we are willing to consider. Sometimes we take a too narrow view and nothing fits inside the context boundaries—then widening of the context (sometimes explosively) is in order. But some context is necessary, otherwise you’d be utterly lost.
Well, you got some evidence that you have a strong tendency to believe (though I think there were some quite discouraging psych experiments about the degree to which people are willing to believe the social consensus over their own lying eyes). And yes, there is a pretty major difference between hearsay and personal experience. But still, I’m not sure where is the boundary that you wish to draw—see stage magic, optical illusions, convincing conmen, and general trickery.
There is a traditional division of models into explanatory models and forecasting models. The point of a forecasting model is to provide a forecast—and that’s how it is judged. If it provides good forecasts, it might well be a black box and that’s not important. But for explanatory models being a black box is forbidden. The point of an explanatory model is to provide insight and, potentially, show what possible interventions could achieve.
Is that something related to your change of perspective as you open the box?
There is a fair amount of literature on it—see e.g. Pearl—but, basically, a causal model makes stronger claims then, say, a correlational model. A correlational model would say things like “any time you see X you should expect to see Y”—and it might well be a very robust and well supported by evidence claim. A causal model, on the other hand, would say that X causes Y and that, specifically, changing X (an “intervention”) would lead to an appropriate change in Y. A correlational model does not make such a claim.
Interpreting correlational models as causal is a very common mistake.
You test causal models by interventions—does manipulating X lead to the changes you expect in Y? If you are limited to passive observation, establishing causal models is… difficult.
Isn’t that just the hearsay vs personal experience difference?
Hmmm. OK, let me try to get at it from another side. Let’s say that Gearness is the property of being tied into the wider understanding of how the world works.
Generally speaking, you have an interconnected network of various models of how the world is constructed. Some are implied by others, some are explicitly dependent on others, etc. This network is vaguely tree-like in the sense that some models are closer to the roots and changes in them have wide-ranging repercussions (e.g. a religious (de)conversion) and some models are leaves and changes in them affect little if anything else (e.g. learning that whales on dying usually sink to the ocean floor).
Gearness would then be the degree to which a model is implied and constrained by “surrounding” knowledge. Does that make any sense?
Then the second test would be basically about the implications of a particular model / result for the surrounding knowledge. Is it deeply enmeshed or does it stand by itself? And the third test is about the same thing as well—how well does the model fit into the overall picture.