For whosoever hath good inductive biases, to him more evidence shall be given, and he shall have an abundance: but whosoever hath not good inductive biases, from him shall be taken away even what little evidence that he hath.
Let me see if I’ve cottoned on by coming up with an example.
Say you work with someone for years, and often on Mondays they come in late & with a headache. Other days, their hands are shaking, or they say socially inappropriate things in meetings.
“Good inductive bias” appears to mean you update in the correct direction (alcoholism/drug addiction) on each of these separate occasions, whereas “bad inductive bias” means you shrug each occurrence off and then get presented with each new occurrence, as it were, de novo. So this could be glossed as basically “update incrementally.” Have I got the gist?
I think what’s mildly confusing is the normatively positive use of the word “bias,” which typically suggests deviation from ideal reasoning. But I suppose it is a bias in the sense that one could go too far and update on every little piece of random noise...
I think what’s mildly confusing is the normatively positive use of the word “bias,” which typically suggests deviation from ideal reasoning. But I suppose it is a bias in the sense that one could go too far and update on every little piece of random noise...
“Inductive bias” is a technical term, where the word bias isn’t meant negatively.
I think that’s it, though there are at least two sorts of bad bias. The one you describe (nothing is important enough to notice or remember) is one, but there’s also having a bad theory (“that annoying person is aiming it all at me”, for example, which would lead to not noticing evidence of things going wrong which have nothing to do with malice).
This is reminding me of one of my favorite bits from Illuminatus!. There’s a man with filing cabinets [1] full of information about the first Kennedy assassination. He’s convinced that someday, he’ll find the one fact which will make it all make sense. He doesn’t realize that half of what’s he’s got is lies people made up to cover their asses.
In the novel, there were five conspiracies to kill JFK—but that character isn’t going to find out about them.
Matthew (slightly paraphrased...)
What does this mean?
If you have good judgement about what things imply, you’ll be good at gathering evidence.
If you have poor judgement about what things imply, you’ll lose track of the meaning of the evidence you’ve got.
Let me see if I’ve cottoned on by coming up with an example.
Say you work with someone for years, and often on Mondays they come in late & with a headache. Other days, their hands are shaking, or they say socially inappropriate things in meetings.
“Good inductive bias” appears to mean you update in the correct direction (alcoholism/drug addiction) on each of these separate occasions, whereas “bad inductive bias” means you shrug each occurrence off and then get presented with each new occurrence, as it were, de novo. So this could be glossed as basically “update incrementally.” Have I got the gist?
I think what’s mildly confusing is the normatively positive use of the word “bias,” which typically suggests deviation from ideal reasoning. But I suppose it is a bias in the sense that one could go too far and update on every little piece of random noise...
“Inductive bias” is a technical term, where the word bias isn’t meant negatively.
I think that’s it, though there are at least two sorts of bad bias. The one you describe (nothing is important enough to notice or remember) is one, but there’s also having a bad theory (“that annoying person is aiming it all at me”, for example, which would lead to not noticing evidence of things going wrong which have nothing to do with malice).
This is reminding me of one of my favorite bits from Illuminatus!. There’s a man with filing cabinets [1] full of information about the first Kennedy assassination. He’s convinced that someday, he’ll find the one fact which will make it all make sense. He doesn’t realize that half of what’s he’s got is lies people made up to cover their asses.
In the novel, there were five conspiracies to kill JFK—but that character isn’t going to find out about them.
[1] The story was written before the internet.
It’s been far too long since I’ve heard this underlying point acknowledged! Thankyou!