To the extent that using prior information about the world is useful in understanding the future, it’s sort of nonsensical to say someone shouldn’t think of themselves as Bayesian. To the extent someone is perhaps ignoring a correct methodological approach because of a misguided/misunderstood appeal to Bayesianism, that’s fine.
For example, back in the academic world I worked on research forecasting the U.S. yield curve. We did this using a series of non-linear equations to fit each day (cross-section), then used filtering dynamics to jointly capture the time-series. Figuring out a way to make this already insanely complex model work within a Bayesian framework wouldn’t only be perhaps too hard, but not particularly useful. There is no nice quantifiable information that would fit in a prior, given the constraints we have on data, math, and computational ability, that would make the model formally Bayesian.
Having said that, to the extent that we tweaked the model using our prior information on less structured scientific theories (e.g. Efficient market hypothesis) -- it certainly was Bayesian. Sometimes the model worked perfectly, computed perfectly, but something didn’t match up with how we wanted to map it to reality. In that sense we had our own neural-prior and found the posterior less likely.
It’s really hard for me to see under what model of the world (correct) Bayesian analysis could be misleading.
One classic example of an unsophisticated position that’s common in analytic philosophy is the idea that all intellectual discourse is supposed to be based on logic. In Is semiotics bullshit? PhilGoetz stumbles about a professor of semiotics who claims: “People have an extra-computational ability to make correct judgements at better-than-random probability that have no logical basis.”
I think the claim that people can make correct judgements at better-than random probability that have no logical basis is nonsensical. Lots of this sort of writing and theorizing of the world is from a time, and from people, who existed before modern computational powers and machine learning. In the past the view was that the human mind was this almost-mystical device for processing reality, and we had to ground our beliefs in some sort of formal logic for them to follow. At least in my experience from working with stuff like neural-nets, I only see vast amounts of information, which our brains can filter out to predict the future. To reason from analogy, sometimes when doing some sort of ML problem you’ll add a bunch of data and you have no logical clue why it would improve your model… But then your predictions/classification score increase.
In this context what does it even mean to call this logical or non-logical? It’s nothing more than using past observed information patterns to classify and predict future information patterns. It’s strictly empirical. I can’t think of any logical decomposition of that which would add meaning.
To the extent someone is perhaps ignoring a correct methodological approach because of a misguided/misunderstood appeal to Bayesianism, that’s fine.
If someone sees themselves as a Bayesian with a capital “B” that person is likely prefer Bayesian methods of modeling a problem.
If I have a personal problem and do Gendlin’s Focusing, I come up with an intuitive solution. There’s little logic involved. There are real life cases where it makes more sense to follow the intuitive solution.
Is there ever a case where priors are irrelevant to a distinction or justification? That’s the difference between pure Bayesian reasoning and alternatives.
OP gave the example of the function of organs for a different purpose, but it works well here. To a pure Bayesian reasoner, there is no difference between saying that the heart has a function and saying that the heart is correlated with certain behaviors, because priors alone are not sufficient to distinguish the two. Priors alone are not sufficient to distinguish the two because the distinction has to do with ideals and definitions, not with correlations and experience.
If a person has issues with erratic blood flow leading to some hospital visit, why should we look at the heart for problems? Suppose there were a problem found with the heart. Why should we address the problem at that level as opposed to fixing the blood flow issue in some more direct way? What if there was no reason for believing that the heart problem would lead to anything but the blood flow problem? What’s the basis for addressing the underlying cause as opposed to addressing solely the issue that more directly led to a hospital visit?
There is no basis unless you recognize that addressing underlying causes tends to resolve issues more cleanly, more reliably, more thoroughly, and more persistently than addressing symptoms, and that the underlying cause only be identified by distinguishing erroneous functioning from other abnormalities. Pure Bayesian reasoners can’t make the distinction because the distinction has to do with ideals and definitions, not with correlations and experience.
It’s really hard for me to see under what model of the world (correct) Bayesian analysis could be misleading.
If you wanted a model that was never misleading, you might as well use first order logic to explain everything. Or go straight for the vacuous case and don’t try to explain anything. That problem is that that doesn’t generalize well, and it’s too restrictive. It’s about broadening your notion of reasoning so that you consider alternative justifications and more applications.
I don’t understand what you mean by ‘ideals and definitions,’ and how these are not influenced by past empirical observations and observed correlations. Any definition can simply be reduced to past observed correlations. The function of a heart is based strictly on past observations, and our mapping them to a functional model of how the heart behaves.
My argument seems trivial to me, because the idea that there is some non-empirical or correlated knowledge not based on past information seems nonsensical.
The distinction between “ideal” and “definition” is fuzzy the way I’m using it, so you can think of them as the same thing for simplicity.
Symmetry is an example of an ideal. It’s not a thing you directly observe. You can observe a symmetry, but there are infinitely many kinds of symmetries, and you have some general notion of symmetry that unifies all of them, including ones you’ve never seen. You can construct a symmetry that you’ve never seen, and you can do it algorithmically based on your idea of what symmetries are given a bit of time to think about the problem. You can even construct symmetries that, at first glance, would not look like a symmetry to someone else, and you can convince that someone else that what you’ve constructed is a symmetry.
The set of natural numbers is an example of something that’s defined, not observed. Each natural number is defined sequentially, starting from 1.
Addition is an example of something that’s defined, not observed.
The general notion of a bottle is an ideal.
In terms of philosophy, an ideal is the Platonic Form of a thing. In terms of category theory, an ideal is an initial or terminal object. In terms of category theory, a definition is a commutative diagram.
I didn’t say these things weren’t influenced by past observations and correlations. I said past observations and correlations were irrelevant for distinguishing them. Meaning, for example, you can distinguish between more natural numbers than your past experiences should allow.
I’m going to risk going down a meaningless rabbit hole here of semantic nothingness --
But I still disagree with your distinction, although I do appreciate the point you’re making. I view, and think the correct way to view, the human brain as simply a special case of any other computer. You’re correct that we have, as a collective species, proven and defined these abstract patterns. Yet even all these patterns are based on observations and rules of reasoning between our mind and the empirical reality. We can use our neurons to generate more sequences in a pattern, but the idea of an infinite set of numbers is only an abstraction or an appeal to something that could exist.
Similarly, a silicon computer can hold functions and mappings, but can never create an array of all numbers. They reduce down to electrical on-off switches, no matter how complex the functions are.
There is also no rule that says natural numbers or any category can’t change tomorrow. Or that right outside of the farthest information set in the horizon of space available to humans, the gravitational and laws of mathematics all shift by 0.1. It is sort of nonsensical, but it’s part of the view that the only difference between things that feel real and inherently distinguishable is our perception of how certain they are to continue based on prior information.
In my experience talking about this with people before, it’s not the type of thing people change their mind on (not implying your view is necessarily wrong). It’s a view of reality that we develop pretty foundationally, but I figured I’d write out my thoughts anyway for fun. It’s also sort of a self-indulgent argument about how we perceive reality. But, hey, it’s late and I’m relaxing.
I don’t understand what point you’re making with the computer, as we seem to be in complete agreement there. Nothing about the notion of ideals and definitions suggests that computers can’t have them or their equivalent. It’s obvious enough that computers can represent them, as you demonstrated with your example of natural numbers. It’s obvious enough that neurons and synapses can encode these things, and that they can fire in patterned ways based on them because… well that’s what neurons do, and neurons seem to be doing to bulk of the heavy lifting as far as thinking goes.
Where we disagree is in saying that all concepts that our neurons recognize are equivalent and that they should be reasoned about in the same way. There are clearly some notions that we recognize as being valid only after seeing sufficient evidence. For these notions, I think bayesian reasoning is perfectly well-suited. There are also clearly notions we recognize as being valid for which no evidence is required. For these, I think we need something else. For these notions, only usefulness is required, and sometimes not even that. Bayesian reasoning cannot deal with this second kind because their acceptability has nothing to do with evidence.
You argue that this second kind is irrelevant because these things exist solely in people’s minds. The problem is that the same concepts recur again and again in many people minds. I think I would agree with you if we only ever had to deal with a physical world in which people’s minds did not matter all that much, but that’s not the world we live in. If you want to be able to reliably convey your ideas to others, if you want to understand how people think at a more fundamental level, if you want your models to be useful to someone other than yourself, if you want to develop ideas that people will recognize as valid, if you want to generalize ideas that other people have, if you want your thoughts to be integrated with those of a community for mutual benefit, then you cannot ignore these abstract patterns because these abstract patterns constitute such a vast amount of how people think.
It also, incidentally, has a tremendous impact on how your own brain thinks and the kinds of patterns your brain lets you consciously recognize. If you want to do better generalizing your own ideas in reliable and useful ways, then you need to understand how they work.
For what it’s worth, I do think there are physically-grounded reasons for why this is so.
To the extent that using prior information about the world is useful in understanding the future, it’s sort of nonsensical to say someone shouldn’t think of themselves as Bayesian. To the extent someone is perhaps ignoring a correct methodological approach because of a misguided/misunderstood appeal to Bayesianism, that’s fine.
For example, back in the academic world I worked on research forecasting the U.S. yield curve. We did this using a series of non-linear equations to fit each day (cross-section), then used filtering dynamics to jointly capture the time-series. Figuring out a way to make this already insanely complex model work within a Bayesian framework wouldn’t only be perhaps too hard, but not particularly useful. There is no nice quantifiable information that would fit in a prior, given the constraints we have on data, math, and computational ability, that would make the model formally Bayesian.
Having said that, to the extent that we tweaked the model using our prior information on less structured scientific theories (e.g. Efficient market hypothesis) -- it certainly was Bayesian. Sometimes the model worked perfectly, computed perfectly, but something didn’t match up with how we wanted to map it to reality. In that sense we had our own neural-prior and found the posterior less likely.
It’s really hard for me to see under what model of the world (correct) Bayesian analysis could be misleading.
I think the claim that people can make correct judgements at better-than random probability that have no logical basis is nonsensical. Lots of this sort of writing and theorizing of the world is from a time, and from people, who existed before modern computational powers and machine learning. In the past the view was that the human mind was this almost-mystical device for processing reality, and we had to ground our beliefs in some sort of formal logic for them to follow. At least in my experience from working with stuff like neural-nets, I only see vast amounts of information, which our brains can filter out to predict the future. To reason from analogy, sometimes when doing some sort of ML problem you’ll add a bunch of data and you have no logical clue why it would improve your model… But then your predictions/classification score increase.
In this context what does it even mean to call this logical or non-logical? It’s nothing more than using past observed information patterns to classify and predict future information patterns. It’s strictly empirical. I can’t think of any logical decomposition of that which would add meaning.
If someone sees themselves as a Bayesian with a capital “B” that person is likely prefer Bayesian methods of modeling a problem.
If I have a personal problem and do Gendlin’s Focusing, I come up with an intuitive solution. There’s little logic involved. There are real life cases where it makes more sense to follow the intuitive solution.
Is there ever a case where priors are irrelevant to a distinction or justification? That’s the difference between pure Bayesian reasoning and alternatives.
OP gave the example of the function of organs for a different purpose, but it works well here. To a pure Bayesian reasoner, there is no difference between saying that the heart has a function and saying that the heart is correlated with certain behaviors, because priors alone are not sufficient to distinguish the two. Priors alone are not sufficient to distinguish the two because the distinction has to do with ideals and definitions, not with correlations and experience.
If a person has issues with erratic blood flow leading to some hospital visit, why should we look at the heart for problems? Suppose there were a problem found with the heart. Why should we address the problem at that level as opposed to fixing the blood flow issue in some more direct way? What if there was no reason for believing that the heart problem would lead to anything but the blood flow problem? What’s the basis for addressing the underlying cause as opposed to addressing solely the issue that more directly led to a hospital visit?
There is no basis unless you recognize that addressing underlying causes tends to resolve issues more cleanly, more reliably, more thoroughly, and more persistently than addressing symptoms, and that the underlying cause only be identified by distinguishing erroneous functioning from other abnormalities. Pure Bayesian reasoners can’t make the distinction because the distinction has to do with ideals and definitions, not with correlations and experience.
If you wanted a model that was never misleading, you might as well use first order logic to explain everything. Or go straight for the vacuous case and don’t try to explain anything. That problem is that that doesn’t generalize well, and it’s too restrictive. It’s about broadening your notion of reasoning so that you consider alternative justifications and more applications.
I don’t understand what you mean by ‘ideals and definitions,’ and how these are not influenced by past empirical observations and observed correlations. Any definition can simply be reduced to past observed correlations. The function of a heart is based strictly on past observations, and our mapping them to a functional model of how the heart behaves.
My argument seems trivial to me, because the idea that there is some non-empirical or correlated knowledge not based on past information seems nonsensical.
The distinction between “ideal” and “definition” is fuzzy the way I’m using it, so you can think of them as the same thing for simplicity.
Symmetry is an example of an ideal. It’s not a thing you directly observe. You can observe a symmetry, but there are infinitely many kinds of symmetries, and you have some general notion of symmetry that unifies all of them, including ones you’ve never seen. You can construct a symmetry that you’ve never seen, and you can do it algorithmically based on your idea of what symmetries are given a bit of time to think about the problem. You can even construct symmetries that, at first glance, would not look like a symmetry to someone else, and you can convince that someone else that what you’ve constructed is a symmetry.
The set of natural numbers is an example of something that’s defined, not observed. Each natural number is defined sequentially, starting from 1.
Addition is an example of something that’s defined, not observed. The general notion of a bottle is an ideal.
In terms of philosophy, an ideal is the Platonic Form of a thing. In terms of category theory, an ideal is an initial or terminal object. In terms of category theory, a definition is a commutative diagram.
I didn’t say these things weren’t influenced by past observations and correlations. I said past observations and correlations were irrelevant for distinguishing them. Meaning, for example, you can distinguish between more natural numbers than your past experiences should allow.
I’m going to risk going down a meaningless rabbit hole here of semantic nothingness --
But I still disagree with your distinction, although I do appreciate the point you’re making. I view, and think the correct way to view, the human brain as simply a special case of any other computer. You’re correct that we have, as a collective species, proven and defined these abstract patterns. Yet even all these patterns are based on observations and rules of reasoning between our mind and the empirical reality. We can use our neurons to generate more sequences in a pattern, but the idea of an infinite set of numbers is only an abstraction or an appeal to something that could exist.
Similarly, a silicon computer can hold functions and mappings, but can never create an array of all numbers. They reduce down to electrical on-off switches, no matter how complex the functions are.
There is also no rule that says natural numbers or any category can’t change tomorrow. Or that right outside of the farthest information set in the horizon of space available to humans, the gravitational and laws of mathematics all shift by 0.1. It is sort of nonsensical, but it’s part of the view that the only difference between things that feel real and inherently distinguishable is our perception of how certain they are to continue based on prior information.
In my experience talking about this with people before, it’s not the type of thing people change their mind on (not implying your view is necessarily wrong). It’s a view of reality that we develop pretty foundationally, but I figured I’d write out my thoughts anyway for fun. It’s also sort of a self-indulgent argument about how we perceive reality. But, hey, it’s late and I’m relaxing.
I don’t understand what point you’re making with the computer, as we seem to be in complete agreement there. Nothing about the notion of ideals and definitions suggests that computers can’t have them or their equivalent. It’s obvious enough that computers can represent them, as you demonstrated with your example of natural numbers. It’s obvious enough that neurons and synapses can encode these things, and that they can fire in patterned ways based on them because… well that’s what neurons do, and neurons seem to be doing to bulk of the heavy lifting as far as thinking goes.
Where we disagree is in saying that all concepts that our neurons recognize are equivalent and that they should be reasoned about in the same way. There are clearly some notions that we recognize as being valid only after seeing sufficient evidence. For these notions, I think bayesian reasoning is perfectly well-suited. There are also clearly notions we recognize as being valid for which no evidence is required. For these, I think we need something else. For these notions, only usefulness is required, and sometimes not even that. Bayesian reasoning cannot deal with this second kind because their acceptability has nothing to do with evidence.
You argue that this second kind is irrelevant because these things exist solely in people’s minds. The problem is that the same concepts recur again and again in many people minds. I think I would agree with you if we only ever had to deal with a physical world in which people’s minds did not matter all that much, but that’s not the world we live in. If you want to be able to reliably convey your ideas to others, if you want to understand how people think at a more fundamental level, if you want your models to be useful to someone other than yourself, if you want to develop ideas that people will recognize as valid, if you want to generalize ideas that other people have, if you want your thoughts to be integrated with those of a community for mutual benefit, then you cannot ignore these abstract patterns because these abstract patterns constitute such a vast amount of how people think.
It also, incidentally, has a tremendous impact on how your own brain thinks and the kinds of patterns your brain lets you consciously recognize. If you want to do better generalizing your own ideas in reliable and useful ways, then you need to understand how they work.
For what it’s worth, I do think there are physically-grounded reasons for why this is so.