I am going to ignore the parts of Mitchell’s comment where Mitchell repeats points I already responded to, which leaves us with one point:
The only way I can see to impose a quantitative framework upon this disagreement is to construct a Bayesian belief network encompassing all the key propositions in both your argument and my argument, and then we try to find where your probabilistic dependencies are different to mine.
I do not know what you mean by a Bayesian belief network about an argument. I humbly suggest that when you wrote that sentence, you were confused about how a quantitative framework (as you call) or a formal epistemological treatment (as I would call it) would go. Please allow me to give the miminum amount of exposition necessary for present purposes. Although it is a clear improvement over what Mitchell wrote, there might be mistakes in the following exposition because I came to “technical epistemology” after the age of 40 and life circumstances have prevented me from giving it the study it deserves. If Eliezer, Anna Salamon or Steve Rayhawk takes exception to anything I say below, then believe them, not what I say below.
In the gospel according to Jaynes, Pearl, Solomonoff, etc, there is one Bayesian belief network out of all the possible Bayesian belief networks that is the accurate model of reality. If we knew which one it was, we would be able to use it to answer any question about any cause-and-effect relationship that is in principle answerable—or so it seems to me according to my untutored understanding. Parenthetically, in Causality Pearl opines that systems of equations similar to the structural equation models pioneered by the econometricians are probably a better representation than Bayesian belief networks. Hutter and Schmidhuber I think use Turing machines instead of Bayesian belief networks. Needless to say, if you have a formal model of reality in one representation (Bayesian belief network, say), it is a fairly easy mathematical exercise to put it into a different representation.
So, there is one “objectively true” model of reality, but I do not know which one it is. Consequently, what I have as my model of reality is a distribution over models—er, to be precise, a distribution over candidates for the One True model. (I think I used the word “hypothesis” in my previous comment, but right now I prefer “candidate model”.) By “distribution” I mean a mapping from candidates to real numbers in the interval between 0 and 1. I will refer to these real numbers as probabilities. There are an uncountable number of candidates, and the only way to get the probabilities of the candidates to sum to 1 is if the probabilities of arithmetically longer candidates are geometrically smaller. This is the formal version of Occam’s Razor. Why do the probabilities need to sum to 1? Well, the short answer is the Kolmogorov axioms say so. Who made the Kolmogorov axioms God? Cox’s theorem did.
Since physics is the study of fundamental reality, when I say “our civilization’s standard physical model” I refer to our civilization’s standard model of fundamental reality. The word “fundamental” is in there to indicate that “Fairbanks is the capital of Alaska” is not in the model. Our civilization’s model of fundamental reality remains _in_formal. To produce a formal model would require more than one generation of scientific effort in my humble estimate. In other words, to get it done would entail some community of scientists working on it till they became experts at the work, which I would think would take at least ten years. Then that first generation of scientists would have to train a second generation. But maybe I am wrong and it would take only one generation of scientific research to produce a formal model sufficiently useful that researchers wielding the model could compete with professional physicists trained the conventional way. (It would be a very cool achievement and parenthetically a potent way to remove human cognitive biases from scientific research it seems to me according to my untutored understanding of formal epistemology).
Even though I do not have a formal model of fundamental reality, my knowledge of formal epistemology which I have attempted to summarize briefly above is still useful because (probabalistically speaking, that is, excluding the freak case where all the air molecules go to one half of the room) any process that produces a true model of reality must approximate the process by which evidence updates a distribution on candidate models of reality outlined briefly above. (In particular, the process of natural selection that produced the scientists that produced our civilization’s physical models must approximate the process outlined above.)
Well, that should be enough to correct Mitchell’s false or confusing or easily-misinterpreted statements (quoted above) about formal epistemology—which is the only end for which I have any patience left in this top-level submission by Mitchell.
In the gospel according to Jaynes, Pearl, Solomonoff, etc, there is one Bayesian belief network out of all the possible Bayesian belief networks, which is the accurate model of reality. If we knew which one it was, we would be able to use it to answer any question about any cause-and-effect relationship that is in principle answerable—or so it seems to me according to my untutored understanding.
If I have The True Belief Network then I don’t need to predict cause-and-effect relationships. I just know the full state of the timeless universe. I mean to ask, why is a belief network constrained to representing physical laws and not physical state? After all, my current network has a bit of both...
If I have The True Belief Network then I don’t need to predict cause-and-effect relationships. I just know the full state of the timeless universe. I mean to ask, why is a belief network constrained to representing physical laws and not physical state?
I did not say it is constrained to represent physical laws, wedrifid.
Could it be that you believe that my use of “cause-and-effect relationship” implies that constraint? If so, I’m not conceding the implication.
I’m not asking you to concede anything. I’m trying to explore your meaning. What would you (or, for that matter, the experts you cite) say is the One True Model? I can imagine various types of mathematical abstractions but aren’t sure which kind you are referring to.
The casual reader might be saying to himself, “There goes Hollerith with another long comment about what he is calling formal epistemology. Why doesn’t he have the manners to refrain from injecting a long thread on an unrelated topic into Mitchell’s article on monads, consciousness, etc?”
Well, two replies to that. First, I say formal epistemology is not unrelated. Mitchell has been writing for many years around these parts on how consciousness presents a problem for standard physics. He has even solicited donations to support him in researching the matter further, saying that it is dangerous to have a singularity without having done that research. So, one of the ways formal epistemology enters naturally into this comment section is that I humbly sumbit that anyone engaged in such a project that Mitchell is engaged in should have as part of his technical background an education in formal epistemology. It leads to crisper thinking, and given how many resources Mitchell is devoting to the project, his dedicating some of those resources to learning formal epistemology is probably a good use of his time (and, oh, by the way, I’m not going to pay anymore attention to his writings on consciousness, ontology, etc, till he does).
Second, now that it has become plain that my mention of formal epistemology might lead to a long thread of conversation, I will indeed move the conversation to this place. It might move back to Less Wrong in the form of a top-level article by me with a title something like Why most people here should probably learn technical epistemology, a.k.a., the math of rationality. This prospective article would cover no ground that Eliezer has not already covered, but when it is important to publicize some point, then it often wins to have more than one voice making that point.
It would be possible for a person to maintain that only the natural numbers exist, and that there is nothing else. They could point to all the things which can be described using natural numbers; and if you insisted that some particular thing was not actually a natural number, but merely had a relationship to the numbers, they would keep returning their focus to the numerical part of the description of everything, and handwave away every other aspect as not really real, or as itself just being another number.
In the discussion of whether color can be reduced to the motions of particles in space, I feel myself to be in a comparable situation. The discussion of color as such repeatedly turns into a discussion of particles in the light source or particles in the brain…
Perhaps someone out there has conducted the subjective experiment of attending to actual color for a moment, and asking themselves afresh whether this thing could “really” be just particles in motion. The first thing to ask yourself is whether this alleged identity derives any impetus at all from the intrinsic nature of particles in motion. If somehow you knew nothing of color experiences or of neuropsychology, would you have any reason to think, in contemplating any assortment of particles circulating in space, that “color” or “the experience of color” was there? I think not. The motivation for the identity comes entirely from the belief that the world in general has already been explained by a physics of this form, and so color (and everything else about consciousness) must, somehow, also reduce to particles in motion. There is nothing in the intrinsic nature of color or the intrinsic nature of particles in motion to make you think that it is even possible for one to be the other.
That is the sort of argument that you have to resort to with someone who thinks that color is particles, or that everything is a number. You have to draw their attention to their actual experience, and make them question from the very beginning whether what they are saying makes sense. But Richard, I have no idea how to do that within these epistemic formalisms you promote, which seem to mostly be good for arriving at the simplest possible causal structure for hidden causes, and say nothing about how to correctly think about appearances as such, or how to ensure that you are placing a thing in the right ontological category.
But can this formal epistemology be the whole of epistemology? What is your formal epistemic basis for thinking that something exists, or that you have experienced blueness, or that 1+1=2?
It might move back to Less Wrong in the form of a top-level article by me with a title something like Why most people here should probably learn technical epistemology, a.k.a., the math of rationality. This prospective article would cover no ground that Eliezer has not already covered, but when it is important to publicize some point, then it often wins to have more than one voice making that point.
I do not know what you mean by a Bayesian belief network about an argument.
What I called a BBN (it may be a generalization of the standard concept) is a belief system schema constructed to be capable of representing your reasoning and my reasoning. Nodes are propositions and arrows are inferential steps. The schema must contain a node for every proposition that I use and every proposition that you use, and similarly must contain an arrow for every inferential step appearing in the argument of either person. Once we have that diagram, our two arguments may then be represented as each flowing through a portion of it. We arrive at opposite truth values for a common terminal proposition, so the arguments are in contradiction. To resolve the contradiction or at least identify its cause, we move upstream and try to identify where initial conditions differ.
This process will most likely require one to state opinions regarding certain implicit premises, used by the other person, which did not even play a role in one’s own argument, as well as to express differing opinions about the arrows, i.e., about the implications of one proposition for the truth of another. One of us may regard the truth of B as independent of the truth of A, whereas the other would say that if A is true, then B is definitely false—or probably false. It is merely a formal process meant to guarantee that the sources of disagreement are mutually understood, something which should happen anyway if the disagreement has developed in a lucid and orderly fashion.
I am going to ignore the parts of Mitchell’s comment where Mitchell repeats points I already responded to, which leaves us with one point:
I do not know what you mean by a Bayesian belief network about an argument. I humbly suggest that when you wrote that sentence, you were confused about how a quantitative framework (as you call) or a formal epistemological treatment (as I would call it) would go. Please allow me to give the miminum amount of exposition necessary for present purposes. Although it is a clear improvement over what Mitchell wrote, there might be mistakes in the following exposition because I came to “technical epistemology” after the age of 40 and life circumstances have prevented me from giving it the study it deserves. If Eliezer, Anna Salamon or Steve Rayhawk takes exception to anything I say below, then believe them, not what I say below.
In the gospel according to Jaynes, Pearl, Solomonoff, etc, there is one Bayesian belief network out of all the possible Bayesian belief networks that is the accurate model of reality. If we knew which one it was, we would be able to use it to answer any question about any cause-and-effect relationship that is in principle answerable—or so it seems to me according to my untutored understanding. Parenthetically, in Causality Pearl opines that systems of equations similar to the structural equation models pioneered by the econometricians are probably a better representation than Bayesian belief networks. Hutter and Schmidhuber I think use Turing machines instead of Bayesian belief networks. Needless to say, if you have a formal model of reality in one representation (Bayesian belief network, say), it is a fairly easy mathematical exercise to put it into a different representation.
So, there is one “objectively true” model of reality, but I do not know which one it is. Consequently, what I have as my model of reality is a distribution over models—er, to be precise, a distribution over candidates for the One True model. (I think I used the word “hypothesis” in my previous comment, but right now I prefer “candidate model”.) By “distribution” I mean a mapping from candidates to real numbers in the interval between 0 and 1. I will refer to these real numbers as probabilities. There are an uncountable number of candidates, and the only way to get the probabilities of the candidates to sum to 1 is if the probabilities of arithmetically longer candidates are geometrically smaller. This is the formal version of Occam’s Razor. Why do the probabilities need to sum to 1? Well, the short answer is the Kolmogorov axioms say so. Who made the Kolmogorov axioms God? Cox’s theorem did.
Since physics is the study of fundamental reality, when I say “our civilization’s standard physical model” I refer to our civilization’s standard model of fundamental reality. The word “fundamental” is in there to indicate that “Fairbanks is the capital of Alaska” is not in the model. Our civilization’s model of fundamental reality remains _in_formal. To produce a formal model would require more than one generation of scientific effort in my humble estimate. In other words, to get it done would entail some community of scientists working on it till they became experts at the work, which I would think would take at least ten years. Then that first generation of scientists would have to train a second generation. But maybe I am wrong and it would take only one generation of scientific research to produce a formal model sufficiently useful that researchers wielding the model could compete with professional physicists trained the conventional way. (It would be a very cool achievement and parenthetically a potent way to remove human cognitive biases from scientific research it seems to me according to my untutored understanding of formal epistemology).
Even though I do not have a formal model of fundamental reality, my knowledge of formal epistemology which I have attempted to summarize briefly above is still useful because (probabalistically speaking, that is, excluding the freak case where all the air molecules go to one half of the room) any process that produces a true model of reality must approximate the process by which evidence updates a distribution on candidate models of reality outlined briefly above. (In particular, the process of natural selection that produced the scientists that produced our civilization’s physical models must approximate the process outlined above.)
Well, that should be enough to correct Mitchell’s false or confusing or easily-misinterpreted statements (quoted above) about formal epistemology—which is the only end for which I have any patience left in this top-level submission by Mitchell.
If I have The True Belief Network then I don’t need to predict cause-and-effect relationships. I just know the full state of the timeless universe. I mean to ask, why is a belief network constrained to representing physical laws and not physical state? After all, my current network has a bit of both...
I did not say it is constrained to represent physical laws, wedrifid.
Could it be that you believe that my use of “cause-and-effect relationship” implies that constraint? If so, I’m not conceding the implication.
I’m not asking you to concede anything. I’m trying to explore your meaning. What would you (or, for that matter, the experts you cite) say is the One True Model? I can imagine various types of mathematical abstractions but aren’t sure which kind you are referring to.
The casual reader might be saying to himself, “There goes Hollerith with another long comment about what he is calling formal epistemology. Why doesn’t he have the manners to refrain from injecting a long thread on an unrelated topic into Mitchell’s article on monads, consciousness, etc?”
Well, two replies to that. First, I say formal epistemology is not unrelated. Mitchell has been writing for many years around these parts on how consciousness presents a problem for standard physics. He has even solicited donations to support him in researching the matter further, saying that it is dangerous to have a singularity without having done that research. So, one of the ways formal epistemology enters naturally into this comment section is that I humbly sumbit that anyone engaged in such a project that Mitchell is engaged in should have as part of his technical background an education in formal epistemology. It leads to crisper thinking, and given how many resources Mitchell is devoting to the project, his dedicating some of those resources to learning formal epistemology is probably a good use of his time (and, oh, by the way, I’m not going to pay anymore attention to his writings on consciousness, ontology, etc, till he does).
Second, now that it has become plain that my mention of formal epistemology might lead to a long thread of conversation, I will indeed move the conversation to this place. It might move back to Less Wrong in the form of a top-level article by me with a title something like Why most people here should probably learn technical epistemology, a.k.a., the math of rationality. This prospective article would cover no ground that Eliezer has not already covered, but when it is important to publicize some point, then it often wins to have more than one voice making that point.
It would be possible for a person to maintain that only the natural numbers exist, and that there is nothing else. They could point to all the things which can be described using natural numbers; and if you insisted that some particular thing was not actually a natural number, but merely had a relationship to the numbers, they would keep returning their focus to the numerical part of the description of everything, and handwave away every other aspect as not really real, or as itself just being another number.
In the discussion of whether color can be reduced to the motions of particles in space, I feel myself to be in a comparable situation. The discussion of color as such repeatedly turns into a discussion of particles in the light source or particles in the brain…
Perhaps someone out there has conducted the subjective experiment of attending to actual color for a moment, and asking themselves afresh whether this thing could “really” be just particles in motion. The first thing to ask yourself is whether this alleged identity derives any impetus at all from the intrinsic nature of particles in motion. If somehow you knew nothing of color experiences or of neuropsychology, would you have any reason to think, in contemplating any assortment of particles circulating in space, that “color” or “the experience of color” was there? I think not. The motivation for the identity comes entirely from the belief that the world in general has already been explained by a physics of this form, and so color (and everything else about consciousness) must, somehow, also reduce to particles in motion. There is nothing in the intrinsic nature of color or the intrinsic nature of particles in motion to make you think that it is even possible for one to be the other.
That is the sort of argument that you have to resort to with someone who thinks that color is particles, or that everything is a number. You have to draw their attention to their actual experience, and make them question from the very beginning whether what they are saying makes sense. But Richard, I have no idea how to do that within these epistemic formalisms you promote, which seem to mostly be good for arriving at the simplest possible causal structure for hidden causes, and say nothing about how to correctly think about appearances as such, or how to ensure that you are placing a thing in the right ontological category.
But can this formal epistemology be the whole of epistemology? What is your formal epistemic basis for thinking that something exists, or that you have experienced blueness, or that 1+1=2?
I look forward to that.
What I called a BBN (it may be a generalization of the standard concept) is a belief system schema constructed to be capable of representing your reasoning and my reasoning. Nodes are propositions and arrows are inferential steps. The schema must contain a node for every proposition that I use and every proposition that you use, and similarly must contain an arrow for every inferential step appearing in the argument of either person. Once we have that diagram, our two arguments may then be represented as each flowing through a portion of it. We arrive at opposite truth values for a common terminal proposition, so the arguments are in contradiction. To resolve the contradiction or at least identify its cause, we move upstream and try to identify where initial conditions differ.
This process will most likely require one to state opinions regarding certain implicit premises, used by the other person, which did not even play a role in one’s own argument, as well as to express differing opinions about the arrows, i.e., about the implications of one proposition for the truth of another. One of us may regard the truth of B as independent of the truth of A, whereas the other would say that if A is true, then B is definitely false—or probably false. It is merely a formal process meant to guarantee that the sources of disagreement are mutually understood, something which should happen anyway if the disagreement has developed in a lucid and orderly fashion.