Contrast this to the notion we have in probability theory, of an exact quantitative rational judgment. If 1% of women presenting for a routine screening have breast cancer, and 80% of women with breast cancer get positive mammographies, and 10% of women without breast cancer get false positives, what is the probability that a routinely screened woman with a positive mammography has breast cancer? 7.5%. You cannot say, “I believe she doesn’t have breast cancer, because the experiment isn’t definite enough.” You cannot say, “I believe she has breast cancer, because it is wise to be pessimistic and that is what the only experiment so far seems to indicate.” 7.5% is the rational estimate given this evidence, not 7.4% or 7.6%. The laws of probability are laws.
I’m having trouble understanding what is meant by this essay. Where do you think these probabilities in your likelihood estimation problem came from in the first place? What on Earth does it mean to be precisely certain of a probability?
In my mind, these are summaries of prior experience—things that came out of the dynamic state machine we know of as the world. Let’s start at the beginning: You have a certain incidence of cancer in the population. What is it? You don’t know yet, you have to test—this is a science problem. Hey, what is cancer in the first place? Science problem. You have this test—how are you going to figure out what the probabilities of false positives and false negatives are? You have to try the test, or you have no information on how effective it is, or what to plug into the squares in your probability problem.
Bayesian reasoning can be made to converge on something like a reasonable distribution of confidence if you keep feeding it information. Where does the information you feed it come from? If not experience, the process is eating it’s own tail, and cannot be giving you information about the world! Prior probabilities, absent any experience of the problem domain, are arbitrary, are they not?
Also, for more complicated problems such as following a distribution around in dynamic system: You also have to have a model of what the system is doing—that is also an assumption, not a certainty! If your theory about the world is wrong or inapplicable, your probability distribution is going to propagate from it’s initial value to a final value, and that final value will not accord with the statistics of the data coming from the external world. You’ll have to try different models, until you find a better one that starts spitting out the right output statistics for the right input statistics. You have no way of knowing that a model is right or wrong a-priori. Following bayesian statistics around in a deterministic state machine is a straightforward generalization of following single states around in a deterministic state machine but your idea of the dynamics is distinct (usually far simpler for one thing) from what the world is actually doing.
Prior probabilities with no experience in a domain at all is an incoherent notion, since that implies you don’t know what the words you’re using even refer to. Priors include all prior knowledge, including knowledge about the general class of problems like the one you’re trying to eyeball a prior for.
If you’re asked to perform experiments on finding out what tapirs eat—and you don’t know what tapirs even are, except that they eat something apparently, judging by the formulation of the problem—you’re already going to assign a prior of ~0 of ‘they eat candy wrappers and rocks and are poisoned by everything and anything else, including non-candy-wrapper plastics and objects made of stone’, because you have prior information on what ‘eating’ refers to and how it tends to work. You’re probably going to assign a high prior probability to the guess that tapirs are animals, and on the basis of that assign a high prior probability to them being either herbivores, omnivores or carnivores—or insectivores, unless you include that as carnivores—since that’s what you know most animals are.
Priors are all prior information. It would be thoroughly irrational of you to give the tapirs candy wrappers and then when they didn’t eat them, assume it was the wrong brand and start trying different ones.
For additional clarification on what priors mean, imagine that if you didn’t manage to give the tapirs something they actually are willing to eat within 24 hours, your family is going to be executed.
In that situation, what’s the rational thing to do? Are you going to start with metal sheets, car tires and ceramic pots, or are you going to start trying different kinds of animal food?
Also, for more complicated problems such as following a distribution around in dynamic system: You also have to have a model of what the system is doing—that is also an assumption, not a certainty!
I’m sure you have multiple possible model of the system. If you have accounted for the possibility that your model is incorrect, then it will not be an assumption, it will be something that can be approximated into a distribution of confidence.
Contrast this to the notion we have in probability theory, of an exact quantitative rational judgment. If 1% of women presenting for a routine screening have breast cancer, and 80% of women with breast cancer get positive mammographies, and 10% of women without breast cancer get false positives, what is the probability that a routinely screened woman with a positive mammography has breast cancer? 7.5%. You cannot say, “I believe she doesn’t have breast cancer, because the experiment isn’t definite enough.” You cannot say, “I believe she has breast cancer, because it is wise to be pessimistic and that is what the only experiment so far seems to indicate.” 7.5% is the rational estimate given this evidence, not 7.4% or 7.6%. The laws of probability are laws.
I’m having trouble understanding what is meant by this essay. Where do you think these probabilities in your likelihood estimation problem came from in the first place? What on Earth does it mean to be precisely certain of a probability?
In my mind, these are summaries of prior experience—things that came out of the dynamic state machine we know of as the world. Let’s start at the beginning: You have a certain incidence of cancer in the population. What is it? You don’t know yet, you have to test—this is a science problem. Hey, what is cancer in the first place? Science problem. You have this test—how are you going to figure out what the probabilities of false positives and false negatives are? You have to try the test, or you have no information on how effective it is, or what to plug into the squares in your probability problem.
Bayesian reasoning can be made to converge on something like a reasonable distribution of confidence if you keep feeding it information. Where does the information you feed it come from? If not experience, the process is eating it’s own tail, and cannot be giving you information about the world! Prior probabilities, absent any experience of the problem domain, are arbitrary, are they not?
Also, for more complicated problems such as following a distribution around in dynamic system: You also have to have a model of what the system is doing—that is also an assumption, not a certainty! If your theory about the world is wrong or inapplicable, your probability distribution is going to propagate from it’s initial value to a final value, and that final value will not accord with the statistics of the data coming from the external world. You’ll have to try different models, until you find a better one that starts spitting out the right output statistics for the right input statistics. You have no way of knowing that a model is right or wrong a-priori. Following bayesian statistics around in a deterministic state machine is a straightforward generalization of following single states around in a deterministic state machine but your idea of the dynamics is distinct (usually far simpler for one thing) from what the world is actually doing.
Prior probabilities with no experience in a domain at all is an incoherent notion, since that implies you don’t know what the words you’re using even refer to. Priors include all prior knowledge, including knowledge about the general class of problems like the one you’re trying to eyeball a prior for.
If you’re asked to perform experiments on finding out what tapirs eat—and you don’t know what tapirs even are, except that they eat something apparently, judging by the formulation of the problem—you’re already going to assign a prior of ~0 of ‘they eat candy wrappers and rocks and are poisoned by everything and anything else, including non-candy-wrapper plastics and objects made of stone’, because you have prior information on what ‘eating’ refers to and how it tends to work. You’re probably going to assign a high prior probability to the guess that tapirs are animals, and on the basis of that assign a high prior probability to them being either herbivores, omnivores or carnivores—or insectivores, unless you include that as carnivores—since that’s what you know most animals are.
Priors are all prior information. It would be thoroughly irrational of you to give the tapirs candy wrappers and then when they didn’t eat them, assume it was the wrong brand and start trying different ones.
For additional clarification on what priors mean, imagine that if you didn’t manage to give the tapirs something they actually are willing to eat within 24 hours, your family is going to be executed.
In that situation, what’s the rational thing to do? Are you going to start with metal sheets, car tires and ceramic pots, or are you going to start trying different kinds of animal food?
Only responding to this part.
I’m sure you have multiple possible model of the system. If you have accounted for the possibility that your model is incorrect, then it will not be an assumption, it will be something that can be approximated into a distribution of confidence.