I mean what exactly did you expert, a free lunch? Getting causal info and using it is hard.
You miss the point. When it comes to interviewing candidates for job then we found out that unstructured human assessment doesn’t happen that good.
It could very well be that the standard unstructured way of reading papers is not optimal and that we should have Bayesian beliefs nets in which we plug numbers such as whether the experiment is experimental or observational.
Whether MetaMed or someone else succeeds at that task and provides a good improvement on the status quo isn’t certain but there are ideas to explore.
Is it clear that MetaMed as group of self professed Bayesians provide a useful service? Maybe, maybe not. On the other hand the philosophy on which MetaMed operates is not the standard philosophy on which the medical establishment operates.
I don’t know how Metamed works (and it’s sort of their secret sauce, so they probably will not tell us without an NDA). I am guessing it is some combination of doing (a) through (e) above for someone who cannot do it themselves, and possibly some B stats. Which seems like a perfectly sensible business model to me!
I don’t think the secret sauce is in the B stats part of what they are doing, though. If we had a hypothetical company called “Freqmed” that also humanwaved (a) through (e), and then used F stats I doubt they would get non-sensible answers. It’s about being sensible, not your identity as a statistician.
I can be F with Bayes nets. Bayes nets are just a conditional independence model.
I don’t know how successful Metamed will be, but I honestly wish them the best of luck. I certainly think there is a lot of crazy out there in data analysis, and it’s a noble thing to try to make money off of making things more sensible.
The thing is, I don’t know about a lot of the things that get talked about on LW. I do know about B and F a little bit, and about causality a little bit. And a huge chunk of stuff people say is just plain wrong. So I tell them it’s wrong, but they keep going and don’t change what they say at all. So how should I update—that folks on this rationalist community generally don’t know what they are talking about and refuse to change?
It’s like wikipedia—the first sentence in the article on confounders is wrong on wikipedia (there is a very simple 3 node example that violates that definition). The talk page on Bayesian networks is a multi-year tale of woe and ignorance. I once got into an edit war with a resident bridge troll for that article, and eventually gave up and left, because he had more time. What does that tell me about wikipedia?
But we don’t. MetaMed did come out of a certain kind of thinking. The project had a motivation.
I do know about B and F a little bit, and about causality a little bit.
Just because you know what the people in the statistic community mean when they say “Bayesian” doesn’t automatically mean that you know what someone on LW means when he says Bayesian.
If you look at the “What Bayesianism taught me”, there a person who changed their beliefs through learning about Bayesianism. Do the points he makes have something to do with Frequentism vs. Bayesianism? Not directly.
On the other hand he did change major beliefs about he thinks about how the world and epistemology.
That means that the term Bayesianism as used in that article isn’t completely empty.
It’s about being sensible
Sensiblism might be a fun name for a philosophy. On the first LW meetup where I attended one of the participants had a scooter. My first question was about his traveling speed and how much time he effectively wins by using it. On that question he gave a normal answer.
My second question was over the accident rate of scooters. He replied something along the lines: “I really don’t know, I should research the issue more in depth and get the numbers.” That not the kind of answer normal people give when faced with the question for safety of the mode of travel.
You could say he’s simply sensible while 99% of the population that out there that would answer the question differently isn’t. On the other hand it’s quite difficult to explain to those 99% that they aren’t sensible.
If you prod them a bit they might admit that knowing accident risks is useful for making a decision about one’s mode of travel but they don’t update on a deep level.
Then people like you come and say: “Well of course we should be sensible. There no need to point is about explicitly or to give it a fancy name. Being sensible should go without saying.”
The problem is that in practice it doesn’t go without saying and speaking about it is hard. Calling it Bayesianism might be a very confusing way to speak about it but it seems to be an improvement over having no words at all. Maybe tabooing Bayesianism as word on LW would be the right choice. Maybe the word produces more problems than it solves.
It’s like wikipedia—the first sentence in the article on confounders is wrong on wikipedia.
“In statistics, a confounding variable (also confounding factor, a confound, or confounder) is an extraneous variable in a statistical model that correlates (directly or inversely) with both the dependent variable and the independent variable.” is at the moment that sentence. How would you change the sentence? There no reason why we shouldn’t fix that issue right now.
How would you change the sentence? There no reason why we shouldn’t fix that issue right now.
Counterexamples to a definition (this example is under your definition but is clearly not what we mean by confounder) are easier than a definition. A lot of analytic philosophy is about this. Defining “intuitive terms” is often not as simple as it seems. See, e.g.:
If you think you can make a “sensible” edit based on this paper, I will be grateful if you did so!
re: the rest of your post, words mean things. B is a technical term. I think if you redefine B as internal jargon for LW you will be incomprehensible to stats/ML people, and you don’t want this. Communication across fields is hard enough as it is (“academic coordination problem”), let’s not make it harder by not using standard terminology.
Maybe tabooing Bayesianism as word on LW would be the right choice. Maybe the word produces more
problems than it solves.
I am 100% behind this idea (and in general taboo technical terms unless you really know a lot about it).
It’s about being sensible, not your identity as a statistician.
Speaking of, an interesting paper which distinguishes the Fisher approach to testing from the Neyman-Pearson approach and shows how you can unify/match some of that with Bayesian methods.
You might want to read a bit more precisely. I did choose my words when I said “promise of a solution” instead of “a solution”.
In particular MetaMed speaks about wanting to produce a system of Bayesian analysis of medical papers. (Bayesian mathematical assessment of diagnosis)
You miss the point. When it comes to interviewing candidates for job then we found out that unstructured human assessment doesn’t happen that good.
It could very well be that the standard unstructured way of reading papers is not optimal and that we should have Bayesian beliefs nets in which we plug numbers such as whether the experiment is experimental or observational.
Whether MetaMed or someone else succeeds at that task and provides a good improvement on the status quo isn’t certain but there are ideas to explore.
Is it clear that MetaMed as group of self professed Bayesians provide a useful service? Maybe, maybe not. On the other hand the philosophy on which MetaMed operates is not the standard philosophy on which the medical establishment operates.
I don’t know how Metamed works (and it’s sort of their secret sauce, so they probably will not tell us without an NDA). I am guessing it is some combination of doing (a) through (e) above for someone who cannot do it themselves, and possibly some B stats. Which seems like a perfectly sensible business model to me!
I don’t think the secret sauce is in the B stats part of what they are doing, though. If we had a hypothetical company called “Freqmed” that also humanwaved (a) through (e), and then used F stats I doubt they would get non-sensible answers. It’s about being sensible, not your identity as a statistician.
I can be F with Bayes nets. Bayes nets are just a conditional independence model.
I don’t know how successful Metamed will be, but I honestly wish them the best of luck. I certainly think there is a lot of crazy out there in data analysis, and it’s a noble thing to try to make money off of making things more sensible.
The thing is, I don’t know about a lot of the things that get talked about on LW. I do know about B and F a little bit, and about causality a little bit. And a huge chunk of stuff people say is just plain wrong. So I tell them it’s wrong, but they keep going and don’t change what they say at all. So how should I update—that folks on this rationalist community generally don’t know what they are talking about and refuse to change?
It’s like wikipedia—the first sentence in the article on confounders is wrong on wikipedia (there is a very simple 3 node example that violates that definition). The talk page on Bayesian networks is a multi-year tale of woe and ignorance. I once got into an edit war with a resident bridge troll for that article, and eventually gave up and left, because he had more time. What does that tell me about wikipedia?
But we don’t. MetaMed did come out of a certain kind of thinking. The project had a motivation.
Just because you know what the people in the statistic community mean when they say “Bayesian” doesn’t automatically mean that you know what someone on LW means when he says Bayesian.
If you look at the “What Bayesianism taught me”, there a person who changed their beliefs through learning about Bayesianism. Do the points he makes have something to do with Frequentism vs. Bayesianism? Not directly. On the other hand he did change major beliefs about he thinks about how the world and epistemology.
That means that the term Bayesianism as used in that article isn’t completely empty.
Sensiblism might be a fun name for a philosophy. On the first LW meetup where I attended one of the participants had a scooter. My first question was about his traveling speed and how much time he effectively wins by using it. On that question he gave a normal answer.
My second question was over the accident rate of scooters. He replied something along the lines: “I really don’t know, I should research the issue more in depth and get the numbers.” That not the kind of answer normal people give when faced with the question for safety of the mode of travel.
You could say he’s simply sensible while 99% of the population that out there that would answer the question differently isn’t. On the other hand it’s quite difficult to explain to those 99% that they aren’t sensible. If you prod them a bit they might admit that knowing accident risks is useful for making a decision about one’s mode of travel but they don’t update on a deep level.
Then people like you come and say: “Well of course we should be sensible. There no need to point is about explicitly or to give it a fancy name. Being sensible should go without saying.”
The problem is that in practice it doesn’t go without saying and speaking about it is hard. Calling it Bayesianism might be a very confusing way to speak about it but it seems to be an improvement over having no words at all. Maybe tabooing Bayesianism as word on LW would be the right choice. Maybe the word produces more problems than it solves.
“In statistics, a confounding variable (also confounding factor, a confound, or confounder) is an extraneous variable in a statistical model that correlates (directly or inversely) with both the dependent variable and the independent variable.” is at the moment that sentence. How would you change the sentence? There no reason why we shouldn’t fix that issue right now.
Counterexamples to a definition (this example is under your definition but is clearly not what we mean by confounder) are easier than a definition. A lot of analytic philosophy is about this. Defining “intuitive terms” is often not as simple as it seems. See, e.g.:
http://arxiv.org/abs/1304.0564
If you think you can make a “sensible” edit based on this paper, I will be grateful if you did so!
re: the rest of your post, words mean things. B is a technical term. I think if you redefine B as internal jargon for LW you will be incomprehensible to stats/ML people, and you don’t want this. Communication across fields is hard enough as it is (“academic coordination problem”), let’s not make it harder by not using standard terminology.
I am 100% behind this idea (and in general taboo technical terms unless you really know a lot about it).
But they don’t solve the problem of Wikipedia being in your judgement wrong about this point.
If you look at the dictionary you will find that most words have multiple meanings.They also happen to evolve meaning over time.
Let’s see if I can precommit to not posting here anymore.
Speaking of, an interesting paper which distinguishes the Fisher approach to testing from the Neyman-Pearson approach and shows how you can unify/match some of that with Bayesian methods.