There is always evidence to the contrary. An evaluation weighs the evidence on both sides. You can take any case where the FDA has said “There is no evidence that X”, and look up the notes from the panel they held where they considered the evidence for X and decided that the evidence against X outweighed it.
The phrase “There is no evidence that X” is the single best indicator of someone statistically deluded or dishonest.
I’d normally take “evidence that [clause]” or “evidence for [noun phrase]” to mean ‘(non-negligible) positive net evidence’. (But of course that can still be a lie, or the result of motivated cognition.) If I’m talking about evidence of either sign, I’d say “evidence whether [clause]” or “evidence about [noun phrase]”.
I think your usage is idiosyncratic. People routinely talk about evidence for and against, and evidence for is not the net, but the evidence in favor.
where they considered the evidence for X and decided that the evidence against X outweighed it.
It’s quite standard to talk about evidence for and against a proposition in exactly this way, as he reports the FDA did. Having talked about “the evidence for” and weighing against the “evidence against”, you don’t then deny the existence of the “evidence for” just because, in balance, you find the evidence against more convincing.
You’re slicing the language so thinly, and in such a nonstandard way, it seems like rationalization and motivated reasoning. No evidence means no evidence. No means no. It can mean *very very little too”. Fine. But it doesn’t mean “an appreciable amount that has a greater countervailing amount”.
But here the FDA has taken “The balance of the evidence is not enough for to be sure enough” and said “There is no evidence for”. The evidence cited as “no evidence” should move the estimate towards 84% certain that there is an effect in the general population.
In this case, honest eyeballing of the data would lead one to conclude that there is an effect.
There actually isn’t any evidence against an effect hypothesis, because they’re not testing an effect hypothesis for falsification at all. There just isn’t enough evidence against the null by their arbitrarily too high standard.
And this is the standard statistical test in medicine, whereby people think they’re being rigorously scientific. Still just 2 chromosomes away from chimpanzees.
This is why you never eyeball data. Humans are terrible at understanding randomness. This is why statistical analysis is so important.
Something that is at 84% is not at 95%, which is a low level of confidence to begin with - it is a nice rule of thumb, but really if you’re doing studies like this you want to crank it up even further to deal with problems with publication bias. publish regardless of whether you find an effect or not, and encourage others to do the same.
Publication bias (positive results are much more likely to be reported than negative results) further hurt your ability to draw conclusions.
The reason that the FDA said what they did is that there isn’t evidence to suggest that it does anything. If you don’t have statistical significance, then you don’t really have anything, even if your eyes tell you otherwise.
Some are more terrible than others. A little bit of learning is a dangerous thing. Grown ups eyeball their data and know the limits of standard hypothesis testing.
The reason that the FDA said what they did is that there isn’t evidence to suggest that it does anything.
Yeah, evidence that the FDA doesn’t accept doesn’t exist.
The people who believe that they are grown-ups who can eyeball their data and claim results which fly in the face of statistical rigor are almost invariably the people who are unable to do so. I have seen this time and again, and Dunning-Kruger suggests the same—the least able are very likely to do this based on the idea that they are better able to do it than most, whereas the most able people will look at it and then try to figure out why they’re wrong, and consider redoing the study if they feel that there might be a hidden effect which their present data pool is insufficient to note. However, repeating your experiment is always dangerous if you are looking for an outcome (repeating your experiment until you get the result you want is bad practice, especially if you don’t adjust things so that you are looking for a level of statistical rigor that is sufficient to compensate for the fact that you’re doing it over again), so you have to keep it very carefully in mind and control your experiment and set your expectations accordingly.
The problem we started with was that “statistical rigor” is generally not rigorous. Those employing it don’t know what it would mean under the assumptions of the test, and fewer still know that the assumptions make little sense.
The phrase “There is no evidence that X” is the single best indicator of someone statistically deluded or dishonest.
I’d normally take “evidence that [clause]” or “evidence for [noun phrase]” to mean ‘(non-negligible) positive net evidence’. (But of course that can still be a lie, or the result of motivated cognition.) If I’m talking about evidence of either sign, I’d say “evidence whether [clause]” or “evidence about [noun phrase]”.
I think your usage is idiosyncratic. People routinely talk about evidence for and against, and evidence for is not the net, but the evidence in favor.
It’s quite standard to talk about evidence for and against a proposition in exactly this way, as he reports the FDA did. Having talked about “the evidence for” and weighing against the “evidence against”, you don’t then deny the existence of the “evidence for” just because, in balance, you find the evidence against more convincing.
You’re slicing the language so thinly, and in such a nonstandard way, it seems like rationalization and motivated reasoning. No evidence means no evidence. No means no. It can mean *very very little too”. Fine. But it doesn’t mean “an appreciable amount that has a greater countervailing amount”.
But here the FDA has taken “The balance of the evidence is not enough for to be sure enough” and said “There is no evidence for”. The evidence cited as “no evidence” should move the estimate towards 84% certain that there is an effect in the general population.
Very good point.
In this case, honest eyeballing of the data would lead one to conclude that there is an effect.
There actually isn’t any evidence against an effect hypothesis, because they’re not testing an effect hypothesis for falsification at all. There just isn’t enough evidence against the null by their arbitrarily too high standard.
And this is the standard statistical test in medicine, whereby people think they’re being rigorously scientific. Still just 2 chromosomes away from chimpanzees.
This is why you never eyeball data. Humans are terrible at understanding randomness. This is why statistical analysis is so important.
Something that is at 84% is not at 95%, which is a low level of confidence to begin with - it is a nice rule of thumb, but really if you’re doing studies like this you want to crank it up even further to deal with problems with publication bias. publish regardless of whether you find an effect or not, and encourage others to do the same.
Publication bias (positive results are much more likely to be reported than negative results) further hurt your ability to draw conclusions.
The reason that the FDA said what they did is that there isn’t evidence to suggest that it does anything. If you don’t have statistical significance, then you don’t really have anything, even if your eyes tell you otherwise.
Some are more terrible than others. A little bit of learning is a dangerous thing. Grown ups eyeball their data and know the limits of standard hypothesis testing.
Yeah, evidence that the FDA doesn’t accept doesn’t exist.
The people who believe that they are grown-ups who can eyeball their data and claim results which fly in the face of statistical rigor are almost invariably the people who are unable to do so. I have seen this time and again, and Dunning-Kruger suggests the same—the least able are very likely to do this based on the idea that they are better able to do it than most, whereas the most able people will look at it and then try to figure out why they’re wrong, and consider redoing the study if they feel that there might be a hidden effect which their present data pool is insufficient to note. However, repeating your experiment is always dangerous if you are looking for an outcome (repeating your experiment until you get the result you want is bad practice, especially if you don’t adjust things so that you are looking for a level of statistical rigor that is sufficient to compensate for the fact that you’re doing it over again), so you have to keep it very carefully in mind and control your experiment and set your expectations accordingly.
The problem we started with was that “statistical rigor” is generally not rigorous. Those employing it don’t know what it would mean under the assumptions of the test, and fewer still know that the assumptions make little sense.