The prevalence of fraud, misreporting of experimental conditions as more solid than they were, publication biases, and other problems can produce bogus literatures that look more solid than this (aside from the prior), e.g. parapsychologists have many experiments that appear to replicate to a substantial degree in the parapsychology literature.
Given the initial flurry of attention to attract people to the the topic this field doesn’t look particularly surprising to me on the assumption that it is studying a non-existent phenomenon. If that’s explained, then I don’t see reason to pay attention at all.
ETA: What probability did you assign to cold fusion of this sort existing when you first made this post, and what probability do you now assign in light of the evidence found by other commenters?
Before I read the article, I would have estimated that it was 100 times more likely to be some combination of scam and bad results; after reading the post but no comments (and crucially, not fact-checking the references provided, just observing that links were added that claimed to be references and could easily be checked), I adjusted toward the position that there was strong evidence that fusion was occurring at low temperature, but not in a way that is currently viable for energy production Call it 10-1 against any actual results, and still about 0 chance of anything commercially usable in the foreseeable future.. After reading all of the comments, I’m going to have to call it 10-1 against fusion, but only 5-1 against there being some type of actual anomalous behavior. The lack of reproducible results is hard to overcome, but I think that there is a high chance that there are some phenomena which will be hard to reproduce for some reason or other, and some the evidence here is very much what I would expect if something hard to reproduce was involved. Still no chance of usable technology within the timeframe I will make estimates for.
Well, my probability has gone down quite a bit, obviously. I didn’t know about the failed replication that OrphanWilde linked; if I had, it might have been enough to stop me from writing the whole thing up. That being said, my probability is still high enough to be in the “this is interesting” range. I’ll be keeping an eye out for cold fusion news. I certainly won’t be making any investments based on the assumption that it’s real.
The real reason to doubt cold fusion is real is because of lack of replication and communication between relevant experts to replicate the phenomena. Its been several decades and there isn’t continuous refinement and improvement.
Parapsychology has the same problem.
But, publication bias is a problem throughout science, especially in medicine and psychology but mainstream science believes in and trusts the treatment prescribed by doctors and psychiatrists/psychologists. This baffles me: no one in the mainstream of scientific authority says mental disorders don’t exist, but by the standards of bias and lack of replication, we would have to say so, wouldn’t we?
To be clear: there is no mainstream scientific theory that explains mental disorders. This is so obvious we forget it, right? Yet we don’t say mental disorders do not exist. We use different criteria based upon social status, not the quality of evidence. Now, maybe you don’t do this. I don’t know you that well, but its clear and easy to see the standard of evidence isn’t met in many other areas of science—like genetics and evolutionary development.
I know, I edited my remark after thinking a bit more to read, “no one in the mainstream...”
I am only trying to draw out the difference between claims of net power through some unknown physical process and the obvious flaws of cold fusion as a means of explaining the statistical phenomena. Non existence is obvious and trivial, but sometimes it merely points to our own ignorance. I am not a physicist so I use heuristics to decide who to believe, but they are mere heuristics and really shouldn’t be taken very seriously.
Have you read much of Less Wrong? It is a common theme here that the replication rates of famous published findings in most fields is low, reflecting the aggregate weight of such biases. The work of Ioannidis and others like him comes up frequently in discussions of medicine, psychology, and foreign aid. Some random examples: 1, 2, 3, 4.
And in fact the scope of those problems is fairly mainstream: people like Ioannidis are huge draws at conferences, and almost no one disagrees with them that a large portion of findings are false due to various biases: the problems persist more because of the coordination and incentive problems in finding and switching to better systems.
But this whole idea of the rate of error in repeatability of a field ignores all the experiments that are never done again because the same phenomena is found over and over again.
In general I agree. But I don’t believe you should claim a phenomena does not exist because it doesn’t fit with mainstream science.
If you think science says what can or cannot exist, you would only be rationally correct (that is correct by whatever criteria you have specified), not actually correct because the best answer is unknowable, so far. Thus, I maintain my agnosticism about net power, but not cold fusion. Cold fusion isn’t a real phenomena, but maybe some have obtained net power, or maybe not.
The prevalence of fraud, misreporting of experimental conditions as more solid than they were, publication biases, and other problems can produce bogus literatures that look more solid than this (aside from the prior), e.g. parapsychologists have many experiments that appear to replicate to a substantial degree in the parapsychology literature.
Given the initial flurry of attention to attract people to the the topic this field doesn’t look particularly surprising to me on the assumption that it is studying a non-existent phenomenon. If that’s explained, then I don’t see reason to pay attention at all.
ETA: What probability did you assign to cold fusion of this sort existing when you first made this post, and what probability do you now assign in light of the evidence found by other commenters?
Before I read the article, I would have estimated that it was 100 times more likely to be some combination of scam and bad results; after reading the post but no comments (and crucially, not fact-checking the references provided, just observing that links were added that claimed to be references and could easily be checked), I adjusted toward the position that there was strong evidence that fusion was occurring at low temperature, but not in a way that is currently viable for energy production Call it 10-1 against any actual results, and still about 0 chance of anything commercially usable in the foreseeable future.. After reading all of the comments, I’m going to have to call it 10-1 against fusion, but only 5-1 against there being some type of actual anomalous behavior. The lack of reproducible results is hard to overcome, but I think that there is a high chance that there are some phenomena which will be hard to reproduce for some reason or other, and some the evidence here is very much what I would expect if something hard to reproduce was involved. Still no chance of usable technology within the timeframe I will make estimates for.
Well, my probability has gone down quite a bit, obviously. I didn’t know about the failed replication that OrphanWilde linked; if I had, it might have been enough to stop me from writing the whole thing up. That being said, my probability is still high enough to be in the “this is interesting” range. I’ll be keeping an eye out for cold fusion news. I certainly won’t be making any investments based on the assumption that it’s real.
The real reason to doubt cold fusion is real is because of lack of replication and communication between relevant experts to replicate the phenomena. Its been several decades and there isn’t continuous refinement and improvement.
Parapsychology has the same problem.
But, publication bias is a problem throughout science, especially in medicine and psychology but mainstream science believes in and trusts the treatment prescribed by doctors and psychiatrists/psychologists. This baffles me: no one in the mainstream of scientific authority says mental disorders don’t exist, but by the standards of bias and lack of replication, we would have to say so, wouldn’t we?
To be clear: there is no mainstream scientific theory that explains mental disorders. This is so obvious we forget it, right? Yet we don’t say mental disorders do not exist. We use different criteria based upon social status, not the quality of evidence. Now, maybe you don’t do this. I don’t know you that well, but its clear and easy to see the standard of evidence isn’t met in many other areas of science—like genetics and evolutionary development.
Did not Szasz build a well-respected, successful, and famous career doing just that?
I know, I edited my remark after thinking a bit more to read, “no one in the mainstream...”
I am only trying to draw out the difference between claims of net power through some unknown physical process and the obvious flaws of cold fusion as a means of explaining the statistical phenomena. Non existence is obvious and trivial, but sometimes it merely points to our own ignorance. I am not a physicist so I use heuristics to decide who to believe, but they are mere heuristics and really shouldn’t be taken very seriously.
Have you read much of Less Wrong? It is a common theme here that the replication rates of famous published findings in most fields is low, reflecting the aggregate weight of such biases. The work of Ioannidis and others like him comes up frequently in discussions of medicine, psychology, and foreign aid. Some random examples: 1, 2, 3, 4.
And in fact the scope of those problems is fairly mainstream: people like Ioannidis are huge draws at conferences, and almost no one disagrees with them that a large portion of findings are false due to various biases: the problems persist more because of the coordination and incentive problems in finding and switching to better systems.
The idea of parapsychology as a control group for science is that standards in science need to be improved to be more reliable, so that efforts like the Reproducibility Project find that most reported findings are solid rather than ephemeral.
But this whole idea of the rate of error in repeatability of a field ignores all the experiments that are never done again because the same phenomena is found over and over again.
In general I agree. But I don’t believe you should claim a phenomena does not exist because it doesn’t fit with mainstream science.
If you think science says what can or cannot exist, you would only be rationally correct (that is correct by whatever criteria you have specified), not actually correct because the best answer is unknowable, so far. Thus, I maintain my agnosticism about net power, but not cold fusion. Cold fusion isn’t a real phenomena, but maybe some have obtained net power, or maybe not.