“Modern-day best-practices industrial engineering works pretty well at it’s stated goals, and motivates theoretical progress as a result of subgoals” is not a particularly controversial claim. If you think there’s a way to do more with less, or somehow immunize the market for pure research against adverse selection due to frauds and crackpots, feel free to prove it.
I disagree. I don’t think there’s any consensus on this. The success of prizes/contests for motivating research shows that grand follies like the Concorde or Apollo project are far from the only effective funding mechanism, and most of the arguments for grand follies come from those with highly vested interests in them or conflicts of interest—the US government and affiliated academics are certainly happy to make ‘the Tang argument’ but I don’t see why one would trust them.
You haven’t made an argument that indirect funding is the best way to go and you’ve made baseless claims. There’s nothing to respond to: the burden of proof is on anyone who claims that bizarrely indirect mechanisms through flawed actors with considerable incentive to overstate efficacy and do said indirect mechanism (suppose funding the Apollo Project was an almost complete waste of money compared to the normal grant process; would NASA ever under any circumstances admit this?) is the best or even a good way to go compared to directly incentivizing the goal through contests or grants.
You haven’t made an argument that indirect funding is the best way to go
On this point we are in agreement. I’m not making any assertions about what the absolute best way is to fund research.
and you’ve made baseless claims.
Please be more specific.
There’s nothing to respond to: the burden of proof is on anyone who claims that bizarrely indirect mechanisms through flawed actors
All humans are flawed. Were you perhaps under the impression that research grant applications get approved or denied by a gleaming crystalline logic-engine handed down to us by the Precursors?
Here is the ‘bizarrely indirect’ mechanism by which I am claiming industrial engineering motivates basic research. First, somebody approaches some engineers with a set of requirements that, at a glance, to someone familiar with the current state of the art, seems impossible or at least unreasonably difficult. Money is piled up, made available to the engineers conditional on them solving the problem, until they grudgingly admit that it might be possible after all.
The problem is broken down into smaller pieces: for example, to put a man on the moon, we need some machinery to keep him alive, and a big rocket to get him and the machinery back to Earth, and an even bigger rocket to send the man and the machinery and the return rocket out there in the first place. The Tsiolkovsky rocket equation puts some heavy constraints on the design in terms of mass ratios, so minimizing the mass of the life-support machinery is important.
To minimize life-support mass while fulfilling the original requirement of actually keeping the man alive, the engineers need to understand what exactly the man might otherwise die of. No previous studies on the subject have been done, so they take a batch of laboratory-grade hamsters, pay someone to expose the hamsters to cosmic radiation in a systematic and controlled way, and carefully observe how sick or dead the hamsters become as a result. Basic research, in other words, but focused on a specific goal.
would NASA ever under any circumstances admit this?
They seem to be capable of acknowledging errors, yes. Are you?
“It turns out what we did in Apollo was probably the worst way we could have handled it operationally,” says Kriss Kennedy, project leader for architecture, habitability and integration at NASA’s Johnson Space Center in Houston, Texas, US.
That’s like asking “If homeopathy worked and all the doctors were wrong, would they admit it?” You can’t just flip a bit in the world setting Homeopathy_Works to TRUE and keep everything else the same. If homeopathy worked and yet doctors still didn’t accept it, that would imply that doctors are very different than they are now, and that difference would manifest itself in lots of other ways than just doctors’ opinion on homeopathy.
If funding the Apollo Project was a complete waste of money compared to the normal grant process, the world would be a different place, because that would require levels of incompetency on NASA’s part so great that it would get noticed.
Or for another example: if psi was real, would James Randi believe it?
That’s like asking “If homeopathy worked and all the doctors were wrong, would they admit it?”
No; it’s like asking “If homeopathy didn’t work and all the homeopaths were wrong, would they admit it?” You can find plenty of critics of Big Science and/or government spending on prestige projects, just like you can find plenty of critics of homeopathy.
If funding the Apollo Project was a complete waste of money compared to the normal grant process, the world would be a different place, because that would require levels of incompetency on NASA’s part so great that it would get noticed.
If homeopathy was a complete waste of money compared to normal medicine implying ‘great’ levels of incompetency on homeopaths, how would the world look different than it does?
You can find plenty of critics of Big Science and/or government spending on prestige projects,
Those people generally claim that Apollo was a waste of money period, not that Apollo was a waste of money compared to going to the moon via the normal grant process.
That’s like asking “If homeopathy worked and all the doctors were wrong, would they admit it?” You can’t just flip a bit in the world setting Homeopathy_Works to TRUE and keep everything else the same.
You can look at cases like chiropractors. Over a long time there was a general belief that chiropractors didn’t provide any good for patients because they theory based on which chiropractors practice is in substantial conflict with the theories used by Western medicine.
Suddenly in 2008 Cochrane comes out with the claim that chiropractors actually do provide comparable health benefits for patients with back pain as conventional treatment for backpain.
A lot of the opposition to homeopathy is based on the fact that the theory base of homeopathy is in conflict with standard Western knowledge about how things are supposed to work.
People often fail to notice things for bad reasons.
There are very good reasons why finding that one set of studies shows an unusual result is not taken as proof by either doctors or scientists. (It is also routine for pseudoscientists to latch onto that one or few studies when they happen.)
In other words, chiropractic is not such a case.
[the] theory based on which chiropractors practice is in substantial conflict with the theories used by Western medicine.
I hope you’re not suggesting that the theories used by Western medicine are likely to be wrong here.
Cochrane meta studies are the gold standard. In general they do get taken as proof.
As a matter of simple Bayseianism, P(result is correct|result is unusual) depends on the frequency at which conventional wisdom is wrong, compared to the frequency at which other things (errors and statistical anomalies) exist that produce unusual results. The probability that the result of a study (or meta-study) is correct given that it produces an unusual result is not equivalent to the overall probability that studies from that source are correct, so “Cochrane meta studies are the gold standard” is not the controlling factor. (Imagine that 0.2% of their studies are erroneous, but conventional wisdom is wrong only 0.1% of the time. Then the probability that a study is right given that it produces a result contrary to conventional wisdom is only 1⁄3, even though the probability that studies in general are right is 99.8%.)
That’s why we have maxims like “extraordinary claims require extraordinary evidence”.
FYI it isn’t even clear the review he mentions says what he thinks it says, not to mention the reviewers noted most of the studies had high risk of bias. “Other therapies” as controls in the studies doesn’t necessarily mean therapies that are considered to be effective.
The evidence for chiropractic intervention for lower back pain is good enough that RationalWiki which is full of people who don’t like chiropractics write: “There is evidence that chiropractic can help alleviate symptoms of low back pain.”
RationalWiki then adds that the cost and risks still suggest to that it’s good to stay aware from chiropractors.
Conventional wisdom by people who care about evidence for medical treatment is these days is that chiropractical interventions have effects for alleviate symptoms of low back pain.
That makes it a good test to identify people who pretend to care about evidence-based medicine but who care about medicine being motivated by orthodox theory instead of empirical evidence.
people who don’t like chiropractics write: “There is evidence
Of course they’ll write that. After all, there is evidence. You were implying that there’s good evidence.
RationalWiki then adds that the cost and risks still suggest to that it’s good to stay aware from chiropractors.
In other words, the evidence isn’t all that good.
Conventional wisdom by people who care about evidence for medical treatment is these days is that chiropractical interventions have effects for alleviate symptoms of low back pain.
This is a no true Scotsman fallacy. You’re asserting that anyone who seems to be part of conventional wisdom but doesn’t agree doesn’t count because he doesn’t care about evidence.
No. Saying that costs and side effects aren’t worth something is very different than saying it doesn’t work and produces no effect.
Conventional treatment is often cheaper than chiropractics. Dismissing it on those grounds is very different than dismissing it on grounds that it produces no effect.
Given that they don’t like it they need to make some argument against it ;) Not being able to argue that it doesn’t work make them go for risks and cost effectiveness.
This is a no true Scotsman fallacy. You’re asserting that anyone who seems to be part of conventional wisdom but doesn’t agree doesn’t count because he doesn’t care about evidence.
Cochrane meta studies have a reputation that’s good enough that even venues like RationalWiki accept it when it comes to conclusions that they don’t like.
There no meta study that’s published after the Cochrane results that argues that the Cochrane analysis get’s things wrong. Conventional of evidence-based medicine than suggests to use the Cochrane results as best source of evidence.
It not only RationalWiki. Any good evidence-based source that has a writeup about chiropractics will these days tell you that the evidence suggests that it works for back pain for a value of works that means it works as well as other conventional treatments for back pain.
Saying that costs and side effects aren’t worth something is very different than saying it doesn’t work and produces no effect.
No, they’re not very different at all. In fact they are directly related. Saying that costs and side effects are too great means that costs and side effects are too great for the benefit you get. If there is some probability that the study is bad and there is no benefit, that gets factored into this comparison; the greater the probability that the study is bad, the more the costs and side effects tip the balance against getting the treatment.
Cochrane meta studies have a reputation that’s good enough that even venues like RationalWiki accept it when it comes to conclusions that they don’t like.
You didn’t say that everyone accepts it. You said that everyone who cares about evidence accepts it. This is equivalent to “the people who don’t accept it don’t count because their opinions are not really based on evidence”. Likewise, now you’re claiming “any good evidence-based source” will say that it works. Again, this is a No True Scotsman fallacy; you’re saying that anyone who disagrees can’t really be an evidence-based source.
It’s only a No True Scotsman if you can point to an actual citizen of Scotland who doesn’t meet the ‘true Scotsman’ standard.
You are conflating two claims here. One is that chiropractic is more expensive than conventional treatments for lower back pain, and the other is that chiropractic is less effective than conventional treatments for lower back pain. What support do you have for the latter claim?
Saying that costs and side effects are too great means that costs and side effects are too great for the benefit you get. If there is some probability that the study is bad and there is no benefit, that gets factored into this comparison; the greater the probability that the study is bad, the more the costs and side effects tip the balance against getting the treatment.
If there was some non-negligible probability that the study was bad, RationalWiki would, given their dislike for chiropractics, have seized upon that and discussed it explicitly, would they not?
They describe the Cochrane study as “weak evidence” that chiropractic is as effective as other therapy. This implicitly includes some non-negligible probability that the benefit is less than the study seems to say it is.
“works pretty well” is not a controversial claim, but “motivates theoretical progress” is more iffy.
Offhand, I would say that it motivates incremental progress and applied aspects. I don’t think it motivates attempts at breakthroughs and basic science.
‘Breakthroughs and basic science’ seem to be running in to diminishing returns lately. As a policy matter, I think we (human civilization) should focus more on applying what we already know about the basics, to do what we’re already doing more efficiently.
“Modern-day best-practices industrial engineering works pretty well at it’s stated goals, and motivates theoretical progress as a result of subgoals” is not a particularly controversial claim. If you think there’s a way to do more with less, or somehow immunize the market for pure research against adverse selection due to frauds and crackpots, feel free to prove it.
I disagree. I don’t think there’s any consensus on this. The success of prizes/contests for motivating research shows that grand follies like the Concorde or Apollo project are far from the only effective funding mechanism, and most of the arguments for grand follies come from those with highly vested interests in them or conflicts of interest—the US government and affiliated academics are certainly happy to make ‘the Tang argument’ but I don’t see why one would trust them.
I didn’t say it was the only effective funding mechanism. I didn’t say it was the best. Please respond to the argument I actually made.
You haven’t made an argument that indirect funding is the best way to go and you’ve made baseless claims. There’s nothing to respond to: the burden of proof is on anyone who claims that bizarrely indirect mechanisms through flawed actors with considerable incentive to overstate efficacy and do said indirect mechanism (suppose funding the Apollo Project was an almost complete waste of money compared to the normal grant process; would NASA ever under any circumstances admit this?) is the best or even a good way to go compared to directly incentivizing the goal through contests or grants.
On this point we are in agreement. I’m not making any assertions about what the absolute best way is to fund research.
Please be more specific.
All humans are flawed. Were you perhaps under the impression that research grant applications get approved or denied by a gleaming crystalline logic-engine handed down to us by the Precursors?
Here is the ‘bizarrely indirect’ mechanism by which I am claiming industrial engineering motivates basic research. First, somebody approaches some engineers with a set of requirements that, at a glance, to someone familiar with the current state of the art, seems impossible or at least unreasonably difficult. Money is piled up, made available to the engineers conditional on them solving the problem, until they grudgingly admit that it might be possible after all.
The problem is broken down into smaller pieces: for example, to put a man on the moon, we need some machinery to keep him alive, and a big rocket to get him and the machinery back to Earth, and an even bigger rocket to send the man and the machinery and the return rocket out there in the first place. The Tsiolkovsky rocket equation puts some heavy constraints on the design in terms of mass ratios, so minimizing the mass of the life-support machinery is important.
To minimize life-support mass while fulfilling the original requirement of actually keeping the man alive, the engineers need to understand what exactly the man might otherwise die of. No previous studies on the subject have been done, so they take a batch of laboratory-grade hamsters, pay someone to expose the hamsters to cosmic radiation in a systematic and controlled way, and carefully observe how sick or dead the hamsters become as a result. Basic research, in other words, but focused on a specific goal.
They seem to be capable of acknowledging errors, yes. Are you?
http://www.newscientist.com/article/dn11326
That’s like asking “If homeopathy worked and all the doctors were wrong, would they admit it?” You can’t just flip a bit in the world setting Homeopathy_Works to TRUE and keep everything else the same. If homeopathy worked and yet doctors still didn’t accept it, that would imply that doctors are very different than they are now, and that difference would manifest itself in lots of other ways than just doctors’ opinion on homeopathy.
If funding the Apollo Project was a complete waste of money compared to the normal grant process, the world would be a different place, because that would require levels of incompetency on NASA’s part so great that it would get noticed.
Or for another example: if psi was real, would James Randi believe it?
No; it’s like asking “If homeopathy didn’t work and all the homeopaths were wrong, would they admit it?” You can find plenty of critics of Big Science and/or government spending on prestige projects, just like you can find plenty of critics of homeopathy.
If homeopathy was a complete waste of money compared to normal medicine implying ‘great’ levels of incompetency on homeopaths, how would the world look different than it does?
Those people generally claim that Apollo was a waste of money period, not that Apollo was a waste of money compared to going to the moon via the normal grant process.
You can look at cases like chiropractors. Over a long time there was a general belief that chiropractors didn’t provide any good for patients because they theory based on which chiropractors practice is in substantial conflict with the theories used by Western medicine.
Suddenly in 2008 Cochrane comes out with the claim that chiropractors actually do provide comparable health benefits for patients with back pain as conventional treatment for backpain.
A lot of the opposition to homeopathy is based on the fact that the theory base of homeopathy is in conflict with standard Western knowledge about how things are supposed to work.
People often fail to notice things for bad reasons.
There are very good reasons why finding that one set of studies shows an unusual result is not taken as proof by either doctors or scientists. (It is also routine for pseudoscientists to latch onto that one or few studies when they happen.)
In other words, chiropractic is not such a case.
I hope you’re not suggesting that the theories used by Western medicine are likely to be wrong here.
Cochrane meta studies are the gold standard. In general they do get taken as proof.
The main point is that you don’t need to have a valid theory to be able to produce empirical results.
Then I’m also don’t believe that issues surrounding back pain are very well understood by today’s Western medicine.
As a matter of simple Bayseianism, P(result is correct|result is unusual) depends on the frequency at which conventional wisdom is wrong, compared to the frequency at which other things (errors and statistical anomalies) exist that produce unusual results. The probability that the result of a study (or meta-study) is correct given that it produces an unusual result is not equivalent to the overall probability that studies from that source are correct, so “Cochrane meta studies are the gold standard” is not the controlling factor. (Imagine that 0.2% of their studies are erroneous, but conventional wisdom is wrong only 0.1% of the time. Then the probability that a study is right given that it produces a result contrary to conventional wisdom is only 1⁄3, even though the probability that studies in general are right is 99.8%.)
That’s why we have maxims like “extraordinary claims require extraordinary evidence”.
FYI it isn’t even clear the review he mentions says what he thinks it says, not to mention the reviewers noted most of the studies had high risk of bias. “Other therapies” as controls in the studies doesn’t necessarily mean therapies that are considered to be effective.
The evidence for chiropractic intervention for lower back pain is good enough that RationalWiki which is full of people who don’t like chiropractics write: “There is evidence that chiropractic can help alleviate symptoms of low back pain.” RationalWiki then adds that the cost and risks still suggest to that it’s good to stay aware from chiropractors.
Conventional wisdom by people who care about evidence for medical treatment is these days is that chiropractical interventions have effects for alleviate symptoms of low back pain.
That makes it a good test to identify people who pretend to care about evidence-based medicine but who care about medicine being motivated by orthodox theory instead of empirical evidence.
Of course they’ll write that. After all, there is evidence. You were implying that there’s good evidence.
In other words, the evidence isn’t all that good.
This is a no true Scotsman fallacy. You’re asserting that anyone who seems to be part of conventional wisdom but doesn’t agree doesn’t count because he doesn’t care about evidence.
No. Saying that costs and side effects aren’t worth something is very different than saying it doesn’t work and produces no effect.
Conventional treatment is often cheaper than chiropractics. Dismissing it on those grounds is very different than dismissing it on grounds that it produces no effect. Given that they don’t like it they need to make some argument against it ;) Not being able to argue that it doesn’t work make them go for risks and cost effectiveness.
Cochrane meta studies have a reputation that’s good enough that even venues like RationalWiki accept it when it comes to conclusions that they don’t like.
There no meta study that’s published after the Cochrane results that argues that the Cochrane analysis get’s things wrong. Conventional of evidence-based medicine than suggests to use the Cochrane results as best source of evidence. It not only RationalWiki. Any good evidence-based source that has a writeup about chiropractics will these days tell you that the evidence suggests that it works for back pain for a value of works that means it works as well as other conventional treatments for back pain.
No, they’re not very different at all. In fact they are directly related. Saying that costs and side effects are too great means that costs and side effects are too great for the benefit you get. If there is some probability that the study is bad and there is no benefit, that gets factored into this comparison; the greater the probability that the study is bad, the more the costs and side effects tip the balance against getting the treatment.
You didn’t say that everyone accepts it. You said that everyone who cares about evidence accepts it. This is equivalent to “the people who don’t accept it don’t count because their opinions are not really based on evidence”. Likewise, now you’re claiming “any good evidence-based source” will say that it works. Again, this is a No True Scotsman fallacy; you’re saying that anyone who disagrees can’t really be an evidence-based source.
It’s only a No True Scotsman if you can point to an actual citizen of Scotland who doesn’t meet the ‘true Scotsman’ standard.
You are conflating two claims here. One is that chiropractic is more expensive than conventional treatments for lower back pain, and the other is that chiropractic is less effective than conventional treatments for lower back pain. What support do you have for the latter claim?
I covered that:
If there was some non-negligible probability that the study was bad, RationalWiki would, given their dislike for chiropractics, have seized upon that and discussed it explicitly, would they not?
They describe the Cochrane study as “weak evidence” that chiropractic is as effective as other therapy. This implicitly includes some non-negligible probability that the benefit is less than the study seems to say it is.
“works pretty well” is not a controversial claim, but “motivates theoretical progress” is more iffy.
Offhand, I would say that it motivates incremental progress and applied aspects. I don’t think it motivates attempts at breakthroughs and basic science.
‘Breakthroughs and basic science’ seem to be running in to diminishing returns lately. As a policy matter, I think we (human civilization) should focus more on applying what we already know about the basics, to do what we’re already doing more efficiently.