You could do worse than stick to scientists that work for companies in a competitive industry. A favorite example of mine is chemists who work for Dupont or some such company.
My reasoning is this: if a chemist is bad and he is designing, say, a new glue, then his glue won’t stick and he’ll be fired. If he is not fired because his boss is stupid then his boss will be fired then he’ll be fired. If the president of the company won’t fire him and his boss, then the customers won’t buy the glue (because it doesn’t stick) and the company will go bankrupt and the chemist and his boss and president will lose their jobs. If the customers buy the glue then they’ll have accidents and their ability to keep up their stupidity will be diminished, and meanwhile, smart people who are able to tell whether glue sticks will pay their money to companies that hire competent chemists.
In short, then, one way or another, the bad chemist will lose his job and the good chemist will get a raise. This assumes a competitive industry. In a monopolistic bureaucracy, all bets are off. In a regulated industry all bets are off.
This gets completely past any reliance on a human, individual or collective, arbiter of truth, such as “consensus”. If “consensus opinion” is wrong, i.e., if majority opinion is wrong, then the majority will gradually be fired, and/or lose business, and/or subject itself to harm, and so the majority will decrease, and the minority which is not wrong will increase and become the majority. The arbiter of truth here is not “consensus” but natural selection (of a sort).
At any given time, this process has already been well underway for a long time. So, for example, we can be sure that right now, the majority of chemists employed by companies in competitive industries know what they’re talking about. We can generalize to scientists generally: a company that is managing to survive in a competitive, unregulated industry isn’t going to hire a scientist if the scientist doesn’t add value at least equivalent to his salary. It can’t afford to. So the scientist has to know his stuff.
We can extend this to those schools which train scientists who work in industry. They have to teach valid science, they can’t teach garbage.
For chemists making glue, fine. The stickiness of glue is obvious enough to everyone that making sticky glue will work better than employing scientists to claim that your glue is sticky.
For things like pharmaceuticals, tobacco, etc, you’re far more likely to encounter distortion.
Yes, if the companies are just competing on brands of otherwise identical tobacco products, then we should expect uniform bias; and that’s what we see.
But the pattern of bias in pharmaceuticals surprises me. One might expect that competing companies would be biased towards their own products. If that were so, we could extract unbiased estimates by comparing across drug companies (at least for patented drugs). But that’s not what we see. There might be a small bias towards their own drugs, but it is swamped by a large bias towards in-patent drugs, regardless of owner, against off-patent drugs.
Yes, I can think of explanations, like that they are cooperating in not using up the public good of FDA credulity, but this isn’t what I would have predicted ahead of time.
I don’t mean to imply that I have a good grasp on the biases, just that they are surprising. The particular effect with patents happened with SSRIs, that they fell apart as their patents expired; I probably imply too much generality.
Drug companies study each others’ drugs all the time, because FDA approval of a particular drug requires the claim that it is better, at least for some population. A typical phase 3 study compares the company’s own drug, a similar recent drug, and the standard treatment that the two drugs are trying to displace.
Maybe the bias there is in expecting that the competitor’s drug is similar to their own (therefore also good), and that it’s newer than the standard treatment (and so more advanced and better).
You could do worse than stick to scientists that work for companies in a competitive industry. A favorite example of mine is chemists who work for Dupont or some such company.
My reasoning is this: if a chemist is bad and he is designing, say, a new glue, then his glue won’t stick and he’ll be fired. If he is not fired because his boss is stupid then his boss will be fired then he’ll be fired. If the president of the company won’t fire him and his boss, then the customers won’t buy the glue (because it doesn’t stick) and the company will go bankrupt and the chemist and his boss and president will lose their jobs. If the customers buy the glue then they’ll have accidents and their ability to keep up their stupidity will be diminished, and meanwhile, smart people who are able to tell whether glue sticks will pay their money to companies that hire competent chemists.
In short, then, one way or another, the bad chemist will lose his job and the good chemist will get a raise. This assumes a competitive industry. In a monopolistic bureaucracy, all bets are off. In a regulated industry all bets are off.
This gets completely past any reliance on a human, individual or collective, arbiter of truth, such as “consensus”. If “consensus opinion” is wrong, i.e., if majority opinion is wrong, then the majority will gradually be fired, and/or lose business, and/or subject itself to harm, and so the majority will decrease, and the minority which is not wrong will increase and become the majority. The arbiter of truth here is not “consensus” but natural selection (of a sort).
At any given time, this process has already been well underway for a long time. So, for example, we can be sure that right now, the majority of chemists employed by companies in competitive industries know what they’re talking about. We can generalize to scientists generally: a company that is managing to survive in a competitive, unregulated industry isn’t going to hire a scientist if the scientist doesn’t add value at least equivalent to his salary. It can’t afford to. So the scientist has to know his stuff.
We can extend this to those schools which train scientists who work in industry. They have to teach valid science, they can’t teach garbage.
For chemists making glue, fine. The stickiness of glue is obvious enough to everyone that making sticky glue will work better than employing scientists to claim that your glue is sticky.
For things like pharmaceuticals, tobacco, etc, you’re far more likely to encounter distortion.
Yes, if the companies are just competing on brands of otherwise identical tobacco products, then we should expect uniform bias; and that’s what we see.
But the pattern of bias in pharmaceuticals surprises me. One might expect that competing companies would be biased towards their own products. If that were so, we could extract unbiased estimates by comparing across drug companies (at least for patented drugs). But that’s not what we see. There might be a small bias towards their own drugs, but it is swamped by a large bias towards in-patent drugs, regardless of owner, against off-patent drugs.
Yes, I can think of explanations, like that they are cooperating in not using up the public good of FDA credulity, but this isn’t what I would have predicted ahead of time.
That surprises me too. Do you have a citation for it?
In fact, I’m surprised that drug companies do studies on each other’s drugs often enough that the effect can be discerned.
I don’t mean to imply that I have a good grasp on the biases, just that they are surprising. The particular effect with patents happened with SSRIs, that they fell apart as their patents expired; I probably imply too much generality.
Drug companies study each others’ drugs all the time, because FDA approval of a particular drug requires the claim that it is better, at least for some population. A typical phase 3 study compares the company’s own drug, a similar recent drug, and the standard treatment that the two drugs are trying to displace.
Maybe the bias there is in expecting that the competitor’s drug is similar to their own (therefore also good), and that it’s newer than the standard treatment (and so more advanced and better).