I think that I have only now really understood what Eliezer has been getting at with the past ten or so posts, this idea that you could be a scientist if you generated hypotheses using a robot controlled Ouija board. I think other readers have already said this numerous times, but this strikes me as terribly wrong.
First of all, good luck getting research funding for such hypotheses (and it wouldn’t be fair to leave out funding from the description of Science if you’re including institutional inertia and bias).
And I think we all know that in general, someone who used this method would never be able to get anywhere in academia, simply because they wouldn’t be respected.
That, I think, teaches an important lesson. Individual scientists are not required to come up with correct or even plausible hypotheses because we all know that individual rationality is flawed. But the aggregate community of scientists and the people who fund them work together to evaluate the plausibility of a given hypothesis, and thereby effectively carry out the Bayesian analysis that Eliezer speaks of.
So one of many thousands of scientists can propose an utterly harebrained theory, and even spend his life on it if he wants, and it will barely register as a blip on the collective scientific radar. But when SR and GR were proposed, it was pretty much taken as a given that they were true, because they HAD to be true. I read somewhere that the experiment done by Eddington to verify the bending of light around the sun was far from accurate enough to actually be a verification of relativity. But it was still taken as a verification, because everyone was pretty much convinced anyway. And conversely, no matter how many experiments the cold fusion people do that show some unexpected effects, nobody takes them very seriously.
Now, you might say that this system is horribly inefficient, and many people say this on a regular basis. But here, the problem is simply that no individual human being can process that much information, and so the time it takes for a given data point to propagate through the community is very long. Of course, the internet helps, and if scientific journals were free, that would probably help also. But ultimately, I think this inefficiency is precisely the cost of a network evaluating all of the priors to find out the plausibility of a theory.
Of course, it also reduces a scientist to nothing more than a cog in a machine, and many people who want to be heroic can’t deal with that. But in real life, no scientist is expected to evaluate his own hypothesis. They are expected to come up with a hypothesis, and try to verify it if they can get funding, and let the community decide to what extent the results are valid.
In real life a real scientist must test his own hypothesis and the hypotheses of others.
They must devise and test a hypothesis which lends itself to specific predictions offering a means of testing its validity. All observation, in a special field of science, must be either for or against your hypothesis or my hypothesis, if the observation is to advance science. Science advances only by investigators who know how to disprove the empty theories and are already working on it. Science advances only by disproofs. It can take many years before the scientific community gets it.
I think that I have only now really understood what Eliezer has been getting at with the past ten or so posts, this idea that you could be a scientist if you generated hypotheses using a robot controlled Ouija board. I think other readers have already said this numerous times, but this strikes me as terribly wrong.
First of all, good luck getting research funding for such hypotheses (and it wouldn’t be fair to leave out funding from the description of Science if you’re including institutional inertia and bias).
And I think we all know that in general, someone who used this method would never be able to get anywhere in academia, simply because they wouldn’t be respected.
That, I think, teaches an important lesson. Individual scientists are not required to come up with correct or even plausible hypotheses because we all know that individual rationality is flawed. But the aggregate community of scientists and the people who fund them work together to evaluate the plausibility of a given hypothesis, and thereby effectively carry out the Bayesian analysis that Eliezer speaks of.
So one of many thousands of scientists can propose an utterly harebrained theory, and even spend his life on it if he wants, and it will barely register as a blip on the collective scientific radar. But when SR and GR were proposed, it was pretty much taken as a given that they were true, because they HAD to be true. I read somewhere that the experiment done by Eddington to verify the bending of light around the sun was far from accurate enough to actually be a verification of relativity. But it was still taken as a verification, because everyone was pretty much convinced anyway. And conversely, no matter how many experiments the cold fusion people do that show some unexpected effects, nobody takes them very seriously.
Now, you might say that this system is horribly inefficient, and many people say this on a regular basis. But here, the problem is simply that no individual human being can process that much information, and so the time it takes for a given data point to propagate through the community is very long. Of course, the internet helps, and if scientific journals were free, that would probably help also. But ultimately, I think this inefficiency is precisely the cost of a network evaluating all of the priors to find out the plausibility of a theory.
Of course, it also reduces a scientist to nothing more than a cog in a machine, and many people who want to be heroic can’t deal with that. But in real life, no scientist is expected to evaluate his own hypothesis. They are expected to come up with a hypothesis, and try to verify it if they can get funding, and let the community decide to what extent the results are valid.
In real life a real scientist must test his own hypothesis and the hypotheses of others. They must devise and test a hypothesis which lends itself to specific predictions offering a means of testing its validity. All observation, in a special field of science, must be either for or against your hypothesis or my hypothesis, if the observation is to advance science. Science advances only by investigators who know how to disprove the empty theories and are already working on it. Science advances only by disproofs. It can take many years before the scientific community gets it.