Well from a consequentialist perspective, if people with a stronger desire for scientific integrity self-select out of science, that makes science weaker in the long run.
I think a more realistic norm, which will likely create better outcomes, is for you personally to ensure that your work is at least in the top 40% for quality, and castigate anyone whose work is in the bottom 20%. Either of these practices should cause a gradual increase in quality if widely implemented (assuming these thresholds are tracked & updated as they change over time).
Is it necessary so? Today science means you spend considerable portion of your time doing bullshit instead of actual research. Wouldn’t you be in a much better position doing quality research if you’re earning good salary, saving a big portion of it, and doing science as a hobby?
It’s possible. That’s what I myself am doing—supporting myself with a part-time job while I self-study and do independent FAI research.
However, it’s harder have credibility in the eyes of the public with this path. And for good reason—the public has no easy way to tell apart a crank from a lone genius, since it’s hard to judge expertise in a domain unless you yourself are an expert in it. One could argue that the academia acts as a reasonable approximation of eigendemocracy and thereby solves this problem.
Anyway, if the scientists with credibility are the ones who don’t care about scientific integrity, that seems bad for public epistemology.
Note that Wei Dei also notes that he chose exit from academia, as did many others on Less Wrong and in our social circles (combined with surprising non-entry).
If this is the model of what is going on, that quality and useful research is much easier without academia, but academia is how one gains credibility, then destroying the credibility of academia would be the logical useful action.
quality and useful research is much easier without academia
I think you have to do a lot more to demonstrate this.
destroying the credibility of academia would be the logical useful action.
Did you read Scott Alexander’s recent posts on cultural evolution?
If the credibility of academia is destroyed, it’s not obvious something better will come along to fill that void. Why is it better to destroy than repair? Plus, if something new gets created, it will probably have its own set of flaws. The more pressure is put on your system (in terms of funding and status), the greater the incentive to game things, and the more the cracks will start to show.
I suggest instead of focusing on the destruction of a suboptimal means for ascertaining credibility, you focus on the creation of a superior means for ascertaining credibility. Let’s phase academia out after it has been made obsolete, not before.
The broader institutions around academia have been around since at least the Royal Society, which was founded in 1660. That’s usually the age I would put the rough institutions surrounding academia.
Royal Society in 1660 and current academia are very different beasts. For example the current citations/journal’s game is pretty new phenomenon. Peer-review wasn’t really a thing 100 years ago. Neither complex grant applications.
I thought peer-review had always been a core part of science in some form or another. I think you might be confusing external peer-view and editorial peer-review. As this Wikipedia article says:
The first peer-reviewed publication might have been the Medical Essays and Observationspublished by the Royal Society of Edinburgh in 1731. The present-day peer-review system evolved from this 18th-century process,[5] began to involve external reviewers in the mid-19th-century,[6] and did not become commonplace until the mid-20th-century.[7]
Peer review became a touchstone of the scientific method, but until the end of the 19th century was often performed directly by an editor-in-chief or editorial committee.[8][9][10]Editors of scientific journals at that time made publication decisions without seeking outside input, i.e. an external panel of reviewers, giving established authors latitude in their journalistic discretion. For example, Albert Einstein’s four revolutionary Annus Mirabilispapers in the 1905 issue of Annalen der Physikwere peer-reviewed by the journal’s editor-in-chief, Max Planck, and its co-editor, Wilhelm Wien, both future Nobel prize winners and together experts on the topics of these papers. On another occasion, Einstein was severely critical of the external review process, saying that he had not authorized the editor in chief to show his manuscript “to specialists before it is printed”, and informing him that he would “publish the paper elsewhere”.[11]
It’s true that external peer-review is recent, which I do think is a significant shift. But I would still think that the broader institution of peer-review is basically as old as science.
It’s a huge difference whether the reviewer is some anonymous person unrelated to the journal or whether it’s an editor in chief of the journal itself. I don’t think it’s appropriate to call the latter peer-review (there are no “peers” involved), but that’s not important.
Editor in chief has a strong motivation to have a good quality journal. If he rejects a good article, it’s his loss. On the contrary, anonymous peer have stronger motivation to use this as an opportunity to promote (get cited) his own research than to help journal curate the best science.
Let me try to rephrase the shift I see in science. Over the 20th century, science became bureaucraciesed, the process of “doing science” was largely formalised and standardised. Researchers obsess about impact factors, p-values, h-indexes, anonymous peer reviews, grants, currents...
There are actual rules in place that determine formally whether you are “good” scientist. That wasn’t the case over the most of the history of the science.
Also the “full-time” scientist who never did any other job than academy research was much less common in the past. Take Einstein as an example.
Oh, I think we both definitely agree that science has changed a lot. I do also think that it still very clearly has maintained a lot of its structure from its very early days, and to bring things back to John’s top level point, it is less obvious that that structure would redevelop if we were to give up completely on academia or something like that.
Well from a consequentialist perspective, if people with a stronger desire for scientific integrity self-select out of science, that makes science weaker in the long run.
I think a more realistic norm, which will likely create better outcomes, is for you personally to ensure that your work is at least in the top 40% for quality, and castigate anyone whose work is in the bottom 20%. Either of these practices should cause a gradual increase in quality if widely implemented (assuming these thresholds are tracked & updated as they change over time).
Is it necessary so? Today science means you spend considerable portion of your time doing bullshit instead of actual research. Wouldn’t you be in a much better position doing quality research if you’re earning good salary, saving a big portion of it, and doing science as a hobby?
It’s possible. That’s what I myself am doing—supporting myself with a part-time job while I self-study and do independent FAI research.
However, it’s harder have credibility in the eyes of the public with this path. And for good reason—the public has no easy way to tell apart a crank from a lone genius, since it’s hard to judge expertise in a domain unless you yourself are an expert in it. One could argue that the academia acts as a reasonable approximation of eigendemocracy and thereby solves this problem.
Anyway, if the scientists with credibility are the ones who don’t care about scientific integrity, that seems bad for public epistemology.
Note that Wei Dei also notes that he chose exit from academia, as did many others on Less Wrong and in our social circles (combined with surprising non-entry).
If this is the model of what is going on, that quality and useful research is much easier without academia, but academia is how one gains credibility, then destroying the credibility of academia would be the logical useful action.
I think you have to do a lot more to demonstrate this.
Did you read Scott Alexander’s recent posts on cultural evolution?
If the credibility of academia is destroyed, it’s not obvious something better will come along to fill that void. Why is it better to destroy than repair? Plus, if something new gets created, it will probably have its own set of flaws. The more pressure is put on your system (in terms of funding and status), the greater the incentive to game things, and the more the cracks will start to show.
I suggest instead of focusing on the destruction of a suboptimal means for ascertaining credibility, you focus on the creation of a superior means for ascertaining credibility. Let’s phase academia out after it has been made obsolete, not before.
Academia in the current form isn’t Lindy. It’s not like we’re doing this thousands of years. Current system of Academia is at most 70 years old.
The broader institutions around academia have been around since at least the Royal Society, which was founded in 1660. That’s usually the age I would put the rough institutions surrounding academia.
Royal Society in 1660 and current academia are very different beasts. For example the current citations/journal’s game is pretty new phenomenon. Peer-review wasn’t really a thing 100 years ago. Neither complex grant applications.
I thought peer-review had always been a core part of science in some form or another. I think you might be confusing external peer-view and editorial peer-review. As this Wikipedia article says:
It’s true that external peer-review is recent, which I do think is a significant shift. But I would still think that the broader institution of peer-review is basically as old as science.
It’s a huge difference whether the reviewer is some anonymous person unrelated to the journal or whether it’s an editor in chief of the journal itself. I don’t think it’s appropriate to call the latter peer-review (there are no “peers” involved), but that’s not important.
Editor in chief has a strong motivation to have a good quality journal. If he rejects a good article, it’s his loss. On the contrary, anonymous peer have stronger motivation to use this as an opportunity to promote (get cited) his own research than to help journal curate the best science.
Let me try to rephrase the shift I see in science. Over the 20th century, science became bureaucraciesed, the process of “doing science” was largely formalised and standardised. Researchers obsess about impact factors, p-values, h-indexes, anonymous peer reviews, grants, currents...
There are actual rules in place that determine formally whether you are “good” scientist. That wasn’t the case over the most of the history of the science.
Also the “full-time” scientist who never did any other job than academy research was much less common in the past. Take Einstein as an example.
Oh, I think we both definitely agree that science has changed a lot. I do also think that it still very clearly has maintained a lot of its structure from its very early days, and to bring things back to John’s top level point, it is less obvious that that structure would redevelop if we were to give up completely on academia or something like that.