Yeah, but if you steel-man it, I think he was trying to make something similar to a map-territory distinction. It’s often useful to make a distinction between the data and our best interpretation of the data. Some conclusions don’t require much extrapolation, but others require a great deal.
On LW we happily discuss with very long inferential distances, and talk about regions of hypothesis space with high densities of unknown unknowns. Most scientists, however, work over much smaller inferential distances, with the intent of meticulously build up a rock solid body of knowledge. If things are “open questions” until they are above a confidence interval of, say, 0.99, then just about everything we discuss here is an open question, as the quote suggests.
Using a historical example which happens to be false just complicates things. If I recall, philosophers first hypothesized a round earth around 600 BCE, but didn’t prove it experimentally until 300 BCE.
The act of steelmanning means to argue against a different position then the one the person is holding. It very worthwhile to criticize people for holding positions for the wrong reasons.
I think he was trying to make something similar to a map-territory distinction. It’s often useful to make a distinction between the data and our best interpretation of the data. Some conclusions don’t require much extrapolation, but others require a great deal.
To me what you are saying doesn’t seem like a description of the map-territory distinction. A map is not an extrapolation of the territory but an abstraction of it.
If things are “open questions” until they are above a confidence interval of, say, 0.99
That sentence doesn’t look to me like it’s inspired by looking at what scientists do. I’m not aware of a scientific community having the standard of question being closed when they are over a confidence interval of 0.99.
You might argue that scientists should do things that way, but that doesn’t have much to do with the question of how scientists act in the real world.
It’s often useful to make a distinction between the data and our best interpretation of the data.
Statements about what’s useful are different than statements that describe what scientists do in reality.
Using a historical example which happens to be false just complicates things. If I recall, philosophers first hypothesized a round earth around 600 BCE, but didn’t prove it experimentally until 300 BCE.
I think you missed a point. At 300 BCE they were not centrally concerned with proving via experiment that the earths is round. They instead cared about things making sense intuitively. The idea that it’s important to prove claims via experiment came with Descartes into the scientific mosaic which happened much later.
Nobody at the time between 600 BCE and 300 BCE said: “The part of the Earth that I can see at any given time appears to be flat, which would be the case when looking at a small piece of many differently shaped objects up close, so I don’t have enough information to know what the shape of the Earth is. One reasonable hypothesis is that the Earth is flat, but until we have tools and techniques that can be used to prove or disprove that hypothesis, it is an open question.”
If things are “open questions” until they are above a confidence interval of, say, 0.99, then just about everything we discuss here is an open question, as the quote suggests.
“How does action at a distance work?” wasn’t an open question shortly after Descartes. It became again an open question when Newton was shown to be right by the expedition that measured the shape of the earth.
In biology the central dogma of molecular biology was considered a close question for a long time. Biologists where confident about the fact that a lot of the DNA is junk DNA that doesn’t do anything.
One of the main reasons why we don’t consider the question of whether homeopathy works an open question isn’t just that we lack empiric evidence for it working but that we based on our theories of chemistry we don’t believe that it could work.
Whether or not chiropratics interventions work was a question that scientists considered not to be open for a long time.
I think you’re right that the OP’s characterization of science is naive (though I think the everything-is-open idea is always there as an ideal, and I think that really matters), but all your examples here seem really dubious.
At 300 BCE they were not centrally concerned with proving via experiment that the earths is round. They instead cared about things making sense intuitively. The idea that it’s important to prove claims via experiment came with Descartes into the scientific mosaic which happened much later.
Then how did it come about that Eratosthenes proved via experiment that the earth is round and estimated its radius? (A little later than 300 BCE, I think, but in the right ballpark.)
“How does action at a distance work?” [...] became again an open question when Newton was shown to be right by the expedition that measured the shape of the earth.
Newton’s work was recognized as first-rank science from the beginning, so the idea of action at a distance was taken seriously (at least) from the publication of the Principia. I don’t know what expedition you are thinking of, but I am pretty sure it’s nowhere near the truth to say that action at a distance was thought to be rubbish until an expedition measured the shape of the earth and thereby proved that Newton was right after all.
[EDITED to add: OK, so I guess you mean the French Geodesic Mission of 1735, which found that the earth is bulgy in the middle (as Newton said it should be) rather than elongated towards the poles (as Descartes claimed, on the basis of what we would now regard as a crackpot theory of vortices). That’s hardly conclusive evidence for action at a distance, nor would the opposite result have been anything like conclusive the other way. But, anyway, a large part of the point of that expedition was that “Newton or Descartes?” was an open question. It would be nearer the truth to say that the expedition closed the question. But not much nearer, because as I say the question the survey resolved was not the question of action at a distance.]
the central dogma of molecular biology was considered a close question for a long time
The central dogma, at least as Crick stated it, says that the transcription from nucleotide sequence to amino acid sequence never goes the other way, with proteins getting decoded back into nucleic acids. So far as I know, this is still a closed question. Was it ever thought absolutely certain that there’s no mechanism by which proteins can affect the information in nucleic acids?
Biologists were confident about the fact that a lot of the DNA is junk DNA that doesn’t do anything.
They still are, and so far as I know they never claimed that all non-coding DNA is completely functionless. The “biologists thought most DNA was pure junk but now they’ve been stunned to find that some of it is useful” narrative is, I think, mostly hype.
though I think the everything-is-open idea is always there as an ideal, and I think that really matters),
If you have Einstein saying: “God doesn’t play dice” Einstein wasn’t following the ideal to be open about him playing dice.
Newton’s work was recognized as first-rank science from the beginning, so the idea of action at a distance was taken seriously (at least) from the publication of the Principia.
Newton was accepted much sooner in the UK than on the continent. On the continent the expedition was important.
The idea that Newton was simply accepted everywhere when he published his book is popular science mythology. The field of History and Philosophy of Science interprets things differently and that’s why it’s worthwhile to read people like Kuhn.
Then how did it come about that Eratosthenes proved via experiment that the earth is round and estimated its radius? (A little later than 300 BCE, I think, but in the right ballpark.)
The point of his action wasn’t to prove that it’s round but to estimate the radius.
Aristoteles already had a firm idea of the earth being round and most of the people afterwards believed that the earth was round because of the arguments of Aristoteles and not because of Eratosthenes.
The central dogma, at least as Crick stated it, says that the transcription from nucleotide sequence to amino acid sequence never goes the other way, with proteins getting decoded back into nucleic acids.
The dogma is that DNA get’s transcripted into RNA and that RNA get’s translated into proteins.
So far as I know, this is still a closed question. Was it ever thought absolutely certain that there’s no mechanism by which proteins can affect the information in nucleic acids?
Proteins aren’t the point. The issue are retroviruses who transcribe RNA into DNA and thus violate the dogma. Historically that makes the term “dogma” quite silly as a dogma is by definition closed.
They still are, and so far as I know they never claimed that all non-coding DNA is completely functionless.
Not all but most non-coding DNA. The argument for that was that bacteria with are subject to strong evolutionary pressures and have had a lot more generations to kick out junk have less DNA. If I remember right there’s also a fish who has significantly less DNA than it’s close relatives.
You have biologists making the public think that all human DNA got sequenced on the basis that the 8% that they haven’t sequenced doesn’t really count and is likely unimportant.
Today it’s much more an open question than it was 20 years ago.
Einstein wasn’t following the ideal to be open about him playing dice
We may be at cross purposes. I am not claiming that all scientists, always, have had functionally-open minds about everything they should have had open minds about; I am claiming that science has always espoused in principle the ideal that minds should be open, and that its espousal of this principle leads scientists to have more-open minds than they would have if that principle weren’t around. (I will also claim that this is part of why science works as well as, in practice, it appears to work.)
However: I’m not sure Einstein’s mind was so closed on this point. For sure, he had a strong opinion, but there’s nothing wrong with that. I don’t think the evidence available to Einstein was strong and clear enough that he should have decided that God plays dice after all. And he certainly engaged with the ideas of QM closely enough, e.g., to be one of the originators of the EPR “paradox”, which depends on a careful analysis of what QM predicts in some situations.
It’s not even clear that QM requires that God play dice. For instance, the Everett (“many worlds”) interpretation, popular both here on LW and among physicists, is entirely deterministic. Observers see every possible series of outcomes, and it just turns out that “most” (in the relevant sense) series of outcomes look random.
On the continent the expedition was important.
Yup. In England, Newton was more or less just assumed to be right, whereas in continental Europe it was controversial how right he was. What (so far as I can see) there wasn’t, anywhere, was a consensus that Newton was wrong, and that’s what you appeared to be claiming there was.
The idea that Newton was simply accepted everywhere when he published his book is [...]
… Is one that I never stated nor implied. What I said is that it was very quickly recognized as first-rank science and as something that shouldn’t simply be dismissed; are you claiming otherwise?
Aristoteles already had a firm idea of the earth being round
On the basis of well known repeated observations, which are in fact almost as good a source of information as controlled experiments (you need the experiments when the observations are misleading or insufficient). If Eratosthenes had done his measurement and found “r=infinity”, do you think no one among the ancient Greeks would have considered the possibility that the earth is flat after all and the observations that suggest otherwise are illusions?
The dogma is that DNA gets transcribed into RNA and that RNA gets translated into proteins.
Here is what Francis Crick actually wrote:
The central dogma of molecular biology deals with the detailed residue-by-residue transfer of sequential information. It states that such information cannot be transferred back from protein to either protein or nucleic acid.
This doesn’t say anything about reverse transcription from RNA to DNA being impossible.
It’s true that sometimes the term “central dogma” was applied to something broader (claiming that DNA->RNA is also one-way) but I question whether that was ever treated as a closed question.
Take a look at this draft of Crick’s paper from 1956 in which the Central Dogma is first named. Its very first diagram shows a dotted arrow from RNA back to DNA and says it’s a possibility.
that makes the term “dogma” quite silly as a dogma is by definition closed.
Yeah, Crick said, more than once and in so many words, that “dogma” was a bad choice of term and that when he came up with the name he didn’t really know quite what “dogma” means. (See, e.g., the last two quotations on the Wikipedia page about the Central Dogma.)
Today it’s much more an open question than it was 20 years ago.
That may well be true; that is, current estimates for the probability distribution of what fraction of DNA is how useful are more “optimistic” than the estimates of 20 years ago. But you seem to be suggesting that that’s indicative of some kind of problematic closed-mindedness among scientists, but to me all it suggests is that scientists aren’t always immediately correct about how likely various possibilities are. Which is hardly a surprise and hardly indicates anything very bad.
In passing, he gestured vaguely at a vague conception of science. I guess that doesn’t qualify as an argument, so perhaps there is no argument to steelman. But I think that the vague conception of science he was trying to gesture toward does correspond to a real thing that scientists sometimes do.
In the map-territory analogy, this might correspond to a fuzzy or blank region of the map. A scientifically minded person might well say “One reasonable hypothesis is that the Earth is flat the blank region looks like nearby regions, but until we have tools and techniques that can be used to prove or disprove that hypothesis, it is an open question.”
But here’s the idea I think the author was trying to gesture at. In my experience, most people are way too eager to try and solve problems they don’t fully understand. I’ve often heard scientists and engineers caution against this, but the most notable quote is from the rocket scientist Wernher Von Braun: “One good test is worth a thousand expert opinions”. I’ve seen people like Bill Nye repeat this, and seen plenty of science-themed reminders that test results are often surprising, since the world is often much more complex that we give it credit for.
As for the historical commentary, I completely agree. The scenario isn’t historically plausible. The scientific revolution would have had to happen earlier just to produce someone capable of saying the quote, and society would have had to somehow go through a scientific revolution without noticing that the earth was round.
The world is indeed often surprising but frequently surprising in a way that scientific experiments open new questions that weren’t in the mind of the scientists beforehand.
When looking at something like Reiki, Bill Nye and friends argue don’t consider it an open question whether or not Reiki works just because we don’t have well controlled studies investigating it. They consider it not to work because they don’t believe that there’s ki.
You might argue that they are wrong to do so, but that’s still how they operate.
I’ve seen people like Bill Nye repeat this, and seen plenty of science-themed reminders that test results are often surprising
The phrase science-themed sounds to me more like it refers to science mythology than serious history of science.
To the extend that you want to sensible talk about what scientists do, you have to listen to people who study what scientists do and that’s not the speciality of a rocket scientist like Wernher Von Braun.
Ah, that’s the definition about which we were talking past each other. I certainly wouldn’t say that “Reiki might work, and until we test it we just don’t know!” Perhaps it “works” somewhat through the placebo effect, but even in the unlikely event of a study showing some random placebo controlled health benefit, it would still be astronomically unlikely that ki was the mechanism. (That’s not to say that no one will look at the real mechanism after the fact, and try to pick out some superficial similarity to the idea of “ki”.)
But that’s beside the point. For hypotheses that are worth our time to test, we test them precisely because it’s an open question. Until we take the data, it remains an open question. (at least for certain definitions of “open question”) I think that’s the point the author was trying to get at with his infeasible historical example.
If all the available evidence was well explainable with a flat world then the argument that things could also be explained by a very large ball likely wouldn’t convince people to consider it an open question.
Occam’s razor suggests that you go with the most simple theory and not consider more complex theories simply because they fit the data if there are not additional reasons in their favor.
What happen to be open questions depends a lot on the standards of a given academic community.
By the standards of mathematics P=NP is an open question. By the standards of biology a similar question where all available evidence points in one direction would be considered closed.
A year ago biology as taught via textbooks didn’t consider the question whether or not the lymphatic system extends into the brain to be an open question. The textbooks were clear about there not being a lymphatic system extension into the brain. Now someone found that it extends into the brain and we have to change the textbooks.
Furthermore when the paradigm of a field changes it frequently happens that open questions of the old field get forgotten even through they are not answered.
Yeah, but if you steel-man it, I think he was trying to make something similar to a map-territory distinction. It’s often useful to make a distinction between the data and our best interpretation of the data. Some conclusions don’t require much extrapolation, but others require a great deal.
On LW we happily discuss with very long inferential distances, and talk about regions of hypothesis space with high densities of unknown unknowns. Most scientists, however, work over much smaller inferential distances, with the intent of meticulously build up a rock solid body of knowledge. If things are “open questions” until they are above a confidence interval of, say, 0.99, then just about everything we discuss here is an open question, as the quote suggests.
Using a historical example which happens to be false just complicates things. If I recall, philosophers first hypothesized a round earth around 600 BCE, but didn’t prove it experimentally until 300 BCE.
The act of steelmanning means to argue against a different position then the one the person is holding. It very worthwhile to criticize people for holding positions for the wrong reasons.
To me what you are saying doesn’t seem like a description of the map-territory distinction. A map is not an extrapolation of the territory but an abstraction of it.
That sentence doesn’t look to me like it’s inspired by looking at what scientists do. I’m not aware of a scientific community having the standard of question being closed when they are over a confidence interval of 0.99.
You might argue that scientists should do things that way, but that doesn’t have much to do with the question of how scientists act in the real world.
Statements about what’s useful are different than statements that describe what scientists do in reality.
I think you missed a point. At 300 BCE they were not centrally concerned with proving via experiment that the earths is round. They instead cared about things making sense intuitively. The idea that it’s important to prove claims via experiment came with Descartes into the scientific mosaic which happened much later.
Nobody at the time between 600 BCE and 300 BCE said: “The part of the Earth that I can see at any given time appears to be flat, which would be the case when looking at a small piece of many differently shaped objects up close, so I don’t have enough information to know what the shape of the Earth is. One reasonable hypothesis is that the Earth is flat, but until we have tools and techniques that can be used to prove or disprove that hypothesis, it is an open question.”
“How does action at a distance work?” wasn’t an open question shortly after Descartes. It became again an open question when Newton was shown to be right by the expedition that measured the shape of the earth.
In biology the central dogma of molecular biology was considered a close question for a long time. Biologists where confident about the fact that a lot of the DNA is junk DNA that doesn’t do anything.
One of the main reasons why we don’t consider the question of whether homeopathy works an open question isn’t just that we lack empiric evidence for it working but that we based on our theories of chemistry we don’t believe that it could work.
Whether or not chiropratics interventions work was a question that scientists considered not to be open for a long time.
I think you’re right that the OP’s characterization of science is naive (though I think the everything-is-open idea is always there as an ideal, and I think that really matters), but all your examples here seem really dubious.
Then how did it come about that Eratosthenes proved via experiment that the earth is round and estimated its radius? (A little later than 300 BCE, I think, but in the right ballpark.)
The idea that observation trumps theory was not unknown to the ancient Greeks.
Newton’s work was recognized as first-rank science from the beginning, so the idea of action at a distance was taken seriously (at least) from the publication of the Principia. I don’t know what expedition you are thinking of, but I am pretty sure it’s nowhere near the truth to say that action at a distance was thought to be rubbish until an expedition measured the shape of the earth and thereby proved that Newton was right after all.
[EDITED to add: OK, so I guess you mean the French Geodesic Mission of 1735, which found that the earth is bulgy in the middle (as Newton said it should be) rather than elongated towards the poles (as Descartes claimed, on the basis of what we would now regard as a crackpot theory of vortices). That’s hardly conclusive evidence for action at a distance, nor would the opposite result have been anything like conclusive the other way. But, anyway, a large part of the point of that expedition was that “Newton or Descartes?” was an open question. It would be nearer the truth to say that the expedition closed the question. But not much nearer, because as I say the question the survey resolved was not the question of action at a distance.]
The central dogma, at least as Crick stated it, says that the transcription from nucleotide sequence to amino acid sequence never goes the other way, with proteins getting decoded back into nucleic acids. So far as I know, this is still a closed question. Was it ever thought absolutely certain that there’s no mechanism by which proteins can affect the information in nucleic acids?
They still are, and so far as I know they never claimed that all non-coding DNA is completely functionless. The “biologists thought most DNA was pure junk but now they’ve been stunned to find that some of it is useful” narrative is, I think, mostly hype.
If you have Einstein saying: “God doesn’t play dice” Einstein wasn’t following the ideal to be open about him playing dice.
Newton was accepted much sooner in the UK than on the continent. On the continent the expedition was important.
The idea that Newton was simply accepted everywhere when he published his book is popular science mythology. The field of History and Philosophy of Science interprets things differently and that’s why it’s worthwhile to read people like Kuhn.
The French Geodesic Mission.
The point of his action wasn’t to prove that it’s round but to estimate the radius. Aristoteles already had a firm idea of the earth being round and most of the people afterwards believed that the earth was round because of the arguments of Aristoteles and not because of Eratosthenes.
The dogma is that DNA get’s transcripted into RNA and that RNA get’s translated into proteins.
Proteins aren’t the point. The issue are retroviruses who transcribe RNA into DNA and thus violate the dogma. Historically that makes the term “dogma” quite silly as a dogma is by definition closed.
Not all but most non-coding DNA. The argument for that was that bacteria with are subject to strong evolutionary pressures and have had a lot more generations to kick out junk have less DNA. If I remember right there’s also a fish who has significantly less DNA than it’s close relatives.
You have biologists making the public think that all human DNA got sequenced on the basis that the 8% that they haven’t sequenced doesn’t really count and is likely unimportant.
Today it’s much more an open question than it was 20 years ago.
We may be at cross purposes. I am not claiming that all scientists, always, have had functionally-open minds about everything they should have had open minds about; I am claiming that science has always espoused in principle the ideal that minds should be open, and that its espousal of this principle leads scientists to have more-open minds than they would have if that principle weren’t around. (I will also claim that this is part of why science works as well as, in practice, it appears to work.)
However: I’m not sure Einstein’s mind was so closed on this point. For sure, he had a strong opinion, but there’s nothing wrong with that. I don’t think the evidence available to Einstein was strong and clear enough that he should have decided that God plays dice after all. And he certainly engaged with the ideas of QM closely enough, e.g., to be one of the originators of the EPR “paradox”, which depends on a careful analysis of what QM predicts in some situations.
It’s not even clear that QM requires that God play dice. For instance, the Everett (“many worlds”) interpretation, popular both here on LW and among physicists, is entirely deterministic. Observers see every possible series of outcomes, and it just turns out that “most” (in the relevant sense) series of outcomes look random.
Yup. In England, Newton was more or less just assumed to be right, whereas in continental Europe it was controversial how right he was. What (so far as I can see) there wasn’t, anywhere, was a consensus that Newton was wrong, and that’s what you appeared to be claiming there was.
… Is one that I never stated nor implied. What I said is that it was very quickly recognized as first-rank science and as something that shouldn’t simply be dismissed; are you claiming otherwise?
On the basis of well known repeated observations, which are in fact almost as good a source of information as controlled experiments (you need the experiments when the observations are misleading or insufficient). If Eratosthenes had done his measurement and found “r=infinity”, do you think no one among the ancient Greeks would have considered the possibility that the earth is flat after all and the observations that suggest otherwise are illusions?
Here is what Francis Crick actually wrote:
This doesn’t say anything about reverse transcription from RNA to DNA being impossible.
It’s true that sometimes the term “central dogma” was applied to something broader (claiming that DNA->RNA is also one-way) but I question whether that was ever treated as a closed question.
Take a look at this draft of Crick’s paper from 1956 in which the Central Dogma is first named. Its very first diagram shows a dotted arrow from RNA back to DNA and says it’s a possibility.
Yeah, Crick said, more than once and in so many words, that “dogma” was a bad choice of term and that when he came up with the name he didn’t really know quite what “dogma” means. (See, e.g., the last two quotations on the Wikipedia page about the Central Dogma.)
That may well be true; that is, current estimates for the probability distribution of what fraction of DNA is how useful are more “optimistic” than the estimates of 20 years ago. But you seem to be suggesting that that’s indicative of some kind of problematic closed-mindedness among scientists, but to me all it suggests is that scientists aren’t always immediately correct about how likely various possibilities are. Which is hardly a surprise and hardly indicates anything very bad.
In passing, he gestured vaguely at a vague conception of science. I guess that doesn’t qualify as an argument, so perhaps there is no argument to steelman. But I think that the vague conception of science he was trying to gesture toward does correspond to a real thing that scientists sometimes do.
In the map-territory analogy, this might correspond to a fuzzy or blank region of the map. A scientifically minded person might well say “One reasonable hypothesis is that the Earth is flat the blank region looks like nearby regions, but until we have tools and techniques that can be used to prove or disprove that hypothesis, it is an open question.”
But here’s the idea I think the author was trying to gesture at. In my experience, most people are way too eager to try and solve problems they don’t fully understand. I’ve often heard scientists and engineers caution against this, but the most notable quote is from the rocket scientist Wernher Von Braun: “One good test is worth a thousand expert opinions”. I’ve seen people like Bill Nye repeat this, and seen plenty of science-themed reminders that test results are often surprising, since the world is often much more complex that we give it credit for.
As for the historical commentary, I completely agree. The scenario isn’t historically plausible. The scientific revolution would have had to happen earlier just to produce someone capable of saying the quote, and society would have had to somehow go through a scientific revolution without noticing that the earth was round.
The world is indeed often surprising but frequently surprising in a way that scientific experiments open new questions that weren’t in the mind of the scientists beforehand.
When looking at something like Reiki, Bill Nye and friends argue don’t consider it an open question whether or not Reiki works just because we don’t have well controlled studies investigating it. They consider it not to work because they don’t believe that there’s ki.
You might argue that they are wrong to do so, but that’s still how they operate.
The phrase science-themed sounds to me more like it refers to science mythology than serious history of science. To the extend that you want to sensible talk about what scientists do, you have to listen to people who study what scientists do and that’s not the speciality of a rocket scientist like Wernher Von Braun.
Ah, that’s the definition about which we were talking past each other. I certainly wouldn’t say that “Reiki might work, and until we test it we just don’t know!” Perhaps it “works” somewhat through the placebo effect, but even in the unlikely event of a study showing some random placebo controlled health benefit, it would still be astronomically unlikely that ki was the mechanism. (That’s not to say that no one will look at the real mechanism after the fact, and try to pick out some superficial similarity to the idea of “ki”.)
But that’s beside the point. For hypotheses that are worth our time to test, we test them precisely because it’s an open question. Until we take the data, it remains an open question. (at least for certain definitions of “open question”) I think that’s the point the author was trying to get at with his infeasible historical example.
If all the available evidence was well explainable with a flat world then the argument that things could also be explained by a very large ball likely wouldn’t convince people to consider it an open question. Occam’s razor suggests that you go with the most simple theory and not consider more complex theories simply because they fit the data if there are not additional reasons in their favor.
What happen to be open questions depends a lot on the standards of a given academic community. By the standards of mathematics P=NP is an open question. By the standards of biology a similar question where all available evidence points in one direction would be considered closed.
A year ago biology as taught via textbooks didn’t consider the question whether or not the lymphatic system extends into the brain to be an open question. The textbooks were clear about there not being a lymphatic system extension into the brain. Now someone found that it extends into the brain and we have to change the textbooks.
Furthermore when the paradigm of a field changes it frequently happens that open questions of the old field get forgotten even through they are not answered.