I don’t think that I’m more rational than Bohr, Heisenberg, Dirac, Feynman, Penrose, Schrödinger and Wigner. These people thought about quantum mechanics for decades. I wouldn’t be able to catch up in a week.
We should separate “rationality” from “domain knowledge of quantum physics.”
Certainly, each of these figures had greater domain knowledge of quantum physics than I plan to acquire in my life. But that doesn’t mean I’m powerless to (in some cases) tell when they’re just wrong. The same goes for those with immense domain knowledge of – to take an unusually clear-cut example – Muslim theology.
Consider a case where you talk to a top Muslim philosopher and extract his top 10 reasons for believing in Allah. His arguments have obvious crippling failures, so you try to steel man them, and you check whether you’re just misinterpreting the arguments, but in the end they’re just… terrible arguments. And then you check with lots of other leading Muslim philosophers and say “Give me your best reasons” and you just get nothing good. At that point, I think you’re in a pretty good position to reject the Muslim belief in Allah, rather than saying “Well, maybe there’s some subtle point they know about that I haven’t gotten to in the first 40 hours of investigation. They are the experts on Allah, so I’m probably not in a good position to confidently disagree with them.”
The case with QM is obviously different in many ways, but the same point that it’s possible to (rationally) confidently disagree with the plurality or majority of experts on something still stands, I think.
One of the biggest differences between Muslim theology and QM is that the QM experts seem to have much better rationality than the Muslim theologians, so it’s not as easy to conclude that you’re evaluating the evidence and arguments more appropriately than they are.
And this brings us to what might be a significant difference between us. I tend to think it’s quite feasible to surpass the rationality skill of (most) top-performing physicists. Perhaps you think that is not that feasible.
Let me make another analogy. Consider the skill of nonfiction writing. Rousseau and Hume were great for their time, but Pinker and Dawkins write better because they’ve got enormous advantages: they get to study Rousseau and Hume, they get to study later writers, they have instant access to online writing-helpers like Thesaurus.com and reading-level measures, they get to benefit from hundreds of years of scientific study of psychology and communication, they get to email drafts to readers around the world for feedback, etc.
Similarly, I think it’s quite feasible to outperform (in rationality) most of the top-performing physicists. I’m not sure that I have reached that level yet, but I think I certainly could.
Here, the enormous advantages available to me don’t come from the fact that I have access to things that the scientist’s don’t have access to, but instead from the fact that the world’s best physicists mostly won’t choose to take advantage of the available resources – which is part of why they so often say silly things as soon as they step outside the lab.
Many qualifications, and specific examples, could be given, but I’ll pause here and give you a chance to respond.
I feel the main disanalogy with the Muslim theology case is that elite common sense does not regard top Muslim philosophers as having much comparative expertise on the question of whether Allah exists, but they would regard top physicists as having very strong comparative expertise on the interpretation of quantum mechanics. By this I mean that elite common sense would generally substantially defer to the opinions of the physicists but not the Muslim philosophers. This disanalogy is sufficiently important to me that I find the overall argument by analogy highly non-compelling.
I note that there are some meaningful respects in which elite common sense would regard the Muslim philosophers as epistemic authorities. They would recognize their authority as people who know about what the famous arguments for Allah’s existence and nature are, what the famous objections and replies are, and what has been said about intricacies of related metaphysical questions, for example.
The big difference is that the atheist arguments are not reliant upon Muslim philosophy in the way in which all possible MWI arguments rely on physics. The relevant questions are subtle aspects of physics—linearity, quantum gravity, etc. related, there are unresolved issues in physics to which many potential solutions are MWI unfriendly—precisely in the ballpark of your “quantum gravity vs. string theory” where you do not even know how two terms relate to each other, nor know how advanced physics relates to MWI.
1.3 billion Muslims—including many national leaders, scientists, and other impressive people throughout the Islamic world—believe that Mohammad was an authority on whether Allah exists.
Of course, there are plenty of others who do not, so I’m not suggesting we conclude it’s probable that he was, but it seems like something we couldn’t rule out, either.
Religion is a set of memes, not in the Internet sense of “catchy saying’ but in the slightly older sense that it has characteristics which lead to it spreading. I suggest modifying the original proposal: in deciding who is trustworthy to many people, you should take into account that beliefs which are good at spreading for reasons unrelated to their truth value can skew the “many people” part.
Given what we know about how religions spread, religious beliefs should be excluded on these grounds alone. If scientists who expressed a particular belief about science were killed, that would apply to scientists too (the fact that most trusted scientists in Stalinist Russia believed in Lysenkoism, since the others were dead or silenced, would not be a reason for you to do so).
So is culture. Are you ready to demand culture-neutrality?
If scientists who expressed a particular belief about science were killed, that would apply to scientists too
Would that apply also to scientists who were prevented from getting grants and being published? How do you know, without hindsight, which of the two warring scientific factions consists of cranks and crooks, and which one does not?
So is culture. Are you ready to demand culture-neutrality?
It is possible for a culture to at least not be inimical to truth. But to the extent that a religion is not inimical to truth, it has ceased to be a religion.
Would that apply also to scientists who were prevented from getting grants and being published?
If the main reason why scientists don’t express a belief is that if they do they would be arbitrarily denied grants and publication, then it would apply. However, in the modern Western world, almost every example* where someone made this claim has turned out to be a crank whose lack of publication was for very good reasons. As such, my default assumption will be that this has not occurred unless I have a specific reason to believe that it has.
I can think of a few cases involving politically correct subjects but those tend to have other kinds of problems.
We should separate “rationality” from “domain knowledge of quantum physics.”
If by rationality we mean more accurate and correct reasoning… it so happens that in the case of existence of many worlds, any rational conclusion is highly dependent on the intricacies of physics (specifically related to the quantum field theory and various attempts at quantum gravity), and consequently, conclusions being reached without such knowledge is a sign of very poor rationality just as equation of a+b*7+c = 0 having been simplified to a=1 without knowing anything about the values of b and c is a sign of very poor knowledge of mathematics, not of some superior equation simplification skill.
(The existence of Allah, God, Santa Claus, or what ever is, of course, not likewise dependent on theology).
My impression was that the conclusion in fact just depends on one’s interpretation of Occam’s razor, rather than the intricacies of physics. I had allowed myself to reach a fairly tentative conclusion about Many Worlds because physicists seem to agree that both are equally consistent with the data. We are then left with the question of what ‘the simplest explanation’ means, which is an epistemology question, not a physics question, and one I feel comfortable answering.
Yes, you are (mistaken). As numerous PhD physicists have been saying numerous times on this site and elsewhere, the issue is that QM is not consistent with observations (does not include gravity). Neither is QFT.
The question is one of fragility of MWI over potential TOEs, and, relying on exact linearity it is arguably very fragile.
Furthermore with regards to Occam’s razor and specifically formalizations of it, that is also a subtle question requiring domain specific training.
In particular, in Solomonoff Induction, codes have to produce output that exactly matches the observations. Complete with the exact picture of photon noise on the double slit experiment’s screen.
Outputting a list of worlds like MWI does is not even an option.
Interesting. I’ll assume an agnostic position again for the time being.
Can you point me towards some of the best comments?
I was aware that both theories are inconsistent with data with respect to gravity, obviously if either of them weren’t, the choice would be clear.
What do you mean by ‘fragility over’ potential theories of everything? That the TOEs suggested thus far tend not to be compatible with it? Presumably not given that the people generating the TOEs are likely to start with the most popular theory.
Whats the standard response by MW enthusiasts to your point on Solomonof induction? My understanding would then suggest that neither MW nor Copenhagen can give an exact picture of photon noise, in which case the problem would seem to be with Solomonoff induction as a formalization.
Can you point me towards some of the best comments?
There’s some around this thread (responses to Luke’s comment). Also I think that QM sequence has responses from physicists.
What do you mean by ‘fragility over’ potential theories of everything?
The MWI is concluded from exactly linear quantum mechanics. Because we know QM to be only an approximation, we lack any strong reasons to expect exact linearity in the final TOE. Furthermore even though exact linearity is arguably favoured by the Occam’s razor over any purely speculative non-linear theory, that does not imply that it is more probable than all of the nonlinear theories together (which would have same linear approximation).
In my opinion, things like multitude of potential worlds allow for e.g. elegantly (and compactly) expressing some conservation laws as survivor bias (via some sort of instability destroying observers in the world where said laws do not hold). Whenever that is significant to TOEs is, of course, purely speculative.
Whats the standard response by MW enthusiasts to your point on Solomonof induction?
As far as I know, the arguments that Solomonoff induction supports MWI never progressed beyond mere allusions to such support.
My understanding would then suggest that neither MW nor Copenhagen can give an exact picture of photon noise
In raw form, yes, neither interpretation fits and it’s unclear how to compare complexities of them formally.
in which case the problem would seem to be with Solomonoff induction as a formalization.
I explored some on how S.I. would work on data from quantum experiments here. Basically, the task is to represent said photon noise with the minimum amount of code and data, which can be done in two steps by calculating probabilities as per QM and Born rule, and using the probability density function to decode photon coordinates from the subsequent input bits. (analogous to collapse), or perhaps more compactly in one step by doing QM with some sort of very clever bit manipulation on strings of random noise as to obtain desired probability distribution in the end.
Here, the enormous advantages available to me don’t come from the fact that I have access to things that the scientist’s don’t have access to
You need to keep in mind that they have access to a very, very huge number of things that you don’t have access to (and have potential access to more things than you think they do), and can get the taste of the enormous space of issues any of which can demolish the case for MWI that you see completely, from the outside. For example, non-linearity of the equations absolutely kills MWI. Now, only a subset of physicists considers the non-linearity to be likely. Others believe in other such things, most of which just kill MWI outright, with all the argument for it flying apart like a card house in the wind of a category 6 hurricane. Expecting a hurricane, experts choose not to bother with the question of mechanical stability of the card house on it’s own—they would rather debate in which direction, and how far, the cards will land.
Consider a case where you talk to a top Muslim philosopher and extract his top 10 reasons for believing in Allah.
Consider a case where you talk to a top flat-earth believer and extract his top 10 reasons for believing the earth is flat.
The act of selecting a Muslim philosopher confounds the accuracy of his belief with the method whereby you selected him. It’s like companies searching out a scientist who happens to take a view of some question congenial to that company, then booming his research.
Thanks for the thoughtful response. I agree that there are many instances in which it’s possible to rationally come to confident conclusions that differ from those of subject matter experts. I realize that my earlier comment was elliptical, and will try to clarify. The relevant points to my mind are:
The extraordinary intellectual caliber of the best physicists
Though difficult to formalize, I think that there’s a meaningful sense in which one can make statements of the general form “person A has intellectual caliber n times that of person B.” Of course, this is domain specific to some degree, but I think that the concept hangs together somewhat even across domains.
One operationalization of this is “if person B reaches a correct conclusion on a given subject, person A could reach it n times as fast.” Another is “it would take n copies of person B to do person A’s work.” These things are hard to estimate, but one can become better calibrated by using the rule “if person A has intellectual caliber n times that of person B and person B has intellectual caliber m times that of person C, then person A has intellectual caliber n*m times that of person C.”
In almost all domains, I think that the highest intellectual caliber people have no more than 5x my intellectual caliber. Physics is different. From what I’ve heard, the distribution of talent in physics is similar to that of math. The best mathematicians are 100x+ my intellectual caliber. I had a particularly striking illustrative experience with Don Zagier, who pinpointed a crucial weakness in an analogy that I had been exploring for 6 months (and which I had run by a number of other mathematicians) in a mere ~15 minutes. I would not be surprised if he himself were to have an analogous experience with the historical greats.
When someone is < 5x one’s intellectual caliber, an argument of the type “this person may be smarter than me, but I’ve focused a lot more on having accurate views, so I trust my judgment over him or her” seems reasonable. But when one gets to people who are 100x+ one’s intellectual caliber, the argument becomes much weaker. Model uncertainty starts to play a major role. It could be that people who are that much more powerful easily come to the correct conclusion on a given question without even needing to put conscious effort into having accurate beliefs.
The intrinsic interest of the question of interpretation of quantum mechanics
The question of what quantum mechanics means has been considered one of the universe’s great mysteries. As such, people interested in physics have been highly motivated to understand it. So I think that the question is privileged relative to other questions that physicists would have opinions on — it’s not an arbitrary question outside of the domain of their research accomplishments.
Solicitation of arguments from those with opposing views
In the Muslim theology example, you spend 40 hours engaging with the Muslim philosophers. This seems disanalogous to the present case, in that as far as I know, Eliezer’s quantum mechanics sequence hasn’t been vetted by any leading physicists who disagree with the many world’s interpretation of quantum mechanics. I also don’t know of any public record of ~40 hours of back and forth analogous to the one that you describe. I know that Eliezer might cite an example in his QM sequence, and will take a look.
The intrinsic interest of the question of interpretation of quantum mechanics
The question of what quantum mechanics means has been considered one of the universe’s great mysteries. As such, people interested in physics have been highly motivated to understand it. So I think that the question is privileged relative to other questions that physicists would have opinions on — it’s not an arbitrary question outside of the domain of their research accomplishments.
My understanding is that the interpretation of QM is (1) not regarded as a very central question in physics, being seen more as a “philosophy” question and being worked on to a reasonable extent by philosophers of physics and physicists who see it as a hobby horse, (2) is not something that physics expertise—having good physical intuition, strong math skills, detailed knowledge of how to apply QM on concrete problems—is as relevant for as many other questions physicists work on, and (3) is not something about which there is an extremely enormous amount to say. These are some of the main reasons I feel I can update at all from the expert distribution of physicists on this question. I would hardly update at all from physicist opinions on, say, quantum gravity vs. string theory, and I think it would basically be crazy for me to update substantially in one direction or the other if I had comparable experience on that question.
[ETA: As evidence of (1), I might point to the prevalence of the “shut up and calculate” mentality which seems to have been reasonably popular in physics for a while. I’d also point to the fact that Copenhagen is popular but really, really, really, really not good. And I feel that this last claim is not just Nick Beckstead’s idiosyncratic opinion, but the opinion of every philosopher of physics I have ever spoken with about this issue.]
I believe you are using bad terminology. ‘Quantum gravity’ refers to any attempt to reconcile quantum mechanics and general relativity, and string theory is one such theory (as well as a theory of everything). Perhaps you are referring to loop quantum gravity, or more broadly, to any theory of quantum gravity other than string theory?
Perhaps I should have meant loop quantum gravity. I confess that I am speaking beyond my depth, and was just trying to give an example of a central dispute in current theoretical physics. That is the type of case where I would not like to lean heavily on my own perspective.
My understanding is that the interpretation of QM is (1) not regarded as a very central question in physics, being seen more as a “philosophy” question and being worked on to a reasonable extent by philosophers of physics and physicists who see it as a hobby horse,
I agree if we’re talking about the median theoretical physicist at a top 5 school, but when one gets further toward the top of the hierarchy, one starts to see a high density of people who are all-around intellectually curious and who explore natural questions that they come across independently of whether they’re part of their official research.
(2) is not something that physics expertise—having good physical intuition, strong math skills, detailed knowledge of how to apply QM on concrete problems—is as relevant for as many other questions physicists work on
I agree, but a priori I suspect that philosophers of physics and others without heavy subject matter knowledge of quantum mechanics have leaned too heavily on this. Spending one’s life thinking about something can result subconscious acquisition of implicit knowledge of things that are obliquely related. People who haven’t had this experience may be at a disadvantage.
I would hardly update at all from physicist opinions on, say, quantum gravity vs. string theory, and I think it would basically be crazy for me to update substantially in one direction or the other if I had comparable experience on that question.
I actually think that it’s possible for somebody without subject matter knowledge to rationally develop priors that are substantially different from expert consensus here. One can do this by consulting physicists who visibly have high epistemic rationality outside of physics, by examining sociological factors that may have led to the status quo, and by watching physicists who disagree debate each other and see which of the points they respond to and which ones they don’t.
[ETA: As evidence of (1), I might point to the prevalence of the “shut up and calculate” mentality which seems to have been reasonably popular in physics for a while. I’d also point to the fact that Copenhagen is popular but really, really, really, really not good. And I feel that this last claim is not just Nick Beckstead’s idiosyncratic opinion, but the opinion of every philosopher of physics I have ever spoken with about this issue.]
I agree, but a priori I suspect that philosophers of physics and others without heavy subject matter knowledge of quantum mechanics have leaned too heavily on this. Spending one’s life thinking about something can result subconscious acquisition of implicit knowledge of things that are obliquely related. People who haven’t had this experience may be at a disadvantage.
But note that philosophers of physics sometimes make whole careers thinking about this, and they are among the most high-caliber philosophers. They may be at an advantage in terms of this criterion.
I can’t think of a reference in print for my claim about what almost all philosophers think. I think a lot of them would find it too obvious to say, and wouldn’t bother to write a paper about it. But, for what it’s worth, I attended a couple of conferences on philosophy of physics held at Rutgers, with many leading people in the field, and talked about this question and never heard anyone express an opposing opinion. And I was taught about interpretations of QM from some leading people in philosophy of physics.
What I’m anchoring on here is the situation in the field of philosophy of math, where lack of experience with the practice of math seriously undercuts most philosophers’ ability to do it well. There are exceptions, for example I consider Imre Lakatos to be one. Maybe the situation is different in philosophy of physics.
I agree, but a priori I suspect that philosophers of physics and others without heavy subject matter knowledge of quantum mechanics have leaned too heavily on this. Spending one’s life thinking about something can result subconscious acquisition of implicit knowledge of things that are obliquely related. People who haven’t had this experience may be at a disadvantage.
But note that philosophers of physics sometimes make whole careers thinking about this, and they are among the most high-caliber philosophers. They may be at an advantage in terms of this criterion.
I can’t think of a reference in print for my claim about what almost all philosophers think. I think a lot of them would find it too obvious to say, and wouldn’t bother to write a paper about it. But, for what it’s worth, I attended a couple of conferences on philosophy of physics held at Rutgers, with many leading people in the field, and talked about this question and never heard anyone express an opposing opinion. And I was taught about interpretations of QM from some leading people in philosophy of physics.
Let me first say that I find this to be an extremely interesting discussion.
In almost all domains, I think that the highest intellectual caliber people have no more than 5x my intellectual caliber. Physics is different. From what I’ve heard, the distribution of talent in physics is similar to that of math. The best mathematicians are 100x+ my intellectual caliber.
I think there is a social norm in mathematics and physics that requires people to say this, but I have serious doubts about whether it is true. Anyone 100x+ your intellectual caliber should be having much, much more impact on the world (to say nothing of mathematics itself) than any of the best mathematicians seem to be having. At the very least, if there really are people of that cognitive level running around, then the rest of the world is doing an absolutely terrible job of extracting information and value from them, and they themselves must not care too much about this fact.
More plausible to me is the hypothesis that the best mathematicians are within the same 5x limit as everyone else, and that you overestimate the difficulty of performing at their level due to cultural factors which discourage systematic study of how to imitate them.
Try this thought experiment: suppose you were a graduate student in mathematics, and went to your advisor and said: “I’d like to solve [Famous Problem X], and to start, I’m going to spend two years closely examining the work of Newton, Gauss, and Wiles, and their contemporaries, to try to discern at a higher level of generality what the cognitive stumbling blocks to solving previous problems were, and how they overcame them, and distill these meta-level insights into a meta-level technique of my own which I’ll then apply to [Famous Problem X].” What do you think the reaction would be? How many times do you think such a thing has ever been proposed, let alone attempted, by a serious student or (even) senior researcher?
Try this thought experiment: suppose you were a graduate student in mathematics, and went to your advisor and said: “I’d like to solve [Famous Problem X], and to start, I’m going to spend two years closely examining the work of Newton, Gauss, and Wiles, and their contemporaries, to try to discern at a higher level of generality what the cognitive stumbling blocks to solving previous problems were, and how they overcame them, and distill these meta-level insights into a meta-level technique of my own which I’ll then apply to [Famous Problem X].”
This is a terrible idea unless they’re spending half their time pushing their limits on object-level math problems. I just don’t think it works to try to do a meta phase before an object phase unless the process is very, very well-understood and tested already.
I’m sure that’s exactly what the advisor would say (if they bother to give a reasoned reply at all), with the result that nobody ever tries this.
(I’ll also note that it’s somewhat odd to hear this response from someone whose entire mission in life is essentially to go meta on all of humanity’s problems...)
But let me address the point, so as not to be logically rude. The person would be pushing their limits on object-level math problems in the course of “examining the work of Newton, Gauss, and Wiles”, in order to understand said work; otherwise, it can hardly be said to constitute a meaningful examination. I also think it’s important not to confuse meta-ness with (nontechnical) “outside views”; indeed I suspect that a lot of the thought processes of mathematical “geniuses” consist of abstracting over classes of technical concepts that aren’t ordinarily abstracted over, and thus if expressed explicitly (which the geniuses may lack the patience to do) would simply look like another form of mathematics. (Others of their processes, I speculate, consist in obsessive exercising of visual/dynamic mental models of various abstractions.)
Switching back to logical rudeness, I’m not sure the meta-ness is your true rejection; I suspect what you may be really worried about is making sure there are tight feedback loops to which one’s reasoning can be subjected.
(I’ll also note that it’s somewhat odd to hear this response from someone whose entire mission in life is essentially to go meta on all of humanity’s problems...)
That’s not the kind of meta I mean. The dangerous form of meta is when you spend several years preparing to do X, supposedly becoming better at doing X, but not actually doing X, and then try to do X. E.g. college. Trying to improve at doing X while doing X is much, much wiser. I would similarly advise Effective Altruists who are not literally broke to be donating $10 every three months to something while they are trying to increase their incomes and invest in human capital; furthermore, they should not donate to the same thing two seasons in a row, so that they are also practicing the skill of repeatedly assessing which charity is most important.
“Meta” for these purposes is any daily activity which is unlike the daily activity you intend to do ‘later’.
Tight feedback loops are good, but not always available. This is a separate consideration from doing meta while doing object.
The activity of understanding someone else’s proofs may be unlike the activity of producing your own new math from scratch; this would be the problem.
I would similarly advise Effective Altruists who are not literally broke to be donating $10 every three months to something while they are trying to increase their incomes and invest in human capital; furthermore, they should not donate to the same thing two seasons in a row, so that they are also practicing the skill of repeatedly assessing which charity is most important.
This is excellent advice. I have put a note in my calendar thee months hence to reevaluate my small monthly donation.
At the very least, if there really are people of that cognitive level running around, then the rest of the world is doing an absolutely terrible job of extracting information and value from them, and they themselves must not care too much about this fact.
Yes, I think that this is what the situation is.
I’ll also say that I think that there are very few such people — maybe on the order of 10 who are alive today. With such a small absolute number, I don’t think that their observed impact on math is a lot lower than what one would expect a priori, and the prior in favor them having had a huge impact in society isn’t that strong.
More plausible to me is the hypothesis that the best mathematicians are within the same 5x limit as everyone else, and that you overestimate the difficulty of performing at their level due to cultural factors which discourage systematic study of how to imitate them.
“The best mathematicians are 100+x higher in intellectual caliber than I am” and “the difference is in large part due to cultural factors which discourage systematic study of how to imitate them” aren’t mutually exclusive. I’m sympathetic to your position.
What do you think the reaction would be?
To change the subject :-)
How many times do you think such a thing has ever been proposed, let alone attempted, by a serious student or (even) senior researcher?
Anyone 100x+ your intellectual caliber should be having much, much more impact on the world (to say nothing of mathematics itself) than any of the best mathematicians seem to be having.
How do you know how little intellectual caliber JonahSinick has?
The extraordinary intellectual caliber of the best physicists
That is of course exactly why I picked QM and MWI to make my case for nihil supernum. It wouldn’t serve to break a smart person’s trust in a sane world if I demonstrated the insanity of Muslim theologians or politicians; they would just say, “But surely we should still trust in elite physicists.” It is by demonstrating that trust in a sane world fails even at the strongest point which ‘elite common sense’ would expect to find, that I would hope to actually break someone’s emotional trust, and cause them to just give up.
I haven’t fully put together my thoughts on this, but it seems like a bad test to “break someone’s trust in a sane world” for a number of reasons:
this is a case where all the views are pretty much empirically indistinguishable, so it isn’t an area where physicists really care all that much
since the views are empirically indistinguishable, it is probably a low-stakes question, so the argument doesn’t transfer well to breaking our trust in a sane world in high-stakes cases; it makes sense to assume people would apply more rationality in cases where more rationality pays off
as I said in another comment, MWI seems like a case where physics expertise is not really what matters, so this doesn’t really show that the scientific method as applied by physicists is broken; it seems it at most it shows that physics aren’t good at questions that are essentially philosophical; it would be much more persuasive if you showed that e.g., quantum gravity was obviously better than string theory and only 18% of physicists working in the relevant area thought so
From my perspective, the main point is that if you’d expect AI elites to handle FAI competently, you would expect physics elites to handle MWI competently—the risk factors in the former case are even greater. Requires some philosophical reasoning? Check. Reality does not immediately call you out on being wrong? Check. The AI problem is harder than MWI and it has additional risk factors on top of that, like losing your chance at tenure if you decide that your research actually needs to slow down. Any elite incompetence beyond the demonstrated level in MWI doesn’t really matter much to me, since we’re already way under the ‘pass’ threshold for FAI.
I feel this doesn’t address the “low stakes” issues I brought up, or that this may not even by the physicists’ area of competence. Maybe you’d get a different outcome if the fate of the world depended on this issue, as you believe it does with AI.
I also wonder if this analysis leads to wrong historical predictions. E.g., why doesn’t this reasoning suggest that the US government would totally botch the constitution? That requires philosophical reasoning and reality doesn’t immediately call you out on being wrong. And the people setting things up don’t have incentives totally properly aligned. Setting up a decent system of government strikes me as more challenging than the MWI problem in many respects.
How much weight do you actually put on this line of argument? Would you change your mind about anything practical if you found out you were wrong about MWI?
I have an overall sense that there are a lot of governments that are pretty good and that people are getting better at setting up governments over time. The question is very vague and hard to answer, so I am not going to attempt a detailed one. Perhaps you could give it a shot if you’re interested.
I agree that if it were true that the consensus of elite physicists believed that MWI is wrong when there was a decisive case in favor of it, that would be striking. But
There doesn’t seem to be a consensus among elite physicists that MWI is wrong.
Paging through your QM sequence, it doesn’t look as though you’ve systematically addressed all objections that otherwise credible people have raised against MWI. For example, have you been through all of the references that critique MWI cited in this paper? I think given that most experts don’t view the matter as decided, and given the intellectual caliber of the experts, in order have 99+% confidence in this setting, one has to cover all of one’s bases.
One will generally find that correct controversial ideas convince some physicists. There are many physicists who believe MWI (though they perhaps cannot get away with advocating it as rudely as I do), there are physicists signed up for cryonics, there were physicists advocating for Drexler’s molecular nanotechnology before it was cool, and I strongly expect that some physicists who read econblogs have by now started advocating market monetarism (if not I would update against MM). A good new idea should have some physicists in favor of it, and if not it is a warning sign. (Though the endorsement of some physicists is not a proof, obviously many bad ideas can convince a few physicists too.) If I could not convince any physicists of my views on FAI, that would be a grave warning sign indeed. (I’m pretty sure some such already exist.) But that a majority of physicists do not yet believe in MWI does not say very much one way or another.
The cognitive elites do exist and some of them are physicists, therefore you should be able to convince some physicists. But the cognitive elites don’t correspond to a majority of MIT professors or anything like that, so you shouldn’t be able to convince a majority of that particular group. A world which knew what its own elite truthfinders looked like would be a very different world from this one.
Ok, putting aside MWI, maybe our positions are significantly more similar than it initially seemed. I agree with
A world which knew what its own elite truthfinders looked like would be a very different world from this one.
I’ve taken your comments such as
Depends on how crazy the domain experts are being, in this mad world of ours.
to carry connotations of the type “the fraction of people who exhibit high epistemic rationality outside of their immediate areas of expertise is vanishingly small.”
I think that there are thousands of people worldwide who exhibit very high epistemic rationality in most domains that they think about. I think that most of these people are invisible owing to the paucity of elites online. I agree that epistemic standards are generally very poor, and that high status academics generally do poorly outside of their immediate areas of expertise.
I think that there are thousands of people worldwide who exhibit very high epistemic rationality in most domains that they think about. I think that most of these people are invisible owing to the paucity of elites online.
Where does this impression come from? Are they people you’ve encountered personally? If so, what gave you the impression that they exhibited “very high epistemic rationality in most domains that they think about”?
To clarify, when I wrote “very high epistemic rationality” I didn’t mean “very accurate beliefs,” but rather “aware of what they know and what they don’t.” I also see the qualifier “most” as significant — I think that any given person has some marked blind spots. Of course, the boundary that I’m using is fuzzy, but the standard that I have in mind is something like “at the level of the 15 most epistemically rational members of the LW community.”
“Thousands” is at the “1 in a million” level, so in relative terms, my claim is pretty weak. If one disputes the claim, one needs a story explaining how the fraction could be so small. It doesn’t suffice to say “I haven’t personally seen almost any such people,” because there are so many people who one hasn’t seen, and the relevant people may be in unexpected places.
I’ve had the subjective impression that ~ 2% of those who I know outside of the LW/EA spheres fit this description. To be sure, there’s a selection effect, but I’ve had essentially no exposure to physics, business, finance or public policy, nor to people in very populous countries such as India and China. The people who I know who fit this description don’t seem to think that they’re extremely rare, which suggests that their experiences are similar to my own (though I recognize that “it’s a small world,” i.e. these people’s social circles may overlap in nonobvious ways).
Some of the people who GiveWell has had conversations with come across very favorably in the notes. (I recognize that I’m making a jump from “their area of specialty” to “most topics that they think about” Here I’m extrapolating from the people who I know who are very strong in their area of specialty.) I think that Carl’s comment about the Gates Foundation is relevant.
I updated in the direction of people being more rational than I had thought for reason that I gave at the end of my post Many weak arguments and the typical mind.
I don’t have high confidence here: maybe ~ 50% (i.e. I’m giving a median case estimate).
I should also clarify that I don’t think that one needs a silver bullet argument of the type “the people who you would expect to be most trustworthy have the wrong belief on something that they’ve thought about, with very high probability” to conclude with high confidence that epistemic standards are generally very low.
I think that there are many weak arguments that respected authorities are very often wrong.
Vladimir M has made arguments of the type “there’s fierce disagreement among experts at X about matters pertaining to X, so one knows that at least some of them are very wrong.” I think that string theory is a good case study. There are very smart people who strongly advocate for string theory as a promising road for theoretical physics research, and other very smart people who strongly believe that the research program is misguided. If nothing else, one can tell that a lot of the actors involved are very overconfident (even if one doesn’t know who they are).
There are very smart people who strongly advocate for string theory as a promising road for theoretical physics research, and other very smart people who strongly believe that the research program is misguided. If nothing else, one can tell that a lot of the actors involved are very overconfident (even if one doesn’t know who they are).
Or, alternatively, they disagree about who the research program is promising for.
who disagree with the many world’s interpretation of quantum mechanics
That’s too strong of a statement. If you exclude some diehards like Deutsch, the Bohmian school and maybe Penrose’s gravity-induced decoherence, the prevailing attitude is “however many worlds are out there, we can only ever see one”, so, until new evidence comes along, we can safely treat MWI as a single world.
In computer science an elite coder might take 6 months to finish a very hard task (e.g. create some kind of tricky OS kernel), but a poor coder will never complete the task. This makes the elite coder infinitely better than the poor coder. Furthermore the poor coder will ask many questions of other people, impacting their productivity. Thus an elite coder is transfinitely more efficient than a poor coder ;-)
One of the biggest differences between Muslim theology and QM is that the QM experts seem to have much better rationality than the Muslim theologians,
Another huge difference is that much of quantum mechanics is very technical physics. To get to the point where you can even have an opinion you need a fair amount of background information. When assessing expert opinion, you have a hugely difficult problem of trying to discern whether an expert physicist has relevant technical knowledge you do not have OR whether they are making a failure in rationality.
This of course is exactly what Muslim theologians would say about Muslim theology. And I’m perfectly happy to say, “Well, the physicists are right and Muslim theologians are wrong”, but that’s because I’m relying on my own judgment thereon.
The equivalent to asking Muslim theologians about Allah would be to ask many-worlds-believing quantum physicists about many-worlds.
The equivalent of asking quantum physicists about many-worlds would be to ask theologians about Allah, without specifically picking Muslim theologians. And if you ask theologians about Allah (by which I mean the Muslim conception of God—of course “Allah” is just the Arabic for “God”), you’re going to find that quite a few of them don’t think that Allah exists and that some other version of God does.
And that’s not even getting into the problems caused by the fact that religion is a meme that spreads in a way that skews the population of experts, which quantum mechanics doesn’t.
Similarly, I think it’s quite feasible to outperform (in rationality) most of the top-performing physicists. I’m not sure that I have reached that level yet, but I think I certainly could.
Definitely. Scientists in general and physicists in particular are probably no better than other professionals in instrumental rationality outside of their area of expertise, not sure about epistemic rationality. The “top-performing physicists” (what a strange name, physicists are not athletes), whoever they are, are probably not very much better, as you mention. I have seen some of them committing a number of standard cognitive fallacies.
In fact, I personally think that you are way more rational than many famous physicists, since you took pains to improve your rationality skills and became an expert in the area, and they did not.
However, what you have no hope of is to competently judge their results and beliefs about physics, except by relying on the opinions of other physicists and deciding whom to trust how much in case of a disagreement. But I guess we are in agreement here.
We should separate “rationality” from “domain knowledge of quantum physics.”
Certainly, each of these figures had greater domain knowledge of quantum physics than I plan to acquire in my life. But that doesn’t mean I’m powerless to (in some cases) tell when they’re just wrong. The same goes for those with immense domain knowledge of – to take an unusually clear-cut example – Muslim theology.
Consider a case where you talk to a top Muslim philosopher and extract his top 10 reasons for believing in Allah. His arguments have obvious crippling failures, so you try to steel man them, and you check whether you’re just misinterpreting the arguments, but in the end they’re just… terrible arguments. And then you check with lots of other leading Muslim philosophers and say “Give me your best reasons” and you just get nothing good. At that point, I think you’re in a pretty good position to reject the Muslim belief in Allah, rather than saying “Well, maybe there’s some subtle point they know about that I haven’t gotten to in the first 40 hours of investigation. They are the experts on Allah, so I’m probably not in a good position to confidently disagree with them.”
The case with QM is obviously different in many ways, but the same point that it’s possible to (rationally) confidently disagree with the plurality or majority of experts on something still stands, I think.
One of the biggest differences between Muslim theology and QM is that the QM experts seem to have much better rationality than the Muslim theologians, so it’s not as easy to conclude that you’re evaluating the evidence and arguments more appropriately than they are.
And this brings us to what might be a significant difference between us. I tend to think it’s quite feasible to surpass the rationality skill of (most) top-performing physicists. Perhaps you think that is not that feasible.
Let me make another analogy. Consider the skill of nonfiction writing. Rousseau and Hume were great for their time, but Pinker and Dawkins write better because they’ve got enormous advantages: they get to study Rousseau and Hume, they get to study later writers, they have instant access to online writing-helpers like Thesaurus.com and reading-level measures, they get to benefit from hundreds of years of scientific study of psychology and communication, they get to email drafts to readers around the world for feedback, etc.
Similarly, I think it’s quite feasible to outperform (in rationality) most of the top-performing physicists. I’m not sure that I have reached that level yet, but I think I certainly could.
Here, the enormous advantages available to me don’t come from the fact that I have access to things that the scientist’s don’t have access to, but instead from the fact that the world’s best physicists mostly won’t choose to take advantage of the available resources – which is part of why they so often say silly things as soon as they step outside the lab.
Many qualifications, and specific examples, could be given, but I’ll pause here and give you a chance to respond.
I feel the main disanalogy with the Muslim theology case is that elite common sense does not regard top Muslim philosophers as having much comparative expertise on the question of whether Allah exists, but they would regard top physicists as having very strong comparative expertise on the interpretation of quantum mechanics. By this I mean that elite common sense would generally substantially defer to the opinions of the physicists but not the Muslim philosophers. This disanalogy is sufficiently important to me that I find the overall argument by analogy highly non-compelling.
I note that there are some meaningful respects in which elite common sense would regard the Muslim philosophers as epistemic authorities. They would recognize their authority as people who know about what the famous arguments for Allah’s existence and nature are, what the famous objections and replies are, and what has been said about intricacies of related metaphysical questions, for example.
The big difference is that the atheist arguments are not reliant upon Muslim philosophy in the way in which all possible MWI arguments rely on physics. The relevant questions are subtle aspects of physics—linearity, quantum gravity, etc. related, there are unresolved issues in physics to which many potential solutions are MWI unfriendly—precisely in the ballpark of your “quantum gravity vs. string theory” where you do not even know how two terms relate to each other, nor know how advanced physics relates to MWI.
1.3 billion Muslims—including many national leaders, scientists, and other impressive people throughout the Islamic world—believe that Mohammad was an authority on whether Allah exists.
Of course, there are plenty of others who do not, so I’m not suggesting we conclude it’s probable that he was, but it seems like something we couldn’t rule out, either.
Religion is a set of memes, not in the Internet sense of “catchy saying’ but in the slightly older sense that it has characteristics which lead to it spreading. I suggest modifying the original proposal: in deciding who is trustworthy to many people, you should take into account that beliefs which are good at spreading for reasons unrelated to their truth value can skew the “many people” part.
Given what we know about how religions spread, religious beliefs should be excluded on these grounds alone. If scientists who expressed a particular belief about science were killed, that would apply to scientists too (the fact that most trusted scientists in Stalinist Russia believed in Lysenkoism, since the others were dead or silenced, would not be a reason for you to do so).
So is culture. Are you ready to demand culture-neutrality?
Would that apply also to scientists who were prevented from getting grants and being published? How do you know, without hindsight, which of the two warring scientific factions consists of cranks and crooks, and which one does not?
It is possible for a culture to at least not be inimical to truth. But to the extent that a religion is not inimical to truth, it has ceased to be a religion.
If the main reason why scientists don’t express a belief is that if they do they would be arbitrarily denied grants and publication, then it would apply. However, in the modern Western world, almost every example* where someone made this claim has turned out to be a crank whose lack of publication was for very good reasons. As such, my default assumption will be that this has not occurred unless I have a specific reason to believe that it has.
I can think of a few cases involving politically correct subjects but those tend to have other kinds of problems.
If by rationality we mean more accurate and correct reasoning… it so happens that in the case of existence of many worlds, any rational conclusion is highly dependent on the intricacies of physics (specifically related to the quantum field theory and various attempts at quantum gravity), and consequently, conclusions being reached without such knowledge is a sign of very poor rationality just as equation of a+b*7+c = 0 having been simplified to a=1 without knowing anything about the values of b and c is a sign of very poor knowledge of mathematics, not of some superior equation simplification skill.
(The existence of Allah, God, Santa Claus, or what ever is, of course, not likewise dependent on theology).
My impression was that the conclusion in fact just depends on one’s interpretation of Occam’s razor, rather than the intricacies of physics. I had allowed myself to reach a fairly tentative conclusion about Many Worlds because physicists seem to agree that both are equally consistent with the data. We are then left with the question of what ‘the simplest explanation’ means, which is an epistemology question, not a physics question, and one I feel comfortable answering.
Am I mistaken?
Yes, you are (mistaken). As numerous PhD physicists have been saying numerous times on this site and elsewhere, the issue is that QM is not consistent with observations (does not include gravity). Neither is QFT.
The question is one of fragility of MWI over potential TOEs, and, relying on exact linearity it is arguably very fragile.
Furthermore with regards to Occam’s razor and specifically formalizations of it, that is also a subtle question requiring domain specific training.
In particular, in Solomonoff Induction, codes have to produce output that exactly matches the observations. Complete with the exact picture of photon noise on the double slit experiment’s screen. Outputting a list of worlds like MWI does is not even an option.
Interesting. I’ll assume an agnostic position again for the time being.
Can you point me towards some of the best comments?
I was aware that both theories are inconsistent with data with respect to gravity, obviously if either of them weren’t, the choice would be clear.
What do you mean by ‘fragility over’ potential theories of everything? That the TOEs suggested thus far tend not to be compatible with it? Presumably not given that the people generating the TOEs are likely to start with the most popular theory.
Whats the standard response by MW enthusiasts to your point on Solomonof induction? My understanding would then suggest that neither MW nor Copenhagen can give an exact picture of photon noise, in which case the problem would seem to be with Solomonoff induction as a formalization.
There’s some around this thread (responses to Luke’s comment). Also I think that QM sequence has responses from physicists.
The MWI is concluded from exactly linear quantum mechanics. Because we know QM to be only an approximation, we lack any strong reasons to expect exact linearity in the final TOE. Furthermore even though exact linearity is arguably favoured by the Occam’s razor over any purely speculative non-linear theory, that does not imply that it is more probable than all of the nonlinear theories together (which would have same linear approximation).
In my opinion, things like multitude of potential worlds allow for e.g. elegantly (and compactly) expressing some conservation laws as survivor bias (via some sort of instability destroying observers in the world where said laws do not hold). Whenever that is significant to TOEs is, of course, purely speculative.
As far as I know, the arguments that Solomonoff induction supports MWI never progressed beyond mere allusions to such support.
In raw form, yes, neither interpretation fits and it’s unclear how to compare complexities of them formally.
I explored some on how S.I. would work on data from quantum experiments here. Basically, the task is to represent said photon noise with the minimum amount of code and data, which can be done in two steps by calculating probabilities as per QM and Born rule, and using the probability density function to decode photon coordinates from the subsequent input bits. (analogous to collapse), or perhaps more compactly in one step by doing QM with some sort of very clever bit manipulation on strings of random noise as to obtain desired probability distribution in the end.
You need to keep in mind that they have access to a very, very huge number of things that you don’t have access to (and have potential access to more things than you think they do), and can get the taste of the enormous space of issues any of which can demolish the case for MWI that you see completely, from the outside. For example, non-linearity of the equations absolutely kills MWI. Now, only a subset of physicists considers the non-linearity to be likely. Others believe in other such things, most of which just kill MWI outright, with all the argument for it flying apart like a card house in the wind of a category 6 hurricane. Expecting a hurricane, experts choose not to bother with the question of mechanical stability of the card house on it’s own—they would rather debate in which direction, and how far, the cards will land.
Consider a case where you talk to a top flat-earth believer and extract his top 10 reasons for believing the earth is flat.
The act of selecting a Muslim philosopher confounds the accuracy of his belief with the method whereby you selected him. It’s like companies searching out a scientist who happens to take a view of some question congenial to that company, then booming his research.
Thanks for the thoughtful response. I agree that there are many instances in which it’s possible to rationally come to confident conclusions that differ from those of subject matter experts. I realize that my earlier comment was elliptical, and will try to clarify. The relevant points to my mind are:
The extraordinary intellectual caliber of the best physicists
Though difficult to formalize, I think that there’s a meaningful sense in which one can make statements of the general form “person A has intellectual caliber n times that of person B.” Of course, this is domain specific to some degree, but I think that the concept hangs together somewhat even across domains.
One operationalization of this is “if person B reaches a correct conclusion on a given subject, person A could reach it n times as fast.” Another is “it would take n copies of person B to do person A’s work.” These things are hard to estimate, but one can become better calibrated by using the rule “if person A has intellectual caliber n times that of person B and person B has intellectual caliber m times that of person C, then person A has intellectual caliber n*m times that of person C.”
In almost all domains, I think that the highest intellectual caliber people have no more than 5x my intellectual caliber. Physics is different. From what I’ve heard, the distribution of talent in physics is similar to that of math. The best mathematicians are 100x+ my intellectual caliber. I had a particularly striking illustrative experience with Don Zagier, who pinpointed a crucial weakness in an analogy that I had been exploring for 6 months (and which I had run by a number of other mathematicians) in a mere ~15 minutes. I would not be surprised if he himself were to have an analogous experience with the historical greats.
When someone is < 5x one’s intellectual caliber, an argument of the type “this person may be smarter than me, but I’ve focused a lot more on having accurate views, so I trust my judgment over him or her” seems reasonable. But when one gets to people who are 100x+ one’s intellectual caliber, the argument becomes much weaker. Model uncertainty starts to play a major role. It could be that people who are that much more powerful easily come to the correct conclusion on a given question without even needing to put conscious effort into having accurate beliefs.
The intrinsic interest of the question of interpretation of quantum mechanics
The question of what quantum mechanics means has been considered one of the universe’s great mysteries. As such, people interested in physics have been highly motivated to understand it. So I think that the question is privileged relative to other questions that physicists would have opinions on — it’s not an arbitrary question outside of the domain of their research accomplishments.
Solicitation of arguments from those with opposing views
In the Muslim theology example, you spend 40 hours engaging with the Muslim philosophers. This seems disanalogous to the present case, in that as far as I know, Eliezer’s quantum mechanics sequence hasn’t been vetted by any leading physicists who disagree with the many world’s interpretation of quantum mechanics. I also don’t know of any public record of ~40 hours of back and forth analogous to the one that you describe. I know that Eliezer might cite an example in his QM sequence, and will take a look.
My understanding is that the interpretation of QM is (1) not regarded as a very central question in physics, being seen more as a “philosophy” question and being worked on to a reasonable extent by philosophers of physics and physicists who see it as a hobby horse, (2) is not something that physics expertise—having good physical intuition, strong math skills, detailed knowledge of how to apply QM on concrete problems—is as relevant for as many other questions physicists work on, and (3) is not something about which there is an extremely enormous amount to say. These are some of the main reasons I feel I can update at all from the expert distribution of physicists on this question. I would hardly update at all from physicist opinions on, say, quantum gravity vs. string theory, and I think it would basically be crazy for me to update substantially in one direction or the other if I had comparable experience on that question.
[ETA: As evidence of (1), I might point to the prevalence of the “shut up and calculate” mentality which seems to have been reasonably popular in physics for a while. I’d also point to the fact that Copenhagen is popular but really, really, really, really not good. And I feel that this last claim is not just Nick Beckstead’s idiosyncratic opinion, but the opinion of every philosopher of physics I have ever spoken with about this issue.]
A minor quibble.
I believe you are using bad terminology. ‘Quantum gravity’ refers to any attempt to reconcile quantum mechanics and general relativity, and string theory is one such theory (as well as a theory of everything). Perhaps you are referring to loop quantum gravity, or more broadly, to any theory of quantum gravity other than string theory?
Perhaps I should have meant loop quantum gravity. I confess that I am speaking beyond my depth, and was just trying to give an example of a central dispute in current theoretical physics. That is the type of case where I would not like to lean heavily on my own perspective.
I agree if we’re talking about the median theoretical physicist at a top 5 school, but when one gets further toward the top of the hierarchy, one starts to see a high density of people who are all-around intellectually curious and who explore natural questions that they come across independently of whether they’re part of their official research.
I agree, but a priori I suspect that philosophers of physics and others without heavy subject matter knowledge of quantum mechanics have leaned too heavily on this. Spending one’s life thinking about something can result subconscious acquisition of implicit knowledge of things that are obliquely related. People who haven’t had this experience may be at a disadvantage.
I actually think that it’s possible for somebody without subject matter knowledge to rationally develop priors that are substantially different from expert consensus here. One can do this by consulting physicists who visibly have high epistemic rationality outside of physics, by examining sociological factors that may have led to the status quo, and by watching physicists who disagree debate each other and see which of the points they respond to and which ones they don’t.
Can you give a reference?
But note that philosophers of physics sometimes make whole careers thinking about this, and they are among the most high-caliber philosophers. They may be at an advantage in terms of this criterion.
I can’t think of a reference in print for my claim about what almost all philosophers think. I think a lot of them would find it too obvious to say, and wouldn’t bother to write a paper about it. But, for what it’s worth, I attended a couple of conferences on philosophy of physics held at Rutgers, with many leading people in the field, and talked about this question and never heard anyone express an opposing opinion. And I was taught about interpretations of QM from some leading people in philosophy of physics.
What I’m anchoring on here is the situation in the field of philosophy of math, where lack of experience with the practice of math seriously undercuts most philosophers’ ability to do it well. There are exceptions, for example I consider Imre Lakatos to be one. Maybe the situation is different in philosophy of physics.
But note that philosophers of physics sometimes make whole careers thinking about this, and they are among the most high-caliber philosophers. They may be at an advantage in terms of this criterion.
I can’t think of a reference in print for my claim about what almost all philosophers think. I think a lot of them would find it too obvious to say, and wouldn’t bother to write a paper about it. But, for what it’s worth, I attended a couple of conferences on philosophy of physics held at Rutgers, with many leading people in the field, and talked about this question and never heard anyone express an opposing opinion. And I was taught about interpretations of QM from some leading people in philosophy of physics.
Er, http://lesswrong.com/lw/r8/and_the_winner_is_manyworlds/ maybe.
Let me first say that I find this to be an extremely interesting discussion.
I think there is a social norm in mathematics and physics that requires people to say this, but I have serious doubts about whether it is true. Anyone 100x+ your intellectual caliber should be having much, much more impact on the world (to say nothing of mathematics itself) than any of the best mathematicians seem to be having. At the very least, if there really are people of that cognitive level running around, then the rest of the world is doing an absolutely terrible job of extracting information and value from them, and they themselves must not care too much about this fact.
More plausible to me is the hypothesis that the best mathematicians are within the same 5x limit as everyone else, and that you overestimate the difficulty of performing at their level due to cultural factors which discourage systematic study of how to imitate them.
Try this thought experiment: suppose you were a graduate student in mathematics, and went to your advisor and said: “I’d like to solve [Famous Problem X], and to start, I’m going to spend two years closely examining the work of Newton, Gauss, and Wiles, and their contemporaries, to try to discern at a higher level of generality what the cognitive stumbling blocks to solving previous problems were, and how they overcame them, and distill these meta-level insights into a meta-level technique of my own which I’ll then apply to [Famous Problem X].” What do you think the reaction would be? How many times do you think such a thing has ever been proposed, let alone attempted, by a serious student or (even) senior researcher?
This is a terrible idea unless they’re spending half their time pushing their limits on object-level math problems. I just don’t think it works to try to do a meta phase before an object phase unless the process is very, very well-understood and tested already.
I’m sure that’s exactly what the advisor would say (if they bother to give a reasoned reply at all), with the result that nobody ever tries this.
(I’ll also note that it’s somewhat odd to hear this response from someone whose entire mission in life is essentially to go meta on all of humanity’s problems...)
But let me address the point, so as not to be logically rude. The person would be pushing their limits on object-level math problems in the course of “examining the work of Newton, Gauss, and Wiles”, in order to understand said work; otherwise, it can hardly be said to constitute a meaningful examination. I also think it’s important not to confuse meta-ness with (nontechnical) “outside views”; indeed I suspect that a lot of the thought processes of mathematical “geniuses” consist of abstracting over classes of technical concepts that aren’t ordinarily abstracted over, and thus if expressed explicitly (which the geniuses may lack the patience to do) would simply look like another form of mathematics. (Others of their processes, I speculate, consist in obsessive exercising of visual/dynamic mental models of various abstractions.)
Switching back to logical rudeness, I’m not sure the meta-ness is your true rejection; I suspect what you may be really worried about is making sure there are tight feedback loops to which one’s reasoning can be subjected.
That’s not the kind of meta I mean. The dangerous form of meta is when you spend several years preparing to do X, supposedly becoming better at doing X, but not actually doing X, and then try to do X. E.g. college. Trying to improve at doing X while doing X is much, much wiser. I would similarly advise Effective Altruists who are not literally broke to be donating $10 every three months to something while they are trying to increase their incomes and invest in human capital; furthermore, they should not donate to the same thing two seasons in a row, so that they are also practicing the skill of repeatedly assessing which charity is most important.
“Meta” for these purposes is any daily activity which is unlike the daily activity you intend to do ‘later’.
Tight feedback loops are good, but not always available. This is a separate consideration from doing meta while doing object.
The activity of understanding someone else’s proofs may be unlike the activity of producing your own new math from scratch; this would be the problem.
This is excellent advice. I have put a note in my calendar thee months hence to reevaluate my small monthly donation.
Nice to hear from you :-)
Yes, I think that this is what the situation is.
I’ll also say that I think that there are very few such people — maybe on the order of 10 who are alive today. With such a small absolute number, I don’t think that their observed impact on math is a lot lower than what one would expect a priori, and the prior in favor them having had a huge impact in society isn’t that strong.
“The best mathematicians are 100+x higher in intellectual caliber than I am” and “the difference is in large part due to cultural factors which discourage systematic study of how to imitate them” aren’t mutually exclusive. I’m sympathetic to your position.
To change the subject :-)
Basically never.
Not to mention that some of them might be working on Wall Street or something, and not have worked on unsolved problems in mathematics in decades.
How do you know how little intellectual caliber JonahSinick has?
looks at JonahSinick’s profile
follows link to his website
skims the “About me” page
Yes, you have a point.
That is of course exactly why I picked QM and MWI to make my case for nihil supernum. It wouldn’t serve to break a smart person’s trust in a sane world if I demonstrated the insanity of Muslim theologians or politicians; they would just say, “But surely we should still trust in elite physicists.” It is by demonstrating that trust in a sane world fails even at the strongest point which ‘elite common sense’ would expect to find, that I would hope to actually break someone’s emotional trust, and cause them to just give up.
I haven’t fully put together my thoughts on this, but it seems like a bad test to “break someone’s trust in a sane world” for a number of reasons:
this is a case where all the views are pretty much empirically indistinguishable, so it isn’t an area where physicists really care all that much
since the views are empirically indistinguishable, it is probably a low-stakes question, so the argument doesn’t transfer well to breaking our trust in a sane world in high-stakes cases; it makes sense to assume people would apply more rationality in cases where more rationality pays off
as I said in another comment, MWI seems like a case where physics expertise is not really what matters, so this doesn’t really show that the scientific method as applied by physicists is broken; it seems it at most it shows that physics aren’t good at questions that are essentially philosophical; it would be much more persuasive if you showed that e.g., quantum gravity was obviously better than string theory and only 18% of physicists working in the relevant area thought so
[Edited to add a missing “not”]
From my perspective, the main point is that if you’d expect AI elites to handle FAI competently, you would expect physics elites to handle MWI competently—the risk factors in the former case are even greater. Requires some philosophical reasoning? Check. Reality does not immediately call you out on being wrong? Check. The AI problem is harder than MWI and it has additional risk factors on top of that, like losing your chance at tenure if you decide that your research actually needs to slow down. Any elite incompetence beyond the demonstrated level in MWI doesn’t really matter much to me, since we’re already way under the ‘pass’ threshold for FAI.
I feel this doesn’t address the “low stakes” issues I brought up, or that this may not even by the physicists’ area of competence. Maybe you’d get a different outcome if the fate of the world depended on this issue, as you believe it does with AI.
I also wonder if this analysis leads to wrong historical predictions. E.g., why doesn’t this reasoning suggest that the US government would totally botch the constitution? That requires philosophical reasoning and reality doesn’t immediately call you out on being wrong. And the people setting things up don’t have incentives totally properly aligned. Setting up a decent system of government strikes me as more challenging than the MWI problem in many respects.
How much weight do you actually put on this line of argument? Would you change your mind about anything practical if you found out you were wrong about MWI?
What different evidence would you expect to observe in a world where amateur attempts to set up systems of government were usually botched?
(Edit: reworded for (hopefully) clarity.)
I have an overall sense that there are a lot of governments that are pretty good and that people are getting better at setting up governments over time. The question is very vague and hard to answer, so I am not going to attempt a detailed one. Perhaps you could give it a shot if you’re interested.
You meant “is not really”?
Yes, thank you for catching.
I agree that if it were true that the consensus of elite physicists believed that MWI is wrong when there was a decisive case in favor of it, that would be striking. But
There doesn’t seem to be a consensus among elite physicists that MWI is wrong.
Paging through your QM sequence, it doesn’t look as though you’ve systematically addressed all objections that otherwise credible people have raised against MWI. For example, have you been through all of the references that critique MWI cited in this paper? I think given that most experts don’t view the matter as decided, and given the intellectual caliber of the experts, in order have 99+% confidence in this setting, one has to cover all of one’s bases.
One will generally find that correct controversial ideas convince some physicists. There are many physicists who believe MWI (though they perhaps cannot get away with advocating it as rudely as I do), there are physicists signed up for cryonics, there were physicists advocating for Drexler’s molecular nanotechnology before it was cool, and I strongly expect that some physicists who read econblogs have by now started advocating market monetarism (if not I would update against MM). A good new idea should have some physicists in favor of it, and if not it is a warning sign. (Though the endorsement of some physicists is not a proof, obviously many bad ideas can convince a few physicists too.) If I could not convince any physicists of my views on FAI, that would be a grave warning sign indeed. (I’m pretty sure some such already exist.) But that a majority of physicists do not yet believe in MWI does not say very much one way or another.
The cognitive elites do exist and some of them are physicists, therefore you should be able to convince some physicists. But the cognitive elites don’t correspond to a majority of MIT professors or anything like that, so you shouldn’t be able to convince a majority of that particular group. A world which knew what its own elite truthfinders looked like would be a very different world from this one.
Ok, putting aside MWI, maybe our positions are significantly more similar than it initially seemed. I agree with
I’ve taken your comments such as
to carry connotations of the type “the fraction of people who exhibit high epistemic rationality outside of their immediate areas of expertise is vanishingly small.”
I think that there are thousands of people worldwide who exhibit very high epistemic rationality in most domains that they think about. I think that most of these people are invisible owing to the paucity of elites online. I agree that epistemic standards are generally very poor, and that high status academics generally do poorly outside of their immediate areas of expertise.
Where does this impression come from? Are they people you’ve encountered personally? If so, what gave you the impression that they exhibited “very high epistemic rationality in most domains that they think about”?
To clarify, when I wrote “very high epistemic rationality” I didn’t mean “very accurate beliefs,” but rather “aware of what they know and what they don’t.” I also see the qualifier “most” as significant — I think that any given person has some marked blind spots. Of course, the boundary that I’m using is fuzzy, but the standard that I have in mind is something like “at the level of the 15 most epistemically rational members of the LW community.”
“Thousands” is at the “1 in a million” level, so in relative terms, my claim is pretty weak. If one disputes the claim, one needs a story explaining how the fraction could be so small. It doesn’t suffice to say “I haven’t personally seen almost any such people,” because there are so many people who one hasn’t seen, and the relevant people may be in unexpected places.
I’ve had the subjective impression that ~ 2% of those who I know outside of the LW/EA spheres fit this description. To be sure, there’s a selection effect, but I’ve had essentially no exposure to physics, business, finance or public policy, nor to people in very populous countries such as India and China. The people who I know who fit this description don’t seem to think that they’re extremely rare, which suggests that their experiences are similar to my own (though I recognize that “it’s a small world,” i.e. these people’s social circles may overlap in nonobvious ways).
Some of the people who GiveWell has had conversations with come across very favorably in the notes. (I recognize that I’m making a jump from “their area of specialty” to “most topics that they think about” Here I’m extrapolating from the people who I know who are very strong in their area of specialty.) I think that Carl’s comment about the Gates Foundation is relevant.
I updated in the direction of people being more rational than I had thought for reason that I gave at the end of my post Many weak arguments and the typical mind.
I don’t have high confidence here: maybe ~ 50% (i.e. I’m giving a median case estimate).
I should also clarify that I don’t think that one needs a silver bullet argument of the type “the people who you would expect to be most trustworthy have the wrong belief on something that they’ve thought about, with very high probability” to conclude with high confidence that epistemic standards are generally very low.
I think that there are many weak arguments that respected authorities are very often wrong.
Vladimir M has made arguments of the type “there’s fierce disagreement among experts at X about matters pertaining to X, so one knows that at least some of them are very wrong.” I think that string theory is a good case study. There are very smart people who strongly advocate for string theory as a promising road for theoretical physics research, and other very smart people who strongly believe that the research program is misguided. If nothing else, one can tell that a lot of the actors involved are very overconfident (even if one doesn’t know who they are).
Or, alternatively, they disagree about who the research program is promising for.
That’s too strong of a statement. If you exclude some diehards like Deutsch, the Bohmian school and maybe Penrose’s gravity-induced decoherence, the prevailing attitude is “however many worlds are out there, we can only ever see one”, so, until new evidence comes along, we can safely treat MWI as a single world.
In computer science an elite coder might take 6 months to finish a very hard task (e.g. create some kind of tricky OS kernel), but a poor coder will never complete the task. This makes the elite coder infinitely better than the poor coder. Furthermore the poor coder will ask many questions of other people, impacting their productivity. Thus an elite coder is transfinitely more efficient than a poor coder ;-)
See also Stephen Hsu’s comments on this.
Another huge difference is that much of quantum mechanics is very technical physics. To get to the point where you can even have an opinion you need a fair amount of background information. When assessing expert opinion, you have a hugely difficult problem of trying to discern whether an expert physicist has relevant technical knowledge you do not have OR whether they are making a failure in rationality.
This of course is exactly what Muslim theologians would say about Muslim theology. And I’m perfectly happy to say, “Well, the physicists are right and Muslim theologians are wrong”, but that’s because I’m relying on my own judgment thereon.
The equivalent to asking Muslim theologians about Allah would be to ask many-worlds-believing quantum physicists about many-worlds.
The equivalent of asking quantum physicists about many-worlds would be to ask theologians about Allah, without specifically picking Muslim theologians. And if you ask theologians about Allah (by which I mean the Muslim conception of God—of course “Allah” is just the Arabic for “God”), you’re going to find that quite a few of them don’t think that Allah exists and that some other version of God does.
And that’s not even getting into the problems caused by the fact that religion is a meme that spreads in a way that skews the population of experts, which quantum mechanics doesn’t.
Definitely. Scientists in general and physicists in particular are probably no better than other professionals in instrumental rationality outside of their area of expertise, not sure about epistemic rationality. The “top-performing physicists” (what a strange name, physicists are not athletes), whoever they are, are probably not very much better, as you mention. I have seen some of them committing a number of standard cognitive fallacies.
In fact, I personally think that you are way more rational than many famous physicists, since you took pains to improve your rationality skills and became an expert in the area, and they did not.
However, what you have no hope of is to competently judge their results and beliefs about physics, except by relying on the opinions of other physicists and deciding whom to trust how much in case of a disagreement. But I guess we are in agreement here.