A decade late to the party, I’d like to join those skeptical of EY’s use of many-worlds as a slam-dunk test of contrarian correctness. Without going into the physics (for which I’m unqualified), I have to make the obvious general objection that it is sophomoric for an amateur in an intellectual field—even an extremely intelligent and knowledgeable one—to claim a better understanding than those who have spent years studying it professionally. It is of course possible for an amateur to have an insight professionals have missed, but very rare.
I had a similar feeling on reading EY’s Inadequate Equilibria, where I was far from convinced by his example that an amateur can adjudicate between an economics blogger and central bankers and tell who is right. (EY’s argument that the central bankers may have perverse incentives to give a dishonest answer is not that strong, since they may give an honest answer anyway, and that fact that with 20-20 hindsight it might look like they were wrong just shows that economics is an inexact science.) The economics blogger may make points that seem highly plausible and convincing to an amateur, but then, one-sided arguments often do.
Back to physics, any amateur who says “many-worlds is just obvious if you understand it, so those who say otherwise are obviously wrong” is claiming a better understanding than many professionals in the field; again backed with allegations of perverse incentives. Though the latter carry some weight, I’d put my money on the amateur just being overconfident, and having missed something.
If anything I’d judge people on the sophistication of their reasons rather than the opinion itself. E.g. I’d take more notice of someone who had a sophisticated reason for denying that 1 + 1 = 2 than someone who said ‘it’s just obvious, and anyone who says otherwise is an idiot’.
(I for one have doubts that 1 + 1 = 2; the most I’d be prepared to say is that 1 + 1 often equals 2. And I’m in good company here—e.g. Wittgenstein had grave doubts that simple counting and addition (and indeed any following of rules) are determinate processes with definite results, something which he discussed in seminars with Alan Turing among other students of his.)
The one kind of case in which I’d prefer the factual opinion of a sophisticated amateur to a professional is in fields which don’t involve enough intellectual rigour. For example I’d rather believe an amateur with an advanced understanding of evolutionary psychology than some gender studies professors to give a correct explanation of certain social phenomena; not just because the professors may well have an ideological axe to grind, but also because they may lack the scientific rigour necessary to understand the subtleties of causation and statistics.
You’re leaning heavily on the concept “amateur”, which (a) doesn’t distinguish “What’s your level of knowledge and experience with X?” and “Is X your day job?”, and (b) treats people as being generically “good” or “bad” at extremely broad and vague categories of proposition like “propositions about quantum physics” or “propositions about macroeconomics”.
I think (b) is the main mistake you’re making in the quantum physics case. Eliezer isn’t claiming “I’m better at quantum physics than professionals”. He’s claiming that the specific assertion “reifying quantum amplitudes (in the absence of evidence against collapse/agnosticism/nonrealism) violates Ockham’s Razor because it adds ‘stuff’ to the universe” is false, and that a lot of quantum physicists have misunderstood this because their training is in quantum physics, not in algorithmic information theory or formal epistemology.
I think (a) is the main mistake you’re making in the economics case. Eliezer is basically claiming to understand macroeconomics better than key decisionmakers at the Bank of Japan, but based on the results, I think he was just correct about that. As far as I can tell, Eliezer is just really good at economic reasoning, even though it’s not his day job. Cf. Central banks should have listened to Eliezer Yudkowsky (or 1, 2, 3).
Eliezer’s econ case is based on reading Scott Sumner’s blog, so it’s not very informative that Sumner praises Eliezer (3 out of 4 endorsements you linked, the remaining one is anon).
bfinn was discounting Eliezer for being a non-economist, rather than discounting Sumner for being insufficiently mainstream; and bfinn was skeptical in particular that Eliezer understood NGDP targeting well enough to criticize the Bank of Japan. So Sumner seems unusually relevant here, and I’d expect him to pick up on more errors from someone talking at length about his area of specialization.
First, thanks for your comments on my comments, which I thought no-one would read on such an old article!
Re your quantum physics point, with unusual topics like this that overlap with philosophy (specifically metaphysics), it is true that physicists can be out of their depth on that part of it, and so someone with a strong understanding of metaphysics (even if not a professional philosopher as such) can point out errors in the physicists’ metaphysics. That said, saying X is clearly wrong (due to faulty metaphysics) is a weaker claim than that Y is clearly right, particularly if there are many competing views. (As there are AFAIK even in the philosophy of QM.) Just as a professional physicist can’t be certain about getting the metaphysics bit of QM right, even a professional philosopher couldn’t be certain about the physics bit of it; not certain enough to claim a slam-dunk. So without going into the specifics of the case (which I’m not qualified to do) it still seems like an overreach.
Also, more generally, I assume interdisciplinary topics like this (for which a highly knowledgeable amateur could spot flaws in the reasoning of someone who’s a professional in one discipline but not the other) are the exception rather than the rule.
Re the economics case, well, for all I know, EY may well have been right in this case (and for the right reasons), but if so then it’s just a rare example of an amateur who has a very high professional-level understanding of a particular topic (though presumably not of various other parts of economics). I.e. this is an exception.
That said, and without going into the fine details of the case, the professionals here presumably include the top macroeconomists in Japan. Is it really plausible that EY understands the relevant economics and knows more relevant information than them? (E.g. they may well have considered all kinds of facts & figures that aren’t public or at least known to EY.) Which is presumably where the issue of other biases/influences on them would come in; and while I accept that there could be personal/political biases/reasons for doing the economically wrong thing, this can be too easy a way of dismissing expert opinion.
So I’d still put my money on the professional vs the amateur, however persuasive the latter’s arguments might seem to me. And again, the fact that the Bank of Japan’s decision turned out badly may just show that economics is an inexact science, in which correct bets can turn out badly and incorrect bets turn out well.
One other exception I’d like to add to my original comment: it is certainly true that a highly expert professional in a field can be very inexpert in topics that are close to but not within their own specialism. (I know of this in my own case, and have observed it in others, e.g. lawyers. E.g. a corporate lawyer may only have a sketchy understanding of IP law. Though they are well aware of this.)
The alignment problem is arguably another example, like my above response re quantum physics, of a field spilling over into philosophy, such that even a strong amateur philosopher can point things out that the AI professionals hadn’t thought through. I.e. it shows that AI alignment is an interdisciplinary topic which (I assume) went beyond existing mainstream AI.
Huh? Strong evidence for that would be us all being dead.
I want to insist that “it’s unreasonable to strongly update about technological risks until we’re all dead” is not a great heuristic for evaluating GCRs.
The latter has come to be true, in no small part as a result of his writing. This implies that there was indeed something academics were missing about alignment.
Only a minority agree with him. Any number of (contradictory!) ideas will “seem to be right” if the criterion is only that some people agree with them.
A sizable shift has occurred because of him, which is different than your interpretation of my position. If you’re convincing Stuart Russell, who is convincing Turing award winners like Yoshua Bengio and Judea Pearl, then there was something that wasn’t considered.
A decade late to the party, I’d like to join those skeptical of EY’s use of many-worlds as a slam-dunk test of contrarian correctness. Without going into the physics (for which I’m unqualified), I have to make the obvious general objection that it is sophomoric for an amateur in an intellectual field—even an extremely intelligent and knowledgeable one—to claim a better understanding than those who have spent years studying it professionally. It is of course possible for an amateur to have an insight professionals have missed, but very rare.
I had a similar feeling on reading EY’s Inadequate Equilibria, where I was far from convinced by his example that an amateur can adjudicate between an economics blogger and central bankers and tell who is right. (EY’s argument that the central bankers may have perverse incentives to give a dishonest answer is not that strong, since they may give an honest answer anyway, and that fact that with 20-20 hindsight it might look like they were wrong just shows that economics is an inexact science.) The economics blogger may make points that seem highly plausible and convincing to an amateur, but then, one-sided arguments often do.
Back to physics, any amateur who says “many-worlds is just obvious if you understand it, so those who say otherwise are obviously wrong” is claiming a better understanding than many professionals in the field; again backed with allegations of perverse incentives. Though the latter carry some weight, I’d put my money on the amateur just being overconfident, and having missed something.
If anything I’d judge people on the sophistication of their reasons rather than the opinion itself. E.g. I’d take more notice of someone who had a sophisticated reason for denying that 1 + 1 = 2 than someone who said ‘it’s just obvious, and anyone who says otherwise is an idiot’.
(I for one have doubts that 1 + 1 = 2; the most I’d be prepared to say is that 1 + 1 often equals 2. And I’m in good company here—e.g. Wittgenstein had grave doubts that simple counting and addition (and indeed any following of rules) are determinate processes with definite results, something which he discussed in seminars with Alan Turing among other students of his.)
The one kind of case in which I’d prefer the factual opinion of a sophisticated amateur to a professional is in fields which don’t involve enough intellectual rigour. For example I’d rather believe an amateur with an advanced understanding of evolutionary psychology than some gender studies professors to give a correct explanation of certain social phenomena; not just because the professors may well have an ideological axe to grind, but also because they may lack the scientific rigour necessary to understand the subtleties of causation and statistics.
You’re leaning heavily on the concept “amateur”, which (a) doesn’t distinguish “What’s your level of knowledge and experience with X?” and “Is X your day job?”, and (b) treats people as being generically “good” or “bad” at extremely broad and vague categories of proposition like “propositions about quantum physics” or “propositions about macroeconomics”.
I think (b) is the main mistake you’re making in the quantum physics case. Eliezer isn’t claiming “I’m better at quantum physics than professionals”. He’s claiming that the specific assertion “reifying quantum amplitudes (in the absence of evidence against collapse/agnosticism/nonrealism) violates Ockham’s Razor because it adds ‘stuff’ to the universe” is false, and that a lot of quantum physicists have misunderstood this because their training is in quantum physics, not in algorithmic information theory or formal epistemology.
I think (a) is the main mistake you’re making in the economics case. Eliezer is basically claiming to understand macroeconomics better than key decisionmakers at the Bank of Japan, but based on the results, I think he was just correct about that. As far as I can tell, Eliezer is just really good at economic reasoning, even though it’s not his day job. Cf. Central banks should have listened to Eliezer Yudkowsky (or 1, 2, 3).
Eliezer’s econ case is based on reading Scott Sumner’s blog, so it’s not very informative that Sumner praises Eliezer (3 out of 4 endorsements you linked, the remaining one is anon).
bfinn was discounting Eliezer for being a non-economist, rather than discounting Sumner for being insufficiently mainstream; and bfinn was skeptical in particular that Eliezer understood NGDP targeting well enough to criticize the Bank of Japan. So Sumner seems unusually relevant here, and I’d expect him to pick up on more errors from someone talking at length about his area of specialization.
First, thanks for your comments on my comments, which I thought no-one would read on such an old article!
Re your quantum physics point, with unusual topics like this that overlap with philosophy (specifically metaphysics), it is true that physicists can be out of their depth on that part of it, and so someone with a strong understanding of metaphysics (even if not a professional philosopher as such) can point out errors in the physicists’ metaphysics. That said, saying X is clearly wrong (due to faulty metaphysics) is a weaker claim than that Y is clearly right, particularly if there are many competing views. (As there are AFAIK even in the philosophy of QM.) Just as a professional physicist can’t be certain about getting the metaphysics bit of QM right, even a professional philosopher couldn’t be certain about the physics bit of it; not certain enough to claim a slam-dunk. So without going into the specifics of the case (which I’m not qualified to do) it still seems like an overreach.
Also, more generally, I assume interdisciplinary topics like this (for which a highly knowledgeable amateur could spot flaws in the reasoning of someone who’s a professional in one discipline but not the other) are the exception rather than the rule.
Re the economics case, well, for all I know, EY may well have been right in this case (and for the right reasons), but if so then it’s just a rare example of an amateur who has a very high professional-level understanding of a particular topic (though presumably not of various other parts of economics). I.e. this is an exception.
That said, and without going into the fine details of the case, the professionals here presumably include the top macroeconomists in Japan. Is it really plausible that EY understands the relevant economics and knows more relevant information than them? (E.g. they may well have considered all kinds of facts & figures that aren’t public or at least known to EY.) Which is presumably where the issue of other biases/influences on them would come in; and while I accept that there could be personal/political biases/reasons for doing the economically wrong thing, this can be too easy a way of dismissing expert opinion.
So I’d still put my money on the professional vs the amateur, however persuasive the latter’s arguments might seem to me. And again, the fact that the Bank of Japan’s decision turned out badly may just show that economics is an inexact science, in which correct bets can turn out badly and incorrect bets turn out well.
One other exception I’d like to add to my original comment: it is certainly true that a highly expert professional in a field can be very inexpert in topics that are close to but not within their own specialism. (I know of this in my own case, and have observed it in others, e.g. lawyers. E.g. a corporate lawyer may only have a sketchy understanding of IP law. Though they are well aware of this.)
You should also take into account that Eliezer seems to have been right, as an “amateur” AI researcher, about AI alignment being a big deal.
The alignment problem is arguably another example, like my above response re quantum physics, of a field spilling over into philosophy, such that even a strong amateur philosopher can point things out that the AI professionals hadn’t thought through. I.e. it shows that AI alignment is an interdisciplinary topic which (I assume) went beyond existing mainstream AI.
Huh? Strong evidence for that would be us all being dead. Or did you just mean that some people in the field agree with him?
I want to insist that “it’s unreasonable to strongly update about technological risks until we’re all dead” is not a great heuristic for evaluating GCRs.
The latter has come to be true, in no small part as a result of his writing. This implies that there was indeed something academics were missing about alignment.
Only a minority agree with him. Any number of (contradictory!) ideas will “seem to be right” if the criterion is only that some people agree with them.
A sizable shift has occurred because of him, which is different than your interpretation of my position. If you’re convincing Stuart Russell, who is convincing Turing award winners like Yoshua Bengio and Judea Pearl, then there was something that wasn’t considered.