I just wanted to add some context (that I thought of as “obvious background context”, but probably not everyone is tracking), that Eliezer wrote more about the “rule” here in the 8th post of the Inadequate Equilibria sequence:
I’ve now given my critique of modesty as a set of explicit doctrines. I’ve tried to give the background theory, which I believe is nothing more than conventional cynical economics, that explains why so many aspects of the world are not optimized to the limits of human intelligence in the manner of financial prices. I have argued that the essence of rationality is to adapt to whatever world you find yourself in, rather than to be “humble” or “arrogant” a priori. I’ve tried to give some preliminary examples of how we really, really don’t live in the Adequate World where constant self-questioning would be appropriate, the way it is appropriate when second-guessing equity prices. I’ve tried to systematize modest epistemology into a semiformal rule, and I’ve argued that the rule yields absurd consequences.
I was careful to say all this first, because there’s a strict order to debate. If you’re going to argue against an idea, it’s bad form to start off by arguing that the idea was generated by a flawed thought process, before you’ve explained why you think the idea itself is wrong. Even if we’re refuting geocentrism, we should first say how we know that the Sun does not orbit the Earth, and only then pontificate about what cognitive biases might have afflicted geocentrists. As a rule, an idea should initially be discussed as though it had descended from the heavens on a USB stick spontaneously generated by an evaporating black hole, before any word is said psychoanalyzing the people who believe it. Otherwise I’d be guilty of poisoning the well, also known as Bulverism.
But I’ve now said quite a few words about modest epistemology as a pure idea. I feel comfortable at this stage saying that I think modest epistemology’s popularity owes something to its emotional appeal, as opposed to being strictly derived from epistemic considerations. In particular: emotions related to social status and self-doubt.
I agree that “poisoning the well” and “Bulverism” are bad ideas when arguing for or against ideas. If someone wrote a post “Animals are Conscious” then it would be bad form to spend the first section arguing that Yudkowsky is frequently, confidently, egregiously wrong. However, that is not the post that omnizoid wrote, so it is misdirected criticism. Omnizoid’s post is a (failed) attempt at status regulation.
I just wanted to add some context (that I thought of as “obvious background context”, but probably not everyone is tracking), that Eliezer wrote more about the “rule” here in the 8th post of the Inadequate Equilibria sequence:
I agree that “poisoning the well” and “Bulverism” are bad ideas when arguing for or against ideas. If someone wrote a post “Animals are Conscious” then it would be bad form to spend the first section arguing that Yudkowsky is frequently, confidently, egregiously wrong. However, that is not the post that omnizoid wrote, so it is misdirected criticism. Omnizoid’s post is a (failed) attempt at status regulation.