I have confidence that nobody here has secret information that makes human extinction much more likely—because almost no information which currently exists could have more than a marginal bearing on a result which, if likely, is a result of human (that is, intelligent) interaction. Therefore I have confidence that the difference in estimates is largely not due to information, but to models. I have confidence that inductive models—say, “how often does a random species survive any hundred year period, correcting for initial population” give answers over 95% which should be considered the default. Therefore, I have confidence that a community of people who generally give lower estimates is subject to some biases (such as narrative bias).
Doesn’t mean LW’s wrong and I’m right. But to believe that human extinction within a century is likely clearly puts LW in the minority of humanity in your beliefs—even in the minority of rational atheists. And the fact that there is substantial agreement within the LW community on this, when uncertainty is clearly so high that orders of magnitude of disagreement are possible, makes me suspect bias.
Also, I find it funny that people will argue passionately over estimates that differ in log(p/q) from −1 to +1 (~10% to ~90%), but couldn’t care less over the difference from say −9 to −7 (.0001% vs .000001%) or 7 to 9. This is in one sense the right attitude for people who think they can do something about it, but it ends up biasing numbers towards log(p/q)=0 [ie 50%], since you are more likely to get argument from somebody who has an estimate on the other side of 50% as yours is.
The fact that we believe something unusual is only weak evidence for the validity of that unusual belief, you are right on that. And given the hypothesis that we are wrong, which is dominant while all you have is the observation that we believe something unusual, you can draw a conclusion that we are wrong because of some systematic error of judgment that makes most here to claim the unusual belief.
To move past this point, you have to consider the specific arguments, and decide for yourself whether to accept them.
Most of the beliefs people can hold intuitively are about 50% in certainty. The beliefs far away from this point aren’t useful as primitive concepts, classifying the possible events on one side or the other, as most everything is only on one side, and human mind can’t keep track of their levels of certainty. New concepts get constructed, that are more native to human mind and express the high-certainty concepts in question only in combinations, or that are supported by non-intuitive procedures for processing levels of certainty. But if the argument is dependent on use of intuition, you aren’t always capable of moving towards certainty, so you remain in doubt. This is the case for unknown unknowns, in particular.
I have confidence that nobody here has secret information that makes human extinction much more likely—because almost no information which currently exists could have more than a marginal bearing on a result which, if likely, is a result of human (that is, intelligent) interaction. Therefore I have confidence that the difference in estimates is largely not due to information, but to models. I have confidence that inductive models—say, “how often does a random species survive any hundred year period, correcting for initial population” give answers over 95% which should be considered the default. Therefore, I have confidence that a community of people who generally give lower estimates is subject to some biases (such as narrative bias).
Doesn’t mean LW’s wrong and I’m right. But to believe that human extinction within a century is likely clearly puts LW in the minority of humanity in your beliefs—even in the minority of rational atheists. And the fact that there is substantial agreement within the LW community on this, when uncertainty is clearly so high that orders of magnitude of disagreement are possible, makes me suspect bias.
Also, I find it funny that people will argue passionately over estimates that differ in log(p/q) from −1 to +1 (~10% to ~90%), but couldn’t care less over the difference from say −9 to −7 (.0001% vs .000001%) or 7 to 9. This is in one sense the right attitude for people who think they can do something about it, but it ends up biasing numbers towards log(p/q)=0 [ie 50%], since you are more likely to get argument from somebody who has an estimate on the other side of 50% as yours is.
The fact that we believe something unusual is only weak evidence for the validity of that unusual belief, you are right on that. And given the hypothesis that we are wrong, which is dominant while all you have is the observation that we believe something unusual, you can draw a conclusion that we are wrong because of some systematic error of judgment that makes most here to claim the unusual belief.
To move past this point, you have to consider the specific arguments, and decide for yourself whether to accept them.
Most of the beliefs people can hold intuitively are about 50% in certainty. The beliefs far away from this point aren’t useful as primitive concepts, classifying the possible events on one side or the other, as most everything is only on one side, and human mind can’t keep track of their levels of certainty. New concepts get constructed, that are more native to human mind and express the high-certainty concepts in question only in combinations, or that are supported by non-intuitive procedures for processing levels of certainty. But if the argument is dependent on use of intuition, you aren’t always capable of moving towards certainty, so you remain in doubt. This is the case for unknown unknowns, in particular.