Again, even if I don’t have time for it myself, I think it would be useful to gather data on particular disagreements based on such “outside views” with vague similarities, and then see how often each side happened to be right.
Of course, even if the “outside view” happened to be right in most such cases, you might just respond with the same argument that all of these particular disagreements are still radically different from your case. But it might not be a very good response, in that situation.
But the original problem was that, in order to carry an argument about the “outside view” in case of what (arguendo) was so drastic a break with the past as to have only two possible analogues if that, data was being presented about students guessing their homework times. If instead you gather data about predictions made in cases of, say, the industrial revolution or the invention of the printing press, you’re still building a certain amount of your conclusion into your choice of data.
What I would expect to find is that predictive accuracy falls off with dissimilarity and attempted jumps across causal structural gaps, that the outside view wielded with reasonable competence becomes steadily less competitive with the Weak Inside View wielded with reasonable competence. And this could perhaps be extrapolated across larger putative differences, so as to say, “if the difference is this large, this is what will happen”.
But I would also worry that outside-view advocates would take the best predictions and reinterpret them as predictions the outside view “could” have made (by post-hoc choice of reference class) or because the successful predictor referenced historical cases in making their argument (which they easily could have done on the basis of having a particular inside view that caused them to argue for that conclusion), and comparing this performance to a lot of wacky prophets being viewed as “the average performance of the inside view” when a rationalist of the times would have been skeptical in advance even without benefit of hindsight. (And the corresponding wackos who tried to cite historical cases in support not being considered as average outside viewers—though it is true that if you’re going crazy anyway, it’s a bit easier to go crazy with new ideas than with historical precedents.)
And outside-view advocates could justly worry that by selecting famous historical cases, we are likely to be selecting anomalous breaks with the past that (arguendo) could not have been predicted in advance. Or that by filtering on “reasonably competent” predictions with benefit of hindsight, inside view advocates would simply pick one person out of a crowd who happened to get it right, while surely competent outside view advocates would all tend to give similar predictions, and so not be allowed a similar post-hoc selection bias.
Note: I predict that in historical disagreements, people appealing to historical similarities will not all give mostly similar predictions, any more than taw and Robin Hanson constructed the same reference class for the Singularity.
But then by looking for historical disagreements, you’re already filtering out a very large class of cases in which the answer is too obvious for there to be famous disagreements about it.
I should like to have the methodology available for inspection, and both sides comparing what they say they expect in advance, so that it can be known what part of the evidential flow is agreed-upon divergence of predictions, and what part of the alleged evidence is based on one side saying that the other side isn’t being honest about what their theory predicts.
It’s not like we’re going to find a large historical database of people explicitly self-identifying as outside view advocates or weak inside view advocates. Not in major historical disputes, as opposed to homework.
Again, even if I don’t have time for it myself, I think it would be useful to gather data on particular disagreements based on such “outside views” with vague similarities, and then see how often each side happened to be right.
Of course, even if the “outside view” happened to be right in most such cases, you might just respond with the same argument that all of these particular disagreements are still radically different from your case. But it might not be a very good response, in that situation.
But the original problem was that, in order to carry an argument about the “outside view” in case of what (arguendo) was so drastic a break with the past as to have only two possible analogues if that, data was being presented about students guessing their homework times. If instead you gather data about predictions made in cases of, say, the industrial revolution or the invention of the printing press, you’re still building a certain amount of your conclusion into your choice of data.
What I would expect to find is that predictive accuracy falls off with dissimilarity and attempted jumps across causal structural gaps, that the outside view wielded with reasonable competence becomes steadily less competitive with the Weak Inside View wielded with reasonable competence. And this could perhaps be extrapolated across larger putative differences, so as to say, “if the difference is this large, this is what will happen”.
But I would also worry that outside-view advocates would take the best predictions and reinterpret them as predictions the outside view “could” have made (by post-hoc choice of reference class) or because the successful predictor referenced historical cases in making their argument (which they easily could have done on the basis of having a particular inside view that caused them to argue for that conclusion), and comparing this performance to a lot of wacky prophets being viewed as “the average performance of the inside view” when a rationalist of the times would have been skeptical in advance even without benefit of hindsight. (And the corresponding wackos who tried to cite historical cases in support not being considered as average outside viewers—though it is true that if you’re going crazy anyway, it’s a bit easier to go crazy with new ideas than with historical precedents.)
And outside-view advocates could justly worry that by selecting famous historical cases, we are likely to be selecting anomalous breaks with the past that (arguendo) could not have been predicted in advance. Or that by filtering on “reasonably competent” predictions with benefit of hindsight, inside view advocates would simply pick one person out of a crowd who happened to get it right, while surely competent outside view advocates would all tend to give similar predictions, and so not be allowed a similar post-hoc selection bias.
Note: I predict that in historical disagreements, people appealing to historical similarities will not all give mostly similar predictions, any more than taw and Robin Hanson constructed the same reference class for the Singularity.
But then by looking for historical disagreements, you’re already filtering out a very large class of cases in which the answer is too obvious for there to be famous disagreements about it.
I should like to have the methodology available for inspection, and both sides comparing what they say they expect in advance, so that it can be known what part of the evidential flow is agreed-upon divergence of predictions, and what part of the alleged evidence is based on one side saying that the other side isn’t being honest about what their theory predicts.
It’s not like we’re going to find a large historical database of people explicitly self-identifying as outside view advocates or weak inside view advocates. Not in major historical disputes, as opposed to homework.
Which outside view do you propose to use to evaluate the intelligence explosion, if you find that “the” outside view is usually right in such cases?