Just wanted to note that the case under consideration was quite unlike your garden-variety predictions, but more like an industrial project estimate. It gave quite concrete scope, effort and schedule estimates, something rarely done by long-term forecasters. So the relevant reference class would be ambitious scientific and industrial projects. One example is the failed Fifth Generation Computer Systems project. Another was the Manhattan project (not sure if there are circa 1941 estimates evailable for it). Not even the Moon landing project was in the same reference class, because it was clearly “just” an issue of (extremely massive) scaling up, rather than solving unknown problems. Not sure about the controlled nuclear fusion, maybe there are some predictions from 1950s available. Consider looking into Grand Challenges for further examples of the same reference class. Some other projects which come to mind: Feynman gives an account of Wheeler promising to present quantized gravity for the next seminar (because EM was just successfully quantized and gravity is just another field theory, with a spin-2 carrier instead of a spin-1 one).
By the way, there are quite specific reasons one should have, in retrospect, expected a very low probability of the summer AI project to succeed, common to most estimates massaging the data to fit the deadlines and the available budget, rather than working upwards from the scope and risk analysis.
In 1958, a Nobel prize–winning scientist from the United Kingdom named Sir John Cockcroft gathered the nation’s media to announce that he and his research team had demonstrated controlled nuclear fusion in a giant machine nicknamed Zeta. [...] Commercially generated fusion power was two decades away, he predicted. Just months later Cockcroft was forced to admit that his observation of nuclear fusion and subsequent claims were an unfortunate mistake.
Just wanted to note that the case under consideration was quite unlike your garden-variety predictions, but more like an industrial project estimate. It gave quite concrete scope, effort and schedule estimates, something rarely done by long-term forecasters. So the relevant reference class would be ambitious scientific and industrial projects. One example is the failed Fifth Generation Computer Systems project. Another was the Manhattan project (not sure if there are circa 1941 estimates evailable for it). Not even the Moon landing project was in the same reference class, because it was clearly “just” an issue of (extremely massive) scaling up, rather than solving unknown problems. Not sure about the controlled nuclear fusion, maybe there are some predictions from 1950s available. Consider looking into Grand Challenges for further examples of the same reference class. Some other projects which come to mind: Feynman gives an account of Wheeler promising to present quantized gravity for the next seminar (because EM was just successfully quantized and gravity is just another field theory, with a spin-2 carrier instead of a spin-1 one).
By the way, there are quite specific reasons one should have, in retrospect, expected a very low probability of the summer AI project to succeed, common to most estimates massaging the data to fit the deadlines and the available budget, rather than working upwards from the scope and risk analysis.
Good thought.
— Tyler Hamilton, Mad Like Tesla, p. 32
Thanks! I like that reference class.