I still prefer my analysis above: Fusion is not a case of Shorty being wrong, because a steelman Shorty wouldn’t have predicted that we’d get fusion soon. Why? Because we don’t have the key variables. Why? Because controlling the plasma is one of the key variables, and the sun has near-perfect control, whereas we are trying to substitute with various designs which may or may not work.
Shorty is actually arguing for TAI much sooner than 20 years from now; if TAI comes around the HBHL milestone then it could happen any day now, it’s just a matter of spending a billion dollars on compute and then iterating a few times to work out the details, wright-brothers style. Of course we shouldn’t think Shorty is probably correct here; the truth is probably somewhere in between. (Unless we do more historical analyses and find that the case of flight is truly representative of the reference class AI fits in, in which case ho boy singularity here we come)
And yeah, the main purpose of the OP was to argue that certain anti-short-timelines arguments are bogus; this issue of whether timelines are actually short or long is secondary and the case of flight is just one case study, of limited evidential import.
I do take your point that maybe Longs’ argument was drawing on intuitions of the sort you are sketching out. In other words, maybe there’s a steelman of the arguments I think are bogus, such that they become non-bogus. I already agree this is true in at least one way (see Part 3). I like your point about large projects—insofar as we think of AI in that reference class, it seems like our timelines should be “Take whatever the experts say and then double it.” But if we had done this for flight we would have been disastrously wrong. I definitely want to think, talk, and hear more about these issues… I’d like to have a model of what sorts of technologies are like fusion and what sort are like flight, and why.
My own (hinted at in the OP) was going to be something like “When your basic theory of a design problem is developed enough that you have identified the key variables, and there is a natural design that solves the problem in a similar way to the thing you are trying to build, then you can predict roughly when the problem will be solved by saying that it’ll happen around the time that parity-with-the-natural-design is reached in the key variables. What are key variables? I’m not sure how to define them, but one property that seems maybe important is that the design problem becomes easier when you have more of the key variables.”
Another thing worth mentioning is that probably having a healthy competition between different smart people is important. The Wright brother succeeded but there were several other groups around the same time also trying to build flying machines, who were less successful (or who took longer to succeed). If instead there had been one big government-funded project, there’s more room for human error and the usual failures to cause cost overruns and delays. (OTOH having more funding might have made it happen sooner? IDK). In the case of AI, there are enough different projects full of enough smart people working on the problem that I don’t think this is a major constraint. I’d be curious to hear more about the case of fusion. I’ve heard some people say that actually it could have been achieved by now if only it had more funding, and I think I’ve heard other people say that it could have been achieved by now if it was handled by a competitive market instead of a handful of bureaucracies (though I may be misremembering that, maybe no one said that).
I still prefer my analysis above: Fusion is not a case of Shorty being wrong, because a steelman Shorty wouldn’t have predicted that we’d get fusion soon. Why? Because we don’t have the key variables. Why? Because controlling the plasma is one of the key variables, and the sun has near-perfect control, whereas we are trying to substitute with various designs which may or may not work.
Shorty is actually arguing for TAI much sooner than 20 years from now; if TAI comes around the HBHL milestone then it could happen any day now, it’s just a matter of spending a billion dollars on compute and then iterating a few times to work out the details, wright-brothers style. Of course we shouldn’t think Shorty is probably correct here; the truth is probably somewhere in between. (Unless we do more historical analyses and find that the case of flight is truly representative of the reference class AI fits in, in which case ho boy singularity here we come)
And yeah, the main purpose of the OP was to argue that certain anti-short-timelines arguments are bogus; this issue of whether timelines are actually short or long is secondary and the case of flight is just one case study, of limited evidential import.
I do take your point that maybe Longs’ argument was drawing on intuitions of the sort you are sketching out. In other words, maybe there’s a steelman of the arguments I think are bogus, such that they become non-bogus. I already agree this is true in at least one way (see Part 3). I like your point about large projects—insofar as we think of AI in that reference class, it seems like our timelines should be “Take whatever the experts say and then double it.” But if we had done this for flight we would have been disastrously wrong. I definitely want to think, talk, and hear more about these issues… I’d like to have a model of what sorts of technologies are like fusion and what sort are like flight, and why.
I like your suggestions:
My own (hinted at in the OP) was going to be something like “When your basic theory of a design problem is developed enough that you have identified the key variables, and there is a natural design that solves the problem in a similar way to the thing you are trying to build, then you can predict roughly when the problem will be solved by saying that it’ll happen around the time that parity-with-the-natural-design is reached in the key variables. What are key variables? I’m not sure how to define them, but one property that seems maybe important is that the design problem becomes easier when you have more of the key variables.”
Another thing worth mentioning is that probably having a healthy competition between different smart people is important. The Wright brother succeeded but there were several other groups around the same time also trying to build flying machines, who were less successful (or who took longer to succeed). If instead there had been one big government-funded project, there’s more room for human error and the usual failures to cause cost overruns and delays. (OTOH having more funding might have made it happen sooner? IDK). In the case of AI, there are enough different projects full of enough smart people working on the problem that I don’t think this is a major constraint. I’d be curious to hear more about the case of fusion. I’ve heard some people say that actually it could have been achieved by now if only it had more funding, and I think I’ve heard other people say that it could have been achieved by now if it was handled by a competitive market instead of a handful of bureaucracies (though I may be misremembering that, maybe no one said that).