I like the bird-plane analogy. I kind of had the same idea, but for slightly different reason: just as man made flying machines can be superior to birds in a lot of aspects, man made ai will most likely can be superior to a human mind in a similar way.
Regarding your specific points: they may be valid, however, we do not know at which point in time we are talking about flying or AI: Probably a lot of similar arguments could have been made by Leonardo da Vinci when he was designing his flying machine; most likely he understood a lot more about birds and the way they fly than any of his contemporaries or predecessors; yet, he had no chance to succeed for at least three additional centuries. So are we in the era of the Wright Brothers of A.I., or are we still only at da Vinci’s?
I personally think the former is more likely, but I believe the probability of the second one is a lot greater than zero.
So are we in the era of the Wright Brothers of A.I., or are we still only at da Vinci’s?
That depends on how close we are to having the key variables at the human-equivalent level. I think the key variables are size and training time, so the relevant milestone is the HBHL. We are currently just a few orders of magnitude away from the HBHL milestone, depending on how you calculate it. GPT-3 was about three orders of magnitude smaller than the human brain, for example. Given how fast we cross orders of magnitude these days, that means we are in the era of the Wright brothers.
I think this assumes the conclusion—it assumes that we know enough about intelligence to know what the key variables are and how effective they can be at compensating for other variables. Da Vinci could have argued how much more efficient his new designs were getting or how much better his new wings were but none of his designs could have worked no matter how much better he made them.
I don’t disagree with you in general but I think the effect of Longs’ argument should be to stretch out the probability distribution.
Sorry for not making this clear—I agree the probability distribution should be stretched out. I think Longs’ argument is bogus, in the sense of being basically zero evidence for its conclusion as currently stated—but the conclusion may still be right, because there are more fleshed-out arguments one could make that are much better. For example, as you point out, I didn’t really investigate the issue of whether or not Shorty properly identified the key variables in the case of TAI. I think a really good way to critique Shorty is to argue that those aren’t the key variables, or at least that they probably aren’t. As it happens, I do think those are probably the key variables, but I haven’t argued for that yet, and I am still rather uncertain.
(I think Long’s argument that those aren’t the key variables is bad though. It’s too easy to point to things we currently don’t understand; see e.g. how many things we didn’t understand about birds or flight in 1900! Better would be to have an alternative theory of what the key variables are, or a more direct rebuttal of Shorty’s theory of key variables by showing that it makes some incorrect prediction or something.)
I like the bird-plane analogy. I kind of had the same idea, but for slightly different reason: just as man made flying machines can be superior to birds in a lot of aspects, man made ai will most likely can be superior to a human mind in a similar way.
Regarding your specific points: they may be valid, however, we do not know at which point in time we are talking about flying or AI: Probably a lot of similar arguments could have been made by Leonardo da Vinci when he was designing his flying machine; most likely he understood a lot more about birds and the way they fly than any of his contemporaries or predecessors; yet, he had no chance to succeed for at least three additional centuries. So are we in the era of the Wright Brothers of A.I., or are we still only at da Vinci’s?
I personally think the former is more likely, but I believe the probability of the second one is a lot greater than zero.
That depends on how close we are to having the key variables at the human-equivalent level. I think the key variables are size and training time, so the relevant milestone is the HBHL. We are currently just a few orders of magnitude away from the HBHL milestone, depending on how you calculate it. GPT-3 was about three orders of magnitude smaller than the human brain, for example. Given how fast we cross orders of magnitude these days, that means we are in the era of the Wright brothers.
I think this assumes the conclusion—it assumes that we know enough about intelligence to know what the key variables are and how effective they can be at compensating for other variables. Da Vinci could have argued how much more efficient his new designs were getting or how much better his new wings were but none of his designs could have worked no matter how much better he made them.
I don’t disagree with you in general but I think the effect of Longs’ argument should be to stretch out the probability distribution.
Sorry for not making this clear—I agree the probability distribution should be stretched out. I think Longs’ argument is bogus, in the sense of being basically zero evidence for its conclusion as currently stated—but the conclusion may still be right, because there are more fleshed-out arguments one could make that are much better. For example, as you point out, I didn’t really investigate the issue of whether or not Shorty properly identified the key variables in the case of TAI. I think a really good way to critique Shorty is to argue that those aren’t the key variables, or at least that they probably aren’t. As it happens, I do think those are probably the key variables, but I haven’t argued for that yet, and I am still rather uncertain.
(I think Long’s argument that those aren’t the key variables is bad though. It’s too easy to point to things we currently don’t understand; see e.g. how many things we didn’t understand about birds or flight in 1900! Better would be to have an alternative theory of what the key variables are, or a more direct rebuttal of Shorty’s theory of key variables by showing that it makes some incorrect prediction or something.)