Doesn’t that discrepancy (how much answers vary between different ways of asking the question) tell you that the median AI researcher who published at these conferences hasn’t thought about this question sufficiently and/or sanely?
We know that AI expertise and AI forecasting are separate skills and that we shouldn’t expect AI researchers to be skilled at the latter. So even if researchers have thought sufficiently and sanely about the question of “what kinds of capabilities are we still missing that would be required for AGI”, they would still be lacking the additional skill of “how to translate those missing pieces into a timeline estimate”.
Suppose that a researcher’s conception of current missing pieces is a mental object M, their timeline estimate is a probability function P, and their forecasting expertise F is a function that maps M to P. In this model, F can be pretty crazy, creating vast differences in P depending how you ask, while M is still solid.
I think the implication is that these kinds of surveys cannot tell us anything very precise such as “is 15 years more likely than 23”, but we can use what we know about the nature of F in untrained individuals to try to get a sense of what M might be like. My sense is that answers like “20-93 years” often translate to “I think there are major pieces missing and I have no idea of how to even start approaching them, but if I say something that feels like a long time, maybe someone will figure it out in that time”, “0-5 years” means “we have all the major components and only relatively straightforward engineering work is needed for them”, and numbers in between correspond to Ms that are, well, somewhere in between those.
Suppose that a researcher’s conception of current missing pieces is a mental object M, their timeline estimate is a probability function P, and their forecasting expertise F is a function that maps M to P. In this model, F can be pretty crazy, creating vast differences in P depending how you ask, while M is still solid.
Good point. This would be reasonable if you think someone can be super bad at F and still great at M.
Still, I think estimating “how big is this gap?” and “how long will it take to cross it?” might quite related, so I expect the skills to be correlated or even strongly correlated.
I think their relationship depends on whether crossing the gap requires grind or insight. If it’s mostly about grind then a good expert will be able to estimate it, but insight tends to unpredictable by nature.
Another way of looking at my comment above would be that timelines of less than 5 years would imply the remaining steps mostly requiring grind, and timelines of 20+ years would imply that some amount of insight is needed.
We know that AI expertise and AI forecasting are separate skills and that we shouldn’t expect AI researchers to be skilled at the latter. So even if researchers have thought sufficiently and sanely about the question of “what kinds of capabilities are we still missing that would be required for AGI”, they would still be lacking the additional skill of “how to translate those missing pieces into a timeline estimate”.
Suppose that a researcher’s conception of current missing pieces is a mental object M, their timeline estimate is a probability function P, and their forecasting expertise F is a function that maps M to P. In this model, F can be pretty crazy, creating vast differences in P depending how you ask, while M is still solid.
I think the implication is that these kinds of surveys cannot tell us anything very precise such as “is 15 years more likely than 23”, but we can use what we know about the nature of F in untrained individuals to try to get a sense of what M might be like. My sense is that answers like “20-93 years” often translate to “I think there are major pieces missing and I have no idea of how to even start approaching them, but if I say something that feels like a long time, maybe someone will figure it out in that time”, “0-5 years” means “we have all the major components and only relatively straightforward engineering work is needed for them”, and numbers in between correspond to Ms that are, well, somewhere in between those.
Good point. This would be reasonable if you think someone can be super bad at F and still great at M.
Still, I think estimating “how big is this gap?” and “how long will it take to cross it?” might quite related, so I expect the skills to be correlated or even strongly correlated.
I think their relationship depends on whether crossing the gap requires grind or insight. If it’s mostly about grind then a good expert will be able to estimate it, but insight tends to unpredictable by nature.
Another way of looking at my comment above would be that timelines of less than 5 years would imply the remaining steps mostly requiring grind, and timelines of 20+ years would imply that some amount of insight is needed.