This is also because I tend to expect progress to be continuous, though potentially quite fast, and going from current AI to AGI in less than 5 years requires a very sharp discontinuity.
I object! I think your argument from extrapolating when milestones have been crossed is good, but it’s just one argument among many. There are other trends which, if extrapolated, get to AGI in less than five years. For example if you extrapolate the AI-compute trend and the GPT-scaling trends you get something like “GPT-5 will appear 3 years from now and be 3 orders of magnitude bigger and will be human-level at almost all text-based tasks.” No discontinuity required.
Daniel and SDM, what do you think of a bet with 78:22 odds (roughly 4:1) based on the differences in your distributions, i.e: If AGI happens before 2030, SDM owes Daniel $78. If AGI doesn’t happen before 2030, Daniel owes SDM $22.
This was calculated by:
Identifying the earliest possible date with substantial disagreement (in this case, 2030)
Finding the probability each person assigns to the date range of now to 2030:
According to this post, a bet based on the arithmetic mean of 2 differing probability estimates yields the same expected value for each participant. In this case, the mean is (5%+39%)/2=22% chance of AGI before 2030, equivalent to 22:78 odds.
$78 and $22 can be scaled appropriately for whatever size bet you’re comfortable with
The main issue for me is that if I win this bet I either won’t be around to collect on it, or I’ll be around but have much less need for money. So for me the bet you propose is basically “61% chance I pay SDM $22 in 10 years, 39% chance I get nothing.”
Jonas Vollmer helped sponsor my other bet on this matter, to get around this problem. He agreed to give me a loan for my possible winnings up front, which I would pay back (with interest) in 2030, unless I win in which case the person I bet against would pay it. Meanwhile the person I bet against would get his winnings from me in 2030, with interest, assuming I lose. It’s still not great because from my perspective it amounts to a loan with a higher interest rate basically, so it would be better for me to just take out a long-term loan. (The chance of never having to pay it back is nice, but I only never have to pay it back in worlds where I won’t care about money anyway.) Still though it was better than nothing so I took it.
The ‘progress will be continuous’ argument, to apply to our near future, does depend on my other assumptions—mainly that the breakthroughs on that list are separable, so agentive behaviour and long-term planning won’t drop out of a larger GPT by themselves and can’t be considered part of just ‘improving up language model accuracy’.
We currently have partial progress on human-level language comprehension, a bit on cumulative learning, but near zero on managing mental activity for long term planning, so if we were to suddenly reach human level on long-term planning in the next 5 years, that would probably involve a discontinuity, which I don’t think is very likely for the reasons given here.
If language models scale to near-human performance but the other milestones don’t fall in the process, and my initial claim is right, that gives us very transformative AI but not AGI. I think that the situation would look something like this:
So there would be 2 (maybe 3?) breakthroughs remaining. It seems like you think just scaling up a GPT will also resolve those other milestones, rather than just giving us human-like language comprehension. Whereas if I’m right and also those curves do extrapolate, what we would get at the end would be an excellent text generator, but it wouldn’t be an agent, wouldn’t be capable of long-term planning and couldn’t be accurately described as having a utility function over the states of the external world, and I don’t see any reason why trivial extensions of GPT would be able to do that either since those seem like problems that are just as hard as human-like language comprehension. GPT seems like it’s also making some progress on cumulative learning, though it might need some RL-based help with that, but none at all on managing mental activity for longterm planning or discovering new action sets.
As an additional argument, admittedly from authority—Stuart Russell also clearly sees human-like language comprehension as only one of several really hard and independent problems that need to be solved.
A humanlike GPT-N would certainly be a huge leap into a realm of AI we don’t know much about, so we could be surprised and discover that agentive behaviour and having a utility function over states of the external world spontaneously appears in a good enough language model, but that argument has to be made, and you need that argument to hold and GPT to keep scaling for us to reach AGI in the next five years, and I don’t see the conjunction of those two as that likely—it seems as though your argument rests solely on whether GPT scales or not, when there’s also this other conceptual premise that’s much harder to justify.
I’m also not sure if I’ve seen anyone make the argument that GPT-N will also give us these specific breakthroughs—but if you have reasons that GPT scaling would solve all the remaining barriers to AGI, I’d be interested to hear it. Note that this isn’t the same as just pointing out how impressive the results scaling up GPT could be—Gwern’s piece here, for example, seems to be arguing for a scenario more like what I’ve envisaged, where GPT-N ends up a key piece of some future AGI but just provides some of the background ‘world model’:
Models like GPT-3 suggest that large unsupervised models will be vital components of future DL systems, as they can be ‘plugged into’ systems to immediately provide understanding of the world, humans, natural language, and reasoning.
If GPT does scale, and we get human-like language comprehension in 2025, that will mean we’re moving up that list much faster, and in turn suggests that there might not be a large number of additional discoveries required to make the other breakthroughs, which in turn suggests they might also occur within the Deep Learning paradigm, and relatively soon. I think that if this happens, there’s a reasonable chance that when we do build an AGI a big part of its internals looks like a GPT, as gwern suggested, but by then we’re already long past simply scaling up existing systems.
Alternatively, perhaps you’re not including agentive behaviour in your definition of AGI—a par-human text generator for most tasks that isn’t capable of discovering new action sets or managing its mental activity is, I think a ‘mere’ transformative AI and not a genuine AGI.
I object! I think your argument from extrapolating when milestones have been crossed is good, but it’s just one argument among many. There are other trends which, if extrapolated, get to AGI in less than five years. For example if you extrapolate the AI-compute trend and the GPT-scaling trends you get something like “GPT-5 will appear 3 years from now and be 3 orders of magnitude bigger and will be human-level at almost all text-based tasks.” No discontinuity required.
Daniel and SDM, what do you think of a bet with 78:22 odds (roughly 4:1) based on the differences in your distributions, i.e: If AGI happens before 2030, SDM owes Daniel $78. If AGI doesn’t happen before 2030, Daniel owes SDM $22.
This was calculated by:
Identifying the earliest possible date with substantial disagreement (in this case, 2030)
Finding the probability each person assigns to the date range of now to 2030:
Daniel: 39%
SDM: 5%
Finding a fair bet
According to this post, a bet based on the arithmetic mean of 2 differing probability estimates yields the same expected value for each participant. In this case, the mean is (5%+39%)/2=22% chance of AGI before 2030, equivalent to 22:78 odds.
$78 and $22 can be scaled appropriately for whatever size bet you’re comfortable with
The main issue for me is that if I win this bet I either won’t be around to collect on it, or I’ll be around but have much less need for money. So for me the bet you propose is basically “61% chance I pay SDM $22 in 10 years, 39% chance I get nothing.”
Jonas Vollmer helped sponsor my other bet on this matter, to get around this problem. He agreed to give me a loan for my possible winnings up front, which I would pay back (with interest) in 2030, unless I win in which case the person I bet against would pay it. Meanwhile the person I bet against would get his winnings from me in 2030, with interest, assuming I lose. It’s still not great because from my perspective it amounts to a loan with a higher interest rate basically, so it would be better for me to just take out a long-term loan. (The chance of never having to pay it back is nice, but I only never have to pay it back in worlds where I won’t care about money anyway.) Still though it was better than nothing so I took it.
I’ll take that bet! If I do lose, I’ll be far too excited/terrified/dead to worry in any case.
The ‘progress will be continuous’ argument, to apply to our near future, does depend on my other assumptions—mainly that the breakthroughs on that list are separable, so agentive behaviour and long-term planning won’t drop out of a larger GPT by themselves and can’t be considered part of just ‘improving up language model accuracy’.
We currently have partial progress on human-level language comprehension, a bit on cumulative learning, but near zero on managing mental activity for long term planning, so if we were to suddenly reach human level on long-term planning in the next 5 years, that would probably involve a discontinuity, which I don’t think is very likely for the reasons given here.
If language models scale to near-human performance but the other milestones don’t fall in the process, and my initial claim is right, that gives us very transformative AI but not AGI. I think that the situation would look something like this:
If GPT-N reaches par-human:
So there would be 2 (maybe 3?) breakthroughs remaining. It seems like you think just scaling up a GPT will also resolve those other milestones, rather than just giving us human-like language comprehension. Whereas if I’m right and also those curves do extrapolate, what we would get at the end would be an excellent text generator, but it wouldn’t be an agent, wouldn’t be capable of long-term planning and couldn’t be accurately described as having a utility function over the states of the external world, and I don’t see any reason why trivial extensions of GPT would be able to do that either since those seem like problems that are just as hard as human-like language comprehension. GPT seems like it’s also making some progress on cumulative learning, though it might need some RL-based help with that, but none at all on managing mental activity for longterm planning or discovering new action sets.
As an additional argument, admittedly from authority—Stuart Russell also clearly sees human-like language comprehension as only one of several really hard and independent problems that need to be solved.
A humanlike GPT-N would certainly be a huge leap into a realm of AI we don’t know much about, so we could be surprised and discover that agentive behaviour and having a utility function over states of the external world spontaneously appears in a good enough language model, but that argument has to be made, and you need that argument to hold and GPT to keep scaling for us to reach AGI in the next five years, and I don’t see the conjunction of those two as that likely—it seems as though your argument rests solely on whether GPT scales or not, when there’s also this other conceptual premise that’s much harder to justify.
I’m also not sure if I’ve seen anyone make the argument that GPT-N will also give us these specific breakthroughs—but if you have reasons that GPT scaling would solve all the remaining barriers to AGI, I’d be interested to hear it. Note that this isn’t the same as just pointing out how impressive the results scaling up GPT could be—Gwern’s piece here, for example, seems to be arguing for a scenario more like what I’ve envisaged, where GPT-N ends up a key piece of some future AGI but just provides some of the background ‘world model’:
If GPT does scale, and we get human-like language comprehension in 2025, that will mean we’re moving up that list much faster, and in turn suggests that there might not be a large number of additional discoveries required to make the other breakthroughs, which in turn suggests they might also occur within the Deep Learning paradigm, and relatively soon. I think that if this happens, there’s a reasonable chance that when we do build an AGI a big part of its internals looks like a GPT, as gwern suggested, but by then we’re already long past simply scaling up existing systems.
Alternatively, perhaps you’re not including agentive behaviour in your definition of AGI—a par-human text generator for most tasks that isn’t capable of discovering new action sets or managing its mental activity is, I think a ‘mere’ transformative AI and not a genuine AGI.