Appreciate the super thorough response, Aidan. You’re right it turns out that some of the people I mentioned like Gates who are concerned about AI aren’t on record with particularly short timelines.
I agree with FeepingCreature’s comment that some of these surveys and sources are starting to feel quite dated. Apparently GovAI is currently working on replicating the 2015 Grace et al. survey which I’m very much looking forward to. The Bostrom survey you linked to is even older than that—from 2012~13. At least the Gruetzemacher survey is from 2018.
(tangent) One thing I see in some of the surveys as well as in discussion that bothers me is an emphasis on automation of 100% of tasks or 99% of tasks. I think Holden made an important point in his discussion of a weak point in his Most Important Century series, that transformative AI need not depend on everything being automated. In fact, just having full-automation of a small number of specific activities could be all that’s needed:
...it’s also worth noting that the extreme levels of automation need not apply to the whole economy: extreme automation for a relatively small set of activities could be sufficient to reach the conclusions in the series.
For example, it might be sufficient for AI systems to develop increasingly efficient (a) computers; (b) solar panels (for energy); (c) mining and manufacturing robots; (d) space probes (to build more computers in space, where energy and metal are abundant). That could be sufficient (via feedback loop) for explosive growth in available energy, materials and computing power, and there are many ways that such growth could be transformative.
For example and in particular, it could lead to:
Misaligned AI with access to dangerous amounts of materials and energy.
Appreciate the super thorough response, Aidan. You’re right it turns out that some of the people I mentioned like Gates who are concerned about AI aren’t on record with particularly short timelines.
I agree with FeepingCreature’s comment that some of these surveys and sources are starting to feel quite dated. Apparently GovAI is currently working on replicating the 2015 Grace et al. survey which I’m very much looking forward to. The Bostrom survey you linked to is even older than that—from 2012~13. At least the Gruetzemacher survey is from 2018.
(tangent) One thing I see in some of the surveys as well as in discussion that bothers me is an emphasis on automation of 100% of tasks or 99% of tasks. I think Holden made an important point in his discussion of a weak point in his Most Important Century series, that transformative AI need not depend on everything being automated. In fact, just having full-automation of a small number of specific activities could be all that’s needed: