Thanks, this is my favorite answer so far. [EDIT: Now Paul’s is my favorite.] It’s sorta what I had in mind with my list of candidates above. I guess your points #2 and #3 are the ones I’m skeptical of.
Re point 2: If we’ve automated everything, including building factories and entire production cycles, (a) doesn’t that involve figuring out how to make computers do a huge variety of tasks, many of which are quite intellectually difficult, such that plausibly the easiest way to do this is to create general AI rather than loads of narrow AIs (or else R&D tools that automate the automation process for us, but in that case they’d probably also help us get AGI) and (b) even setting aside that, couldn’t we string all these narrow AIs together to make an AGI?
Re point 3: Haven’t we come close to automating everything multiple times in the past century? I don’t know, but I would guess that 90%+ of the industrial/manufacturing jobs done by humans even just 50 years ago are now done by machines. But this 90% automation didn’t lead to a near-doubling of GWP growth rates.
I don’t know, but I would guess that 90%+ of the industrial/manufacturing jobs done by humans even just 50 years ago are now done by machines. But this 90% automation didn’t lead to a near-doubling of GWP growth rates.
90% automation only gives a ~10x increase in per-worker productivity in manufacturing. Since manufacturing is only a fraction of GWP, a 10x productivity increase only makes GWP (per capita) a few times larger. Take humans out of the process completely and the bottleneck is gone. The feedback loop is only constrained the availability of resources.
Good point. I guess I just find it implausible that humans will be COMPLETELY out of the loop prior to AGI. Some parts of the loops involve agenty tasks in which you draw on general world-knowledge to make novel plans and strategies and then execute them learning and adapting constantly.
I’m not sure if the agenty tasks you have in mind are considered part of manufacturing per se or business management. My impression from above is that production work and factory construction is being automated but design/engineering and business management are not. I’m not sure, but it does seem likely that humans could be out of the loop without AGI. (Though of course AGI could happen before narrow AI actually achieves this level in practice).
Thanks, this is my favorite answer so far. [EDIT: Now Paul’s is my favorite.] It’s sorta what I had in mind with my list of candidates above. I guess your points #2 and #3 are the ones I’m skeptical of.
Re point 2: If we’ve automated everything, including building factories and entire production cycles, (a) doesn’t that involve figuring out how to make computers do a huge variety of tasks, many of which are quite intellectually difficult, such that plausibly the easiest way to do this is to create general AI rather than loads of narrow AIs (or else R&D tools that automate the automation process for us, but in that case they’d probably also help us get AGI) and (b) even setting aside that, couldn’t we string all these narrow AIs together to make an AGI?
Re point 3: Haven’t we come close to automating everything multiple times in the past century? I don’t know, but I would guess that 90%+ of the industrial/manufacturing jobs done by humans even just 50 years ago are now done by machines. But this 90% automation didn’t lead to a near-doubling of GWP growth rates.
90% automation only gives a ~10x increase in per-worker productivity in manufacturing. Since manufacturing is only a fraction of GWP, a 10x productivity increase only makes GWP (per capita) a few times larger. Take humans out of the process completely and the bottleneck is gone. The feedback loop is only constrained the availability of resources.
Good point. I guess I just find it implausible that humans will be COMPLETELY out of the loop prior to AGI. Some parts of the loops involve agenty tasks in which you draw on general world-knowledge to make novel plans and strategies and then execute them learning and adapting constantly.
I’m not sure if the agenty tasks you have in mind are considered part of manufacturing per se or business management. My impression from above is that production work and factory construction is being automated but design/engineering and business management are not. I’m not sure, but it does seem likely that humans could be out of the loop without AGI. (Though of course AGI could happen before narrow AI actually achieves this level in practice).