Something like that is what I had in mind, but note that:
It requires humans to fail to see the AGI “spy” that’s embedded into every single powerful computing system, and fail to see this for years. Gwern was assuming humans would catch on in days, so he had his AGI scramble to avoid dying before the nanobots strike.
“Surviving a short amount of time without human input” is not enough; the robots need to be good enough to build more robots (and better robots). This involves the robots being good enough to do essentially every part of the manufacturing economy; we are very far away from this, and a company that does it in a year is not so plausible (and would raise alarm bells fast for anyone who thinks about AI risk). You’re gonna need robot plumbers, robot electricians, etc. You’ll need robots building cooling systems for the construction plants that manufacture robots. You’ll need robots to do the smelting of metals, to drive things from factory A to factory B, to fill the gas in the trucks they are driving, to repair the gasoline lines that supply the gas. Robots will operate fork lifts and cranes. It really sounds roughly “human-body complete”.
Something like that is what I had in mind, but note that:
It requires humans to fail to see the AGI “spy” that’s embedded into every single powerful computing system, and fail to see this for years. Gwern was assuming humans would catch on in days, so he had his AGI scramble to avoid dying before the nanobots strike.
“Surviving a short amount of time without human input” is not enough; the robots need to be good enough to build more robots (and better robots). This involves the robots being good enough to do essentially every part of the manufacturing economy; we are very far away from this, and a company that does it in a year is not so plausible (and would raise alarm bells fast for anyone who thinks about AI risk). You’re gonna need robot plumbers, robot electricians, etc. You’ll need robots building cooling systems for the construction plants that manufacture robots. You’ll need robots to do the smelting of metals, to drive things from factory A to factory B, to fill the gas in the trucks they are driving, to repair the gasoline lines that supply the gas. Robots will operate fork lifts and cranes. It really sounds roughly “human-body complete”.