Unless, of course, the superintelligence will be at least as smart as Robert, capable of figuring out how to screen away reality’s messiness (= the ramp’s complicated physics) and deliberately pick the implementation pipelines that are highly predictable or controllable.
Robert used empirical data a bunch to solve the problem. Getting empirical data for nano-tech seems to involve a bunch of difficult experiments that take lots of time and resources to run.
You could argue that the AI could use data from previous experiments to figure this out, but I expect that very specific experiments would need to be run. (Also, you might need to create new tech to even run these experiments, which in itself requires empirical data.)
Is anyone worried about AI one-shotting comprehensive nano-technology? It can make as many tries as it wants, and in fact, we’ll be giving it as many tries as we can.
You can also take this as a metaphor for why AI probably won’t be able to one-shot comprehensive nanotechnology.
Unless, of course, the superintelligence will be at least as smart as Robert, capable of figuring out how to screen away reality’s messiness (= the ramp’s complicated physics) and deliberately pick the implementation pipelines that are highly predictable or controllable.
“Doing anything at first try is impossible” would be the takeaway if no-one had actually succeeded. “~5% success rate, the correct approach is possible to figure out and it robustly leads to success” makes the takeaway very different. Yes, it’s hard, but you can in fact figure it out.
Robert used empirical data a bunch to solve the problem. Getting empirical data for nano-tech seems to involve a bunch of difficult experiments that take lots of time and resources to run.
You could argue that the AI could use data from previous experiments to figure this out, but I expect that very specific experiments would need to be run. (Also, you might need to create new tech to even run these experiments, which in itself requires empirical data.)
Is anyone worried about AI one-shotting comprehensive nano-technology? It can make as many tries as it wants, and in fact, we’ll be giving it as many tries as we can.
For sure, though all that demonstrates is that it’ll have to import one of the already trained nanotech-focused models