(Not very sure I understood your description right, but here is my take:)
I think your proposal is not explaining some crucial steps, which are in fact hard. In particular, I understood it as “you have AI which can give you blueprints for nano sized machines”. But I think we already have some blueprints, this isn’t an issue. How we assemble them is an issue.
I expect that there will be more issues like this that you would find if you tried writing the plan in more detail.
However, I share the general sentiment behind your post—I also don’t understand why you can’t get some pivotal act by combining human intelligence with some narrow AI. I expect that Eliezer have tried to come up with such combinations and came away with some general takeaways on this being not realistic. But I haven’t done this exercise, so it seems not obvious to me. Perhaps it would be beneficial if many more people tried doing the exercise and then communicated the takeaways.
(Not very sure I understood your description right, but here is my take:)
I think your proposal is not explaining some crucial steps, which are in fact hard. In particular, I understood it as “you have AI which can give you blueprints for nano sized machines”. But I think we already have some blueprints, this isn’t an issue. How we assemble them is an issue.
I expect that there will be more issues like this that you would find if you tried writing the plan in more detail.
However, I share the general sentiment behind your post—I also don’t understand why you can’t get some pivotal act by combining human intelligence with some narrow AI. I expect that Eliezer have tried to come up with such combinations and came away with some general takeaways on this being not realistic. But I haven’t done this exercise, so it seems not obvious to me. Perhaps it would be beneficial if many more people tried doing the exercise and then communicated the takeaways.
I think it would be!