If it has a bug in its utility function, it won’t want to fix it.
If it has a bug in its bug-detection-and-fixing techniques, you can guess what happens.
So, no, you can’t rely on the AGI to fix itself, unless you’re certain that the bugs are localised in regions that will be fixed.
So: bug-free is not needed—and a controlled ascent is possible.
The unreferenced “hubris verging on sheer insanity” asumption seems like a straw man—nobody assumed that in the first place.
If it has a bug in its utility function, it won’t want to fix it.
If it has a bug in its bug-detection-and-fixing techniques, you can guess what happens.
So, no, you can’t rely on the AGI to fix itself, unless you’re certain that the bugs are localised in regions that will be fixed.
So: bug-free is not needed—and a controlled ascent is possible.
The unreferenced “hubris verging on sheer insanity” asumption seems like a straw man—nobody assumed that in the first place.