Once an AI reaches human level intelligence and can run multiple instances in parallel it doesn’t require a human debugger but can be debugged by another AGI instance.
That what human level AGI is per definition about.
That’s like saying a paranoid schizophrenic can solve his problems by performing psychoanalysis against a copy of himself. However I doubt another paranoid schizophrenic would be able to provide very good or effective therapy.
In short you are assuming a working AGI exists to do the debugging, but the setup is that the AGI itself is flawed! Nearly every single engineering project ever demonstrates that things don’t work on the first try, and when an engineered thing fails it fails spectacularly. Biology is somewhat unique in its ability to recover from errors, but only specialized categories of errors that it was trained to overcome in its evolutionary environment.
As an engineering professional I find it extremely unlikely that an AI could successfully achieve hard take-off on the first try. So unlikely that it is not even worth thinking about—LHC creating black holes level of unlikely. When developing AI it would be prudent to seed the simulated environments it is developed and tested inside of with honeypots, and see if it attempts any of the kinds of failure modes x-risk people are worried about. Then and there with an actual engineering prototype would be an appropriate time to consider engineering proactive safeguards. But until then it seems a bit like worrying about aviation safety in the 17th century and then designing a bunch of safety equipment for massive passenger hot air balloons that end up being of zero use in the fixed wing aeroplane days of the 20th century.
However I doubt another paranoid schizophrenic would be able to provide very good or effective therapy.
I don’t see a reason for why being a paranoid schizophrenic makes a person unable to lead another person through a CBT process.
As an engineering professional I find it extremely unlikely that an AI could successfully achieve hard take-off on the first try.
The assumption of an AGI achieving hard take-off on the first try is not required for the main arguments about AGI risk being a problem.
The fact that the AGI first doesn’t engage in particular harmful action X doesn’t imply that if you let it self modify a lot it still doesn’t engage in action X.
Once an AI reaches human level intelligence and can run multiple instances in parallel it doesn’t require a human debugger but can be debugged by another AGI instance.
That what human level AGI is per definition about.
That’s like saying a paranoid schizophrenic can solve his problems by performing psychoanalysis against a copy of himself. However I doubt another paranoid schizophrenic would be able to provide very good or effective therapy.
In short you are assuming a working AGI exists to do the debugging, but the setup is that the AGI itself is flawed! Nearly every single engineering project ever demonstrates that things don’t work on the first try, and when an engineered thing fails it fails spectacularly. Biology is somewhat unique in its ability to recover from errors, but only specialized categories of errors that it was trained to overcome in its evolutionary environment.
As an engineering professional I find it extremely unlikely that an AI could successfully achieve hard take-off on the first try. So unlikely that it is not even worth thinking about—LHC creating black holes level of unlikely. When developing AI it would be prudent to seed the simulated environments it is developed and tested inside of with honeypots, and see if it attempts any of the kinds of failure modes x-risk people are worried about. Then and there with an actual engineering prototype would be an appropriate time to consider engineering proactive safeguards. But until then it seems a bit like worrying about aviation safety in the 17th century and then designing a bunch of safety equipment for massive passenger hot air balloons that end up being of zero use in the fixed wing aeroplane days of the 20th century.
I don’t see a reason for why being a paranoid schizophrenic makes a person unable to lead another person through a CBT process.
The assumption of an AGI achieving hard take-off on the first try is not required for the main arguments about AGI risk being a problem.
The fact that the AGI first doesn’t engage in particular harmful action X doesn’t imply that if you let it self modify a lot it still doesn’t engage in action X.
We are clearly talking past each other and I’ve lost the will to engage further, sorry.