I don’t know when you should stop. All I’m suggesting is that you not turn it on, without a time on which it is supposed to (automatically) switch off. In other words, you should stop it regularly, over and over again. This has the benefit of letting you consider the new information you have received, and decide how to respond to it. Perhaps your design will be “flawed”—and won’t have the risk of going ‘foom’ that you think it will (without further work to revise and change it—by you, before it is capable of ‘improving’). If you decide that it is risky, then the ‘intervention’ isn’t turning it off—it’s just not deciding to turn it back on (which maybe shouldn’t be automatic).
I don’t know when you should stop. All I’m suggesting is that you not turn it on, without a time on which it is supposed to (automatically) switch off. In other words, you should stop it regularly, over and over again. This has the benefit of letting you consider the new information you have received, and decide how to respond to it. Perhaps your design will be “flawed”—and won’t have the risk of going ‘foom’ that you think it will (without further work to revise and change it—by you, before it is capable of ‘improving’). If you decide that it is risky, then the ‘intervention’ isn’t turning it off—it’s just not deciding to turn it back on (which maybe shouldn’t be automatic).