Stopping is one of the simplest possible desires—and you have a better chance of being able to program that in than practically anything else.
I gave several proposals to deal with the possible issues associated with stopping at an unknown point resulting in plans beyond that point still being executed by minions or sub-contractors—including scheduling shutdowns in advance, ensuring a period of quiescence before the shutdown—and not running for extended periods of time.
Well, I think I went into most of this already in my “stopping superintelligence” essay.
Stopping is one of the simplest possible desires—and you have a better chance of being able to program that in than practically anything else.
I gave several proposals to deal with the possible issues associated with stopping at an unknown point resulting in plans beyond that point still being executed by minions or sub-contractors—including scheduling shutdowns in advance, ensuring a period of quiescence before the shutdown—and not running for extended periods of time.
It does seem to be a safety precaution that could reduce the consequences of some possible flaws in an AI design.