resources, whether physical or computational. Presumably the AI is programmed to utilize resources in a parsimonious manner, with terms governing various applications of the resources, including powering the AI, and deciding on what to do. If the AI is programmed to limit what it does at some large but arbitrary point, because we don’t want it taking over the universe or whatever, then this point might end up actually being before we want it to stop doing whatever it’s doing.
resources, whether physical or computational. Presumably the AI is programmed to utilize resources in a parsimonious manner, with terms governing various applications of the resources, including powering the AI, and deciding on what to do. If the AI is programmed to limit what it does at some large but arbitrary point, because we don’t want it taking over the universe or whatever, then this point might end up actually being before we want it to stop doing whatever it’s doing.
That doesn’t sound like an expected utility maximizer.