To retain control, a human manager will need to grant the AGI autonomy on larger timescales in proportion to the AGI’s greater intelligence and speed, giving it bigger and more abstract hierachical goals. As an example, eventually you get to a situation where the CEO just instructs the AGI employees to optimize the bank account directly.
Nitpick: you mean “optimize shareholder value directly.” Keeping the account balances at an appropriate level is the CFO’s job.
Nitpick: you mean “optimize shareholder value directly.” Keeping the account balances at an appropriate level is the CFO’s job.