Right, but general optimization power is also really vague.
This sounds unattractive at the outset, but could one relate optimization power in economic terms? A machine super-optimizer would most likely have to become an economic power on its way to subverting the world’s industrial infrastructure. If one can relate optimization power to economic power, then one could make a strong argument for the inability of human civilization to control a machine super-optimizer.
This sounds unattractive at the outset, but could one relate optimization power in economic terms? A machine super-optimizer would most likely have to become an economic power on its way to subverting the world’s industrial infrastructure. If one can relate optimization power to economic power, then one could make a strong argument for the inability of human civilization to control a machine super-optimizer.