Dog ate my homework excuse, in this particular case. Maximizing real world paperclips when you act upon sensory input is an incredibly tough problem, and it gets zillion times tougher still if you want that agent to start adding new hardware to itself.
edit:
Simultaneously, designing new hardware, or new weapons, or the like, within the simulation space, without proper AGI, is a solved problem. This real world paperclip maximizer has to be more inventive than the less general tools running on same hardware, to pose any danger.
The real world goals are ontologically basic to humans, and seem simple to people with little knowledge of the field. The fact is that doing things to reality based on the sensory input is a very tough extra problem separate from ‘cross domain optimization’. Even if you had some genie that solves any mathematically defined problems, it is still incredibly difficult to get it to paperclip maximize, even though you can use this genie to design anything.
Dog ate my homework excuse, in this particular case. Maximizing real world paperclips when you act upon sensory input is an incredibly tough problem, and it gets zillion times tougher still if you want that agent to start adding new hardware to itself.
edit:
Simultaneously, designing new hardware, or new weapons, or the like, within the simulation space, without proper AGI, is a solved problem. This real world paperclip maximizer has to be more inventive than the less general tools running on same hardware, to pose any danger.
The real world goals are ontologically basic to humans, and seem simple to people with little knowledge of the field. The fact is that doing things to reality based on the sensory input is a very tough extra problem separate from ‘cross domain optimization’. Even if you had some genie that solves any mathematically defined problems, it is still incredibly difficult to get it to paperclip maximize, even though you can use this genie to design anything.