See, this is the perfect encapsulation of what I’m saying—it could design a virus, sure. But when it didn’t understand parts of the economy, that’s all it would be—a design. Taking something from the design stage to the “physical, working product with validated processes that operate with sufficient consistency to achieve the desired outcome” is a vast, vast undertaking, one that requires intimate involvement with the physical world. Until that point is reached, it’s not a “kill all humans but fail to paperclip everyone” virus, it’s just a design concept. Nothing more. More and more I see those difficulties being elided over by hypothetical scenarios that skip straight from the design stage and presuppose that the implementation difficulties aren’t worth consideration, or that if they are they won’t serve as a valid impediment.
See, this is the perfect encapsulation of what I’m saying—it could design a virus, sure. But when it didn’t understand parts of the economy, that’s all it would be—a design. Taking something from the design stage to the “physical, working product with validated processes that operate with sufficient consistency to achieve the desired outcome” is a vast, vast undertaking, one that requires intimate involvement with the physical world. Until that point is reached, it’s not a “kill all humans but fail to paperclip everyone” virus, it’s just a design concept. Nothing more. More and more I see those difficulties being elided over by hypothetical scenarios that skip straight from the design stage and presuppose that the implementation difficulties aren’t worth consideration, or that if they are they won’t serve as a valid impediment.