As the human race, we are delicately positioned. We have the… ability… to create machines that will not only be as intelligent as humans but that will go on to be far more intelligent still. This will spell the end of the human race as we know it. Is that what we want? Should we not at least have an international body monitoring and even controlling what goes on?
When the first nuclear bombs were dropped on Japan, killing thousands of people, we took stock of our actions and realised the threat that such weapons posed to our existence. Despite the results achieved by the Hiroshima and Nagasaki bombs, even deadlier nuclear bombs have been built, much more powerful, much more accurate and much more intelligent. But with nuclear weapons we saw what they could do and we gave ourselves another chance.
With intelligent machines we will not get a second chance. Once the first powerful machine… is switched on, we will most likely not get the opportunity to switch it back off again. We will have started a time bomb ticking on the human race, and we will be unable to switch it off.
Kevin Warwick, March of the Machines (1997)