The much stronger issue he raised is that it may well be that outside imagination and fiction, there is no monolithic ‘intelligence’ thing, and the ‘benevolent ruler of the earth’ software is then more dangerous than e.g. software that uses search and hill climbing to design better microchips, or design cures for diseases, or the like, without being ‘intelligent’ in the science fictional sense, and while lacking any form of real world volition. The “benevolent ruler of the earth” software would then, also, fail to provide any superior technical solutions to our problems, as this “intelligence” does not bring any important advantage over the algorithms normally used for problem solving.
The chip improver would spit out the blueprints, the cure designer would spit out the projected molecular images and DNA sequences, etc—no oracle crap with the ‘utility’ of making people understand something, which appears both near-impossible and entirely unnecessary.
Outside of mystic circles, it is fairly uncontroversial that it is in principle possible to construct out of matter an object capable of general intelligence. Proof is left to the reader.
The much stronger issue he raised is that it may well be that outside imagination and fiction, there is no monolithic ‘intelligence’ thing, and the ‘benevolent ruler of the earth’ software is then more dangerous than e.g. software that uses search and hill climbing to design better microchips, or design cures for diseases, or the like, without being ‘intelligent’ in the science fictional sense, and while lacking any form of real world volition. The “benevolent ruler of the earth” software would then, also, fail to provide any superior technical solutions to our problems, as this “intelligence” does not bring any important advantage over the algorithms normally used for problem solving.
The chip improver would spit out the blueprints, the cure designer would spit out the projected molecular images and DNA sequences, etc—no oracle crap with the ‘utility’ of making people understand something, which appears both near-impossible and entirely unnecessary.
Outside of mystic circles, it is fairly uncontroversial that it is in principle possible to construct out of matter an object capable of general intelligence. Proof is left to the reader.