This is really remarkable! Thanks for posting, I don’t know when I would have seen it otherwise.
Partway through, I Googled a couple passages to quickly confirm that it wasn’t some “modern blog post in an old-timey style”—it feels more specific and prescient to me than I remember Erewhon being (though Gwern’s comment makes me think I should go back and look again, or check Darwin Among the Machines).
Evans’ predictions on Trost’s arguments and overall vibe are also impressive, IMO. (E.g. “if it were not for your incurable dilettanteism in science as in all other things—if you had once understood the action of any delicate machine...” ~= “actually training models gives you the insight that AI risk isn’t a thing.”) I wonder how much Evans-vs.-Trost-style conversation was already going on.
Interesting how this part turned out:
how do I know that they may not be ultimately made to carry, or may not in themselves evolve, conditions of self-supply, self-repair, and reproduction
It doesn’t seem like Evans (or others at the time?) anticipated that instead of individual machines reproducing like humans, they could just run the factories / mines / etc. that produce them. Which is an odd oversight, since by that time machines were certainly used in factories to make more machines—it seems like the “reproduction via external construction” prediction should have been a lot easier to make than the “reasoning machines” prediction.
“Heaven forbid! They seem to be flying about in the air with other germs, and have found a sort of nidus among my melancholy fancies. Nobody really holds them. They bear the same relation to real belief as walking on the head for a show does to running away from an explosion or walking fast to catch the train.”
This is really remarkable! Thanks for posting, I don’t know when I would have seen it otherwise.
Partway through, I Googled a couple passages to quickly confirm that it wasn’t some “modern blog post in an old-timey style”—it feels more specific and prescient to me than I remember Erewhon being (though Gwern’s comment makes me think I should go back and look again, or check Darwin Among the Machines).
Evans’ predictions on Trost’s arguments and overall vibe are also impressive, IMO. (E.g. “if it were not for your incurable dilettanteism in science as in all other things—if you had once understood the action of any delicate machine...” ~= “actually training models gives you the insight that AI risk isn’t a thing.”) I wonder how much Evans-vs.-Trost-style conversation was already going on.
Interesting how this part turned out:
It doesn’t seem like Evans (or others at the time?) anticipated that instead of individual machines reproducing like humans, they could just run the factories / mines / etc. that produce them. Which is an odd oversight, since by that time machines were certainly used in factories to make more machines—it seems like the “reproduction via external construction” prediction should have been a lot easier to make than the “reasoning machines” prediction.
😬