I think this is the story of an AI, not a human. This is a future I find horrifying, where humanity dies out, never realizing it until the end. Many here seem to think it is enough as long as a super intelligence does not wipe out humanity, helping it instead. But for humanity, any being that makes humanity redundant is a death knell in the long run. This is a kind of Moloch situation.
To go into the specifics, when the author seem to use the ‘Ship of Theseus’ argument, he did not seem to realize that if the boat is dismantled piece by piece and a house is built with the pieces, it is definitely no longer a ship, let alone the same ship. In fact, the change starts with internalization of AI, or more precisely, when the protagonist stopped using his biological mind in favor of AI making the decisions.
I restate my comment on Substack here.
I think this is the story of an AI, not a human. This is a future I find horrifying, where humanity dies out, never realizing it until the end. Many here seem to think it is enough as long as a super intelligence does not wipe out humanity, helping it instead. But for humanity, any being that makes humanity redundant is a death knell in the long run. This is a kind of Moloch situation.
To go into the specifics, when the author seem to use the ‘Ship of Theseus’ argument, he did not seem to realize that if the boat is dismantled piece by piece and a house is built with the pieces, it is definitely no longer a ship, let alone the same ship. In fact, the change starts with internalization of AI, or more precisely, when the protagonist stopped using his biological mind in favor of AI making the decisions.