Even if it kill all humans, it will be one human which will survive.
Unless it self-modifies to the point where you’re stretching any meaningful definition of “human”.
Even if his values will evolve it will be natural evolution of human values.
Again, for sufficiently broad definitions of “natural evolution”.
As most human beings don’t like to be alone, he would create new friends that is human simulations. So even worst cases are not as bad as paper clip maximiser.
If we’re to believe Hanson, the first (and possibly only) wave of human em templates will be the most introvert workaholics we can find.
Unless it self-modifies to the point where you’re stretching any meaningful definition of “human”
Its evolution could go wrong from our point of view, but older generation always thinks that younger ones are complete bastards. When I say “natural evolution” I meant complex evolution of values based on their previous state and new experience, and it is rather typical situation for any human being who’s values are evolving from childhood, and also under influence of experiences, texts and social circle.
This idea is very different from Hanson’s em world. Here we deliberately upload only one human, who is trained to become are a core of future friendly AI. He knows that he is going to make some self-improvements but he knows dangers of unlimited self-improvement. His loved ones are still in flesh. He is trained to be not a slave as in the Hanson’s Em world, but a wise ruler.
Unless it self-modifies to the point where you’re stretching any meaningful definition of “human”.
Again, for sufficiently broad definitions of “natural evolution”.
If we’re to believe Hanson, the first (and possibly only) wave of human em templates will be the most introvert workaholics we can find.
Its evolution could go wrong from our point of view, but older generation always thinks that younger ones are complete bastards. When I say “natural evolution” I meant complex evolution of values based on their previous state and new experience, and it is rather typical situation for any human being who’s values are evolving from childhood, and also under influence of experiences, texts and social circle.
This idea is very different from Hanson’s em world. Here we deliberately upload only one human, who is trained to become are a core of future friendly AI. He knows that he is going to make some self-improvements but he knows dangers of unlimited self-improvement. His loved ones are still in flesh. He is trained to be not a slave as in the Hanson’s Em world, but a wise ruler.