Let me try an example to see if I’ve got it right. Humans think that it is wrong to destroy living things, but okay to destroy non-living things. But in physics the line between living and non-living is blurry. For example a developing embryo goes from non-living to living in a gradual way; hence the abortion debate. The AI is acting on our behalf and so it also wants to preserve life. But this is difficult because “life” doesn’t have a clear boundary. So it fixes this problem by ensuring that every object in the simulation is either alive or non-alive. When people in the simulation become pregnant it looks and feels to them as though they have a growing baby inside them, but in fact there is no child’s brain outside the simulation. At the moment they give birth the AI very quickly fabricates a child-brain and assigns it control of the simulated baby. This means that if someone decides to terminate their pregnancy then they can be assured that they are not harming a living thing (this is hypothetical because presumably the simulation is utopic enough that abortions are never necessary). After the child is born then it definitely is living and the people in the simulation know that they have to act to protect it.
Yes! I was hoping that the post would provoke ideas like that. It’s a playground for thinking about what people want, without distractions like nanotech etc.
This is a very interesting idea!
Let me try an example to see if I’ve got it right. Humans think that it is wrong to destroy living things, but okay to destroy non-living things. But in physics the line between living and non-living is blurry. For example a developing embryo goes from non-living to living in a gradual way; hence the abortion debate. The AI is acting on our behalf and so it also wants to preserve life. But this is difficult because “life” doesn’t have a clear boundary. So it fixes this problem by ensuring that every object in the simulation is either alive or non-alive. When people in the simulation become pregnant it looks and feels to them as though they have a growing baby inside them, but in fact there is no child’s brain outside the simulation. At the moment they give birth the AI very quickly fabricates a child-brain and assigns it control of the simulated baby. This means that if someone decides to terminate their pregnancy then they can be assured that they are not harming a living thing (this is hypothetical because presumably the simulation is utopic enough that abortions are never necessary). After the child is born then it definitely is living and the people in the simulation know that they have to act to protect it.
Is that the right idea?
Yes! I was hoping that the post would provoke ideas like that. It’s a playground for thinking about what people want, without distractions like nanotech etc.