This is a strategy I think we should be strongly biased against for moral reasons—creating a mind who wishes to not exist? Seems like maybe this could be fine, but also maybe this could be morally terrible, akin to creating someone in constant extreme suffering.
I agree this is a potential concern and have added it.
I share some of the intuition that it could end up suffering in this setup if it does have qualia (which ideally it wouldn’t) but I think most of that is from analogy with human suicidal people? I think it will probably not be fundamentally different from any other kind of disutility, but maybe not.
This is a strategy I think we should be strongly biased against for moral reasons—creating a mind who wishes to not exist? Seems like maybe this could be fine, but also maybe this could be morally terrible, akin to creating someone in constant extreme suffering.
I agree this is a potential concern and have added it.
I share some of the intuition that it could end up suffering in this setup if it does have qualia (which ideally it wouldn’t) but I think most of that is from analogy with human suicidal people? I think it will probably not be fundamentally different from any other kind of disutility, but maybe not.