I didn’t mean to imply forcing unwanted modifications on everybody “for their own good”—I was talking about under what conditions we might accept things we don’t like (I don’t think this is a very plausible singularity scenario, except as a general “how weird things could get”).
I don’t like limitations on my ability to let my sheep graze, but I may accept them if everyone does so and it reduces overgrazing. I may not like limits on my ability to own guns, but I may accept them if it means living in a safer society. I may not like modifications to my sex drive, but I may be willing to agree in exchange for living in a better society.
In principle, we could find ways of making everybody better off. Of course, the details of how such an agreement is reached matter a lot—markets, democracy, competition between countries, a machine-God enforcing it’s will.
I didn’t mean to imply forcing unwanted modifications on everybody “for their own good”—I was talking about under what conditions we might accept things we don’t like (I don’t think this is a very plausible singularity scenario, except as a general “how weird things could get”).
I don’t like limitations on my ability to let my sheep graze, but I may accept them if everyone does so and it reduces overgrazing. I may not like limits on my ability to own guns, but I may accept them if it means living in a safer society. I may not like modifications to my sex drive, but I may be willing to agree in exchange for living in a better society.
In principle, we could find ways of making everybody better off. Of course, the details of how such an agreement is reached matter a lot—markets, democracy, competition between countries, a machine-God enforcing it’s will.