I’d like a future where people were on a level with me, so I could be of some meaningful use.
However a future without massive disparities of power and knowledge between myself and the inhabitants, would not be able to revive me from cryo sleep.
I already guessed that might be the wish of many people. That’s one reason why I would like to acquire the knowledge to deliberately create a single not-person, a Very Powerful Optimization Process. What does it take to not be a person? That is one of those moral questions that runs into empirical confusions. But if I could create a VPOP that did not have subjective experience (or the confusion we name subjective experience), and did not have any pleasure or pain, or valuation of itself, then I think it might be possible to have around a superintelligence that did not, just by its presence, supersede us as an adult; but was nonetheless capable of guarding the maturation of humans into adults, and, a rather lesser problem, capable of reviving cryonics patients.
If there is anything in there that seems like it should be impossible to understand, then remember that mysteries exist in the map, not in the territory.
The only thing more difficult than creating a Friendly AI, involving even deeper moral issues, is creating a child after the right and proper fashion of creating a child. If I do not wish to be a father, I think that it is acceptable for me to avoid it; and certainly the skill to deliberately not create a child would be less than the skill to deliberately create a healthy child.
So yes I do confess: I wish to create something that has no value of itself, so that it can safeguard the future and our choices, without, in its own presence, automatically setting the form of humankind’s adulthood according to the design decisions of a handful of programmers, and superseding our own worth. An emergency measure should do as little as possible, and everything necessary; if I can avoid creating a child I should do so, if only because it is not necessary.
The various silly people who think I want to keep the flesh around forever, or constrain all adults to the formal outline of an FAI, are only, of course, making things up; their imagination is not wide enough to understand the concept of some possible AIs being people, and some possible AIs being something else. A mind is a mind, isn’t it? A little black box just like yours, but bigger. But there are other possibilities than that, though I can only see them unclearly, at this point.
I already guessed that might be the wish of many people. That’s one reason why I would like to acquire the knowledge to deliberately create a single not-person, a Very Powerful Optimization Process. What does it take to not be a person? That is one of those moral questions that runs into empirical confusions. But if I could create a VPOP that did not have subjective experience (or the confusion we name subjective experience), and did not have any pleasure or pain, or valuation of itself, then I think it might be possible to have around a superintelligence that did not, just by its presence, supersede us as an adult; but was nonetheless capable of guarding the maturation of humans into adults, and, a rather lesser problem, capable of reviving cryonics patients.
If there is anything in there that seems like it should be impossible to understand, then remember that mysteries exist in the map, not in the territory.
The only thing more difficult than creating a Friendly AI, involving even deeper moral issues, is creating a child after the right and proper fashion of creating a child. If I do not wish to be a father, I think that it is acceptable for me to avoid it; and certainly the skill to deliberately not create a child would be less than the skill to deliberately create a healthy child.
So yes I do confess: I wish to create something that has no value of itself, so that it can safeguard the future and our choices, without, in its own presence, automatically setting the form of humankind’s adulthood according to the design decisions of a handful of programmers, and superseding our own worth. An emergency measure should do as little as possible, and everything necessary; if I can avoid creating a child I should do so, if only because it is not necessary.
The various silly people who think I want to keep the flesh around forever, or constrain all adults to the formal outline of an FAI, are only, of course, making things up; their imagination is not wide enough to understand the concept of some possible AIs being people, and some possible AIs being something else. A mind is a mind, isn’t it? A little black box just like yours, but bigger. But there are other possibilities than that, though I can only see them unclearly, at this point.