Extremely high IQ arising from engineering… is that not AI?
This is not a joke. UAI is essentially the fear that “we” will be replaced by another form of intelligence, outcompeted for resources by what is essentially another life form.
But how do “we” not face the same threat from an engineered lifeform just because some of the ingredients are us? If such a new engineered lifeform replaces natural humanity, is that not a UAI? If we can build in some curator instinct or 3 laws or whatever into this engineered superhuman, is that not FAI?
The interesting thing to me here is what we mean by “we.” I think it is more common for a lesswrong poster to identify as “we” with an engineered superhuman in meat substrate than with an engineered non-human intelligence in non-meat substrate.
Considering this, maybe an FAI is just an AI that learns enough about what we think of as human so that it can hack it. It could construct itself so that it felt to us like our descendent, our child. Then “we” do not resent the AI for taking all “our” resources because the AI has successfully lead us to be happy to see our child succeed beyond what we managed.
Perhaps one might say but of course this would be on our list of things we would define as unfriendly. Then we build AI’s that “curate” humans as we are now and we are precluding from enhancing ourselves or evolving past some limit we have preprogrammed in to our FAI?
Extremely high IQ arising from engineering… is that not AI?
This is not a joke. UAI is essentially the fear that “we” will be replaced by another form of intelligence, outcompeted for resources by what is essentially another life form.
But how do “we” not face the same threat from an engineered lifeform just because some of the ingredients are us? If such a new engineered lifeform replaces natural humanity, is that not a UAI? If we can build in some curator instinct or 3 laws or whatever into this engineered superhuman, is that not FAI?
The interesting thing to me here is what we mean by “we.” I think it is more common for a lesswrong poster to identify as “we” with an engineered superhuman in meat substrate than with an engineered non-human intelligence in non-meat substrate.
Considering this, maybe an FAI is just an AI that learns enough about what we think of as human so that it can hack it. It could construct itself so that it felt to us like our descendent, our child. Then “we” do not resent the AI for taking all “our” resources because the AI has successfully lead us to be happy to see our child succeed beyond what we managed.
Perhaps one might say but of course this would be on our list of things we would define as unfriendly. Then we build AI’s that “curate” humans as we are now and we are precluding from enhancing ourselves or evolving past some limit we have preprogrammed in to our FAI?