What I think you’re driving at is that you want it to value the Beatles in some way. Having some sort of useful crossover between our values and its is the entire project of FAI.
I’m just trying to figure out under what circumstances we could consider a completely artificial entity a continuation of our existence. As you pointed out, merely containing our knowledge isn’t enough. Human knowledge is a constantly growing edifice, where each generation adds to and build upon the successes of the past. I wouldn’t expect an AI to find value in everything we have produced, just as we don’t. But if our species were wiped out, I would feel comfortable calling an AI which traveled the universe occasionally writing McCartney- or Lennon-inspired songs “us.” That would be survival. (I could even deal with a Ringo Starr AI, in a pinch.)
What I think you’re driving at is that you want it to value the Beatles in some way. Having some sort of useful crossover between our values and its is the entire project of FAI.
I’m just trying to figure out under what circumstances we could consider a completely artificial entity a continuation of our existence. As you pointed out, merely containing our knowledge isn’t enough. Human knowledge is a constantly growing edifice, where each generation adds to and build upon the successes of the past. I wouldn’t expect an AI to find value in everything we have produced, just as we don’t. But if our species were wiped out, I would feel comfortable calling an AI which traveled the universe occasionally writing McCartney- or Lennon-inspired songs “us.” That would be survival. (I could even deal with a Ringo Starr AI, in a pinch.)
I strongly suspect that that is the same thing as a Friendly AI, and therefore I still consider UFAI an existential risk.