From what I know, the danger of UFAI isn’t that such an AI would be evil like in fiction (anthropomorphized AIs), but rather that it wouldn’t care about us and would want to use resources to achieve goals other than what humans would want (“all that energy and those atoms, I need them to make more computronium, sorry”).
I suppose it’s possible to invent many scenarios where such an evil AI would be possible, but it seems unlikely enough based on the information that I have now that I wouldn’t gamble a chance at life (versus a certain death) based on this sci-fi plot.
But if you are scared of UFAI, you can do something now by supporting FAI research. It might actually be more likely for us to face a UFAI within our current lives than after being woken up from cryonic preservation (since just the fact of being woken up is probably a positive sign of FAI).
From what I know, the danger of UFAI isn’t that such an AI would be evil like in fiction (anthropomorphized AIs), but rather that it wouldn’t care about us and would want to use resources to achieve goals other than what humans would want (“all that energy and those atoms, I need them to make more computronium, sorry”).
I presume he was referring to disutopias and wireheading scenarios that he could hypothetically consider worse than death.
That was my understanding, but I think that any world in which there is an AGI that isn’t Friendly probably won’t be very stable. If that happens, I think there’s a lot more chances that humanity will be destroyed quickly and you won’t be woken up than that a stable but “worse than death” world will form and decide to wake you up.
But maybe I’m missing something that makes such “worse than death” worlds plausible.
From what I know, the danger of UFAI isn’t that such an AI would be evil like in fiction (anthropomorphized AIs), but rather that it wouldn’t care about us and would want to use resources to achieve goals other than what humans would want (“all that energy and those atoms, I need them to make more computronium, sorry”).
I suppose it’s possible to invent many scenarios where such an evil AI would be possible, but it seems unlikely enough based on the information that I have now that I wouldn’t gamble a chance at life (versus a certain death) based on this sci-fi plot.
But if you are scared of UFAI, you can do something now by supporting FAI research. It might actually be more likely for us to face a UFAI within our current lives than after being woken up from cryonic preservation (since just the fact of being woken up is probably a positive sign of FAI).
I presume he was referring to disutopias and wireheading scenarios that he could hypothetically consider worse than death.
That was my understanding, but I think that any world in which there is an AGI that isn’t Friendly probably won’t be very stable. If that happens, I think there’s a lot more chances that humanity will be destroyed quickly and you won’t be woken up than that a stable but “worse than death” world will form and decide to wake you up.
But maybe I’m missing something that makes such “worse than death” worlds plausible.
I think you’re right. The main risk would be Friendly to Someone Else AI.