A FAI is more likley to actually be a FAI if people don’t engage in a last desperate war for ownership of all the universe for eternity at the time of its construction.
The current proposed solution to avoid such negative sum arms race (where aggressive action and recklessness reduce the likelihood of a friendly AI for nearly all other human value sets, but increases the likelihood of one for your particular value set) has been to hope that our values aren’t really different, we’re just (for now) too dumb to see this.
That’s because it seems more likely that there’s only one FAI to rule them all, and whatever values it has will dominate the light-cone.
A FAI is more likley to actually be a FAI if people don’t engage in a last desperate war for ownership of all the universe for eternity at the time of its construction.
The current proposed solution to avoid such negative sum arms race (where aggressive action and recklessness reduce the likelihood of a friendly AI for nearly all other human value sets, but increases the likelihood of one for your particular value set) has been to hope that our values aren’t really different, we’re just (for now) too dumb to see this.
It’s a bit worse than that. The “hope” seems to be more along the lines of:
Nevermind how a nascent valueless AI is supposed to convince itself to go back into the box.