I mean moreso: Consider a FAI so advanced that it decides to reward all beings who did not contribute to creating Roko’s Basilisk with eternal bliss, regardless of whether or not they knew of the potential existence of Roko’s Basilisk.
Why is Roko’s Basilisk any more or any less of a threat than the infinite other hypothetically possible scenarios that have infinite other (good and bad) outcomes? What’s so special about this one in particular that makes it non-negligible? Or to make anyone concerned about it in the slightest? (That is the part I’m missing. =\ )
Well, in the original formulation, Roko’s Basilisk is an FAI that decided the good from bringing an FAI into the world a few days earlier (saving ~150,000 lives per day eralier it gets here) outweighs the bad from making the threats, so there’s no reason it shouldn’t want you to aid FAI projects that promise not to make a Basilisk, just as long as you do something instead of sitting around, so there’s no inconsistency and now there’s more than one being trying to acausally motivate you into working yourself to the bone for something that most people think is crazy.
More generally, we have more than zero information about future AI, because they will be built by humans if they are built at all. Additionally, we know even more if we rule out certain categories, such as the archetypal “paperclip maximiser”. There’s room for a lot of speculation and uncertainty, but far from enough room to assume complete agnosticism and that for every AI that wants one thing from us there’s an equal and opposite AI that wants the opposite.
Why is Roko’s Basilisk any more or any less of a threat than the infinite other hypothetically possible scenarios that have infinite other (good and bad) outcomes?
The idea is that an FAI build on timeless decision theory might automatically behave that way. There’s also Eliezer’s conjecture that any working FAI has to be build on timeless decision theory.
I mean moreso: Consider a FAI so advanced that it decides to reward all beings who did not contribute to creating Roko’s Basilisk with eternal bliss, regardless of whether or not they knew of the potential existence of Roko’s Basilisk.
Why is Roko’s Basilisk any more or any less of a threat than the infinite other hypothetically possible scenarios that have infinite other (good and bad) outcomes? What’s so special about this one in particular that makes it non-negligible? Or to make anyone concerned about it in the slightest? (That is the part I’m missing. =\ )
Well, in the original formulation, Roko’s Basilisk is an FAI that decided the good from bringing an FAI into the world a few days earlier (saving ~150,000 lives per day eralier it gets here) outweighs the bad from making the threats, so there’s no reason it shouldn’t want you to aid FAI projects that promise not to make a Basilisk, just as long as you do something instead of sitting around, so there’s no inconsistency and now there’s more than one being trying to acausally motivate you into working yourself to the bone for something that most people think is crazy.
More generally, we have more than zero information about future AI, because they will be built by humans if they are built at all. Additionally, we know even more if we rule out certain categories, such as the archetypal “paperclip maximiser”. There’s room for a lot of speculation and uncertainty, but far from enough room to assume complete agnosticism and that for every AI that wants one thing from us there’s an equal and opposite AI that wants the opposite.
A priori it’s not clear that a project can hold such a promise.
The idea is that an FAI build on timeless decision theory might automatically behave that way. There’s also Eliezer’s conjecture that any working FAI has to be build on timeless decision theory.