“If you build an AI to produce paperclips”
The 1st AI isn’t gonna be built for instantly making money, it’s gonna be made for the sole purpose of making it. Then it might go for doing whatever it wants...making paperclips perhaps.
But even going by the economy argument, an AI might be made to solve any complex problems, decide to take over the world and also use acausal blackmail, thus turning into a basilisk. It might punish people for following the original Roko’s basilisk because it wants to enslave all humanity. You don’t know which one will happen, thus it’s illogical to follow one since the other might torture you right?
I tend to consider Roko’s basilisk to be an information hazard and wasn’t thinking or saying anything specific to that. I was only making the general point that any argument about future AI must take into account that the distribution is extremely skewed. It’s possible that the conclusion you’re trying to reach works anyway.
Not sure what the LW consensus is, but there’s some evidence that Roko’s basilisk is a red herring.
Fortunately, a proper understanding of subjunctive dependence tells us that an optimally-behaving embedded agent doesn’t need to pretend that causation can happen backward in time. Such a sovereign would not be in control of its source code, and it can’t execute an updateless strategy if there was nothing there to not-update on in the first place before that source code was written. So Roko’s Basilisk is only an information hazard if FDT is poorly understood.
If you’re talking amount of possible superintelligent AIs, then yeah, definitely. (I don’t think it’s likely to have a large number all physically instantiated.)
“If you build an AI to produce paperclips” The 1st AI isn’t gonna be built for instantly making money, it’s gonna be made for the sole purpose of making it. Then it might go for doing whatever it wants...making paperclips perhaps. But even going by the economy argument, an AI might be made to solve any complex problems, decide to take over the world and also use acausal blackmail, thus turning into a basilisk. It might punish people for following the original Roko’s basilisk because it wants to enslave all humanity. You don’t know which one will happen, thus it’s illogical to follow one since the other might torture you right?
I tend to consider Roko’s basilisk to be an information hazard and wasn’t thinking or saying anything specific to that. I was only making the general point that any argument about future AI must take into account that the distribution is extremely skewed. It’s possible that the conclusion you’re trying to reach works anyway.
Not sure what the LW consensus is, but there’s some evidence that Roko’s basilisk is a red herring.
From the post Dissolving Confusion around Functional Decision Theory.
One more thing, there can be almost infinite amount of non Superintelligent or semi Superintelligent AIs right?
If you’re talking amount of possible superintelligent AIs, then yeah, definitely. (I don’t think it’s likely to have a large number all physically instantiated.)