we should assume there are already a large number of unfriendly AIs in the universe, and probably in our galaxy; and that they will assimilate us within a few million years.
Let’s be Bayesian about this.
Observation: Earth has not been assimilated by UFAIs at any point in the last billion years or so. Otherwise life on Earth would be detectably different.
It is unlikely that there are no/few UFAIs in our galaxy/universe, but if they do exist it is unlikely that they would not already have assimilated us.
I don’t have enough information to give exact probabilities, but it’s a lot more likely than you seem to think that we will survive the next billion years without assimilation from an alien UFAI.
Personally, I think the most likely scenario is either that Earth is somehow special and intelligent life is rarer than we give it credit for; or that alien UFAIs are generally not interested in interstellar/intergalactic travel.
EDIT: More rigorously, let Uf be the event “Alien UFAIs are a threat to us”, and Ap be the event “We exist today” (anthropic principle). The prior probability P(Uf) is large, by your arguments, but P(Ap given Uf) is much smaller than P(Ap given not-Uf). Since we observe Ap to be true, the posterior probability P(Uf given Ap) is fairly small.
Personally, I think the most likely scenario is either that Earth is somehow special and intelligent life is rarer than we give it credit for; or that alien UFAIs are generally not interested in interstellar/intergalactic travel.
Given the sort of numbers thrown about in Fermi arguments, believing the former would suggest you are outrageously overconfident in your certainty that your beliefs are correct about the likely activities of AIs. Surely the second conclusion is more reasonable?
Personally, I don’t think it’s at all unreasonable to assign probabilities of any particular planet developing intelligent life on orders of 10^-12 or lower.
I think we can reasonably conclude that Earth has not been assimilated at any point in its entire existence. If it had been assimilated in the distant past, it would not have continued to develop uninfluenced for the rest of its history, unless the AI’s utility function were totally indifferent to our development. So we can extend the observed period over which we have not been assimilated back to a good four and a half billion years or so. The milky way is old enough that intelligent life could have developed well before our solar system ever formed, so we can consider that entire span to contain opportunities for assimilation comparable to those that exist today.
We could make a slightly weaker claim that no Strong AI has assimilated our portion of the galaxy since well before our solar system formed.
Let’s be Bayesian about this.
Observation: Earth has not been assimilated by UFAIs at any point in the last billion years or so. Otherwise life on Earth would be detectably different.
It is unlikely that there are no/few UFAIs in our galaxy/universe, but if they do exist it is unlikely that they would not already have assimilated us.
I don’t have enough information to give exact probabilities, but it’s a lot more likely than you seem to think that we will survive the next billion years without assimilation from an alien UFAI.
Personally, I think the most likely scenario is either that Earth is somehow special and intelligent life is rarer than we give it credit for; or that alien UFAIs are generally not interested in interstellar/intergalactic travel.
EDIT: More rigorously, let Uf be the event “Alien UFAIs are a threat to us”, and Ap be the event “We exist today” (anthropic principle). The prior probability P(Uf) is large, by your arguments, but P(Ap given Uf) is much smaller than P(Ap given not-Uf). Since we observe Ap to be true, the posterior probability P(Uf given Ap) is fairly small.
Given the sort of numbers thrown about in Fermi arguments, believing the former would suggest you are outrageously overconfident in your certainty that your beliefs are correct about the likely activities of AIs. Surely the second conclusion is more reasonable?
Er… yes. But I don’t think it undermines my point that we are unlikely to be assimilated by aliens in the near future.
Personally, I don’t think it’s at all unreasonable to assign probabilities of any particular planet developing intelligent life on orders of 10^-12 or lower.
I think we can reasonably conclude that Earth has not been assimilated at any point in its entire existence. If it had been assimilated in the distant past, it would not have continued to develop uninfluenced for the rest of its history, unless the AI’s utility function were totally indifferent to our development. So we can extend the observed period over which we have not been assimilated back to a good four and a half billion years or so. The milky way is old enough that intelligent life could have developed well before our solar system ever formed, so we can consider that entire span to contain opportunities for assimilation comparable to those that exist today.
We could make a slightly weaker claim that no Strong AI has assimilated our portion of the galaxy since well before our solar system formed.