Anyway your comment got me thinking. So far it seems the territory colonized by humans is a subset of the territory previously colonized by life, not stretching beyond it. And the territory covered by life is also not all of Earth, nevermind the universe. So we can imagine AI occupying the most “cushy” subset of former human territory, with most humans removed from there, some subsisting as rats, some as housecats, some as wild animals periodically hit by incomprehensible dangers coming from the AI zone (similar to oil spills and habitat destruction), and some in S-risk type situations due to the AI remaining concerned with humans in some way.
Though this “concentric circles” model is maybe a bit too neat to imagine, and too similar to existing human myths about gods and so on. So let’s not trust it too much.
So we can imagine AI occupying the most “cushy” subset of former human territory
We can definitely imagine it—this is a salience argument—but why is it at all likely? Also, this argument is subject to reference class tennis: humans have colonized much more and more diverse territory than other apes, or even all other primates.
Once AI can flourish without ongoing human support (building and running machines, generating electricity, reacting to novel environmental challenges), what would plausibly limit AI to human territory, let alone “cushy” human territory? Computers and robots can survive in any environment humans can, and in some where we at present can’t.
Also: the main determinant of human territory is inter-human social dynamics. We are far from colonizing everywhere our technology allows, or (relatedly) breeding to the greatest number we can sustain. We don’t know what the main determinant of AI expansion will be; we don’t even know yet how many different and/or separate AI entities there are likely to be, and how they will cooperate, trade or conflict with each other.
A low quality prior on odds of lucky alignment: we can look at the human intelligence sharp left turn from different perspectives
Worst case scenario S risk: pigs, chickens, cows
X risk: Homo florensis, etc
Disastrously unaligned but then the superintelligence inexplicably started to align itself instead of totally wiping us out: Whales, gorillas
unaligned but that’s randomly fine for us: raccoons, rats
Largely aligned: Housecats
X risk would be passenger pigeons, no?
Anyway your comment got me thinking. So far it seems the territory colonized by humans is a subset of the territory previously colonized by life, not stretching beyond it. And the territory covered by life is also not all of Earth, nevermind the universe. So we can imagine AI occupying the most “cushy” subset of former human territory, with most humans removed from there, some subsisting as rats, some as housecats, some as wild animals periodically hit by incomprehensible dangers coming from the AI zone (similar to oil spills and habitat destruction), and some in S-risk type situations due to the AI remaining concerned with humans in some way.
Though this “concentric circles” model is maybe a bit too neat to imagine, and too similar to existing human myths about gods and so on. So let’s not trust it too much.
We can definitely imagine it—this is a salience argument—but why is it at all likely? Also, this argument is subject to reference class tennis: humans have colonized much more and more diverse territory than other apes, or even all other primates.
Once AI can flourish without ongoing human support (building and running machines, generating electricity, reacting to novel environmental challenges), what would plausibly limit AI to human territory, let alone “cushy” human territory? Computers and robots can survive in any environment humans can, and in some where we at present can’t.
Also: the main determinant of human territory is inter-human social dynamics. We are far from colonizing everywhere our technology allows, or (relatedly) breeding to the greatest number we can sustain. We don’t know what the main determinant of AI expansion will be; we don’t even know yet how many different and/or separate AI entities there are likely to be, and how they will cooperate, trade or conflict with each other.
loved this!