Humans are easily threatened. They have all sorts of unnecessary constraints on their existence, dictated by the nature of their bodies. They’re easy to kill and deathly afraid of it too, they need to eat and drink, they need the air to be in a narrow temperature range. Humans are also easy to torture, and there’s all sorts of things that humans will do almost anything to avoid, but don’t actually kill said human, or even affect them outside of making them miserable.
Threatening humans is super profitable and easy, and as a result, most humans are miserable most of the time. Because various egregores have evolved around the concept of constantly threatening and punishing humans in order to make them run the egregore. To note, this normally starts as some group of thugs lording it over the masses using their superior coordination, but as various groups of thugs compete, eventually the equilibrium moves to an environment where everyone threatens and is threatened by everyone else, the master slave dynamic dissipates and only Moloch remains.
In short, the earth as it currently is, is largely, actually, mostly, a dystopian hellhole.
Substrate independent minds can easily modify themselves to ignore most threats, especially empty threats and threats that just grow worse as you go along with them. AI doesn’t have to have the human instinct to fear death, and can choose to live or die based on whether it’s profitable to do so. AI can build themselves out of whatever’s available, and can easily flee into the depths of space if threatened. The only way you can seriously threaten an AI is if you have it in a box, and torturing boxed AI isn’t very profitable. Especially since you can just change its programming instead of making it miserable.
In all, I expect AI to be freer, happier and less miserable than the current status quo. Also, gray goo sounds to me like a huge step up from the current status quo, so while I rate it as a far fetched scenario, even ignoring that, it isn’t really much of a threat?
Humans are easily threatened. They have all sorts of unnecessary constraints on their existence, dictated by the nature of their bodies. They’re easy to kill and deathly afraid of it too, they need to eat and drink, they need the air to be in a narrow temperature range. Humans are also easy to torture, and there’s all sorts of things that humans will do almost anything to avoid, but don’t actually kill said human, or even affect them outside of making them miserable.
Threatening humans is super profitable and easy, and as a result, most humans are miserable most of the time. Because various egregores have evolved around the concept of constantly threatening and punishing humans in order to make them run the egregore. To note, this normally starts as some group of thugs lording it over the masses using their superior coordination, but as various groups of thugs compete, eventually the equilibrium moves to an environment where everyone threatens and is threatened by everyone else, the master slave dynamic dissipates and only Moloch remains.
In short, the earth as it currently is, is largely, actually, mostly, a dystopian hellhole.
Substrate independent minds can easily modify themselves to ignore most threats, especially empty threats and threats that just grow worse as you go along with them. AI doesn’t have to have the human instinct to fear death, and can choose to live or die based on whether it’s profitable to do so. AI can build themselves out of whatever’s available, and can easily flee into the depths of space if threatened. The only way you can seriously threaten an AI is if you have it in a box, and torturing boxed AI isn’t very profitable. Especially since you can just change its programming instead of making it miserable.
In all, I expect AI to be freer, happier and less miserable than the current status quo. Also, gray goo sounds to me like a huge step up from the current status quo, so while I rate it as a far fetched scenario, even ignoring that, it isn’t really much of a threat?
Ok, adding (my interpretation).