Adding a bunch of dumber AIs and upgrading humans slowly does not change the inexorable logic. Horses now exist only because humans like them, and the same will be true of humans in a post-ASI world—either the ASI(s) care, or we are eliminated through competition (if not force).
I agree that ASI won’t coordinate perfectly. Even without this, and even if for some reason all ASIs decide to respect property rights, it seems straightforwardly true that humans will die out if ASIs don’t care about them for non-instrumental reasons. A world with more capitol than labor is not possible if labor can be created cheaply with capitol—and that’s what you’re describing with the smart-as-necessary AI systems.
Competitive capitalism works well for humans who are stuck on a relatively even playing field, and who have some level of empathy and concern for each other. It will not work for us if those conditions cease to hold.
Competitive capitalism works well for humans who are stuck on a relatively even playing field, and who have some level of empathy and concern for each other.
I think this basically isn’t true, especially the last part. It’s not that humans don’t have some level of empathy for each other; they do. I just don’t think that’s the reason why competitive capitalism works well for humans. I think the reason is instead because people have selfish interests in maintaining the system.
We don’t let Jeff Bezos accumulate billions of dollars purely out of the kindness of our heart. Indeed, it is often considered far kinder and more empathetic to confiscate his money and redistribute it to the poor. The problem with that approach is that abandoning property rights incurs costs on those who rely on the system to be reliable and predictable. If we were to establish a norm that allowed us to steal unlimited money from Jeff Bezos, many people would reason, “What prevents that norm from being used against me?”
The world pretty much runs on greed and selfishness, rather than kindness. Sure, humans aren’t all selfish, we aren’t all greedy. And few of us are downright evil. But those facts are not as important for explaining why our system works. Our system works because it’s an efficient compromise among people who are largely selfish.
Maybe, I think it’s hard to say how captiolism would work if everyone had zero empathy or compassion.
But that doesn’t matter for the issue at hand.
Greed or empathy aside, capitalism currently works because people have capabilities that can’t be expanded without limit and people can’t be created quickly using capitol.
If ai labor can do every task for a thousandth the cost, and new lai labor created at need, we all die if competition is the system. We will be employed for zero tasks. The factor you mention, sub ASI systems, makes the situation worse, not better.
Maybe you’re saying we’d be employed for a while, which might be true. But in the limit, even an enhanced human is only going to have value as a novelty. Which ai probably won’t care about if it isn’t aligned at all. And even if it does, that leads to a few humans surviving as performing monkeys.
I just don’t see how else humans remain competitive with ever improving machines untethered to biology.
Adding a bunch of dumber AIs and upgrading humans slowly does not change the inexorable logic. Horses now exist only because humans like them, and the same will be true of humans in a post-ASI world—either the ASI(s) care, or we are eliminated through competition (if not force).
I agree that ASI won’t coordinate perfectly. Even without this, and even if for some reason all ASIs decide to respect property rights, it seems straightforwardly true that humans will die out if ASIs don’t care about them for non-instrumental reasons. A world with more capitol than labor is not possible if labor can be created cheaply with capitol—and that’s what you’re describing with the smart-as-necessary AI systems.
Competitive capitalism works well for humans who are stuck on a relatively even playing field, and who have some level of empathy and concern for each other. It will not work for us if those conditions cease to hold.
I think this basically isn’t true, especially the last part. It’s not that humans don’t have some level of empathy for each other; they do. I just don’t think that’s the reason why competitive capitalism works well for humans. I think the reason is instead because people have selfish interests in maintaining the system.
We don’t let Jeff Bezos accumulate billions of dollars purely out of the kindness of our heart. Indeed, it is often considered far kinder and more empathetic to confiscate his money and redistribute it to the poor. The problem with that approach is that abandoning property rights incurs costs on those who rely on the system to be reliable and predictable. If we were to establish a norm that allowed us to steal unlimited money from Jeff Bezos, many people would reason, “What prevents that norm from being used against me?”
The world pretty much runs on greed and selfishness, rather than kindness. Sure, humans aren’t all selfish, we aren’t all greedy. And few of us are downright evil. But those facts are not as important for explaining why our system works. Our system works because it’s an efficient compromise among people who are largely selfish.
Maybe, I think it’s hard to say how captiolism would work if everyone had zero empathy or compassion.
But that doesn’t matter for the issue at hand.
Greed or empathy aside, capitalism currently works because people have capabilities that can’t be expanded without limit and people can’t be created quickly using capitol.
If ai labor can do every task for a thousandth the cost, and new lai labor created at need, we all die if competition is the system. We will be employed for zero tasks. The factor you mention, sub ASI systems, makes the situation worse, not better.
Maybe you’re saying we’d be employed for a while, which might be true. But in the limit, even an enhanced human is only going to have value as a novelty. Which ai probably won’t care about if it isn’t aligned at all. And even if it does, that leads to a few humans surviving as performing monkeys.
I just don’t see how else humans remain competitive with ever improving machines untethered to biology.