If the situation was something like “current people, weighted by wealth, deliberate for a while on what to do with our resources” then I agree that’s probably like 5 − 10 times worse than the best approach (which is still a huge haircut) but not clearly catastrophic. But it’s not clear to me that’s what the default outcome of competitive dynamics would look like—sufficiently competitive dynamics could force out altruistic actors if they get outcompeted by non-altruistic actors.
I think one crux between you and I, at least, is that you see this as a considered division of how to divide resources, and I see it as an equilibrium consensus/acceptance of what property rights to enforce in maintenance, creation, and especially transfer of control/usage of resources. You think of static division, I think of equilibria and motion. Both are valid, but experience and resource use is ongoing and it must be accounted for.
I’m happy that the modern world generally approves of self-ownership: a given individual gets to choose what to do (within limits, but it’s nowhere near the case that my body and mind are part of the resources allocated by whatever mechanism is being considered). It’s generally considered an alignment failure if individual will is just a resource that the AI manages. Physical resources (and behavioral resources, which are a sale of the results of some human efforts, a distinct resource from the mind performing the action) are generally owned by someone, and they trade some results to get the results of other people’s resources (including their labor and thought-output).
There could be a side-mechanism for some amount of resources just for existing, but it’s unlikely that it can be the primary transfer/allocation mechanism, as long as individuals have independent and conflicting desires. Current valuable self-owned products (office work, software design, etc.) probably reduces in value a lot. If all human output becomes valueless (in the “tradable for other desired things or activities” sense of valuable), I don’t think current humans will continue to exist.
Wirehead utopia (including real-world “all desires fulfilled without effort or trade”) doesn’t sound appealing or workable for what I know of my own and general human psychology.
self-ownership: a given individual gets to choose what to do (within limits, but it’s nowhere near the case that my body and mind are part of the resources allocated by whatever mechanism is being considered)
for most people, this is just the right to sell their body to the machine. better than being forced at gunpoint, but being forced to by an empty fridge is not that much better, especially with monopoly accumulation as the default outcome. I agree that being able to mark ones’ selfhood boundaries with property contracts is generally good, but the ability to massively expand ones’ property contracts to exclude others from resource access is effectively a sort of scalping—sucking up resources so as to participate in an emergent cabal of resource withholding. In other words,
It’s generally considered an alignment failure if individual will is just a resource that the AI manages.
The core argument that there’s something critically wrong with capitalism is that the stock market has been an intelligence aggregation system for a long time and has a strong tendency to suck up the air in the system.
Utopia would need to involve a load balancing system that can prevent sucking-up-the-air type resource control imbalancing, so as to prevent
for most people, this is just the right to sell their body to the machine.
I think this is a big point of disagreement. For most people, there’s some amount of time/energy that’s sold to the machine, and it’s NOWHERE EVEN CLOSE to selling their actual asset (body and mind). There’s a LOT of leisure time, and a LOT of freedom even within work hours, and the choice to do something different tomorrow. It may not be as rewarding, but it’s available and the ability to make those decisions has not been sold or taken.
yeah like, above a certain level of economic power that’s true, but the overwhelming majority of humans are below that level, and AI is expected to raise that waterline. it’s kind of the primary failure mode I expect.
I mean, the 40 hour work week movement did help a lot. But it was an instance of a large push of organizing to demand constraint on what the aggregate intelligence (which at the time was the stock market—which is a trade market of police-enforceable ownership contracts), could demand of people who were not highly empowered. And it involved leveling a lopsided playing field by things that one side considered dirty tricks, such as strikes. I don’t think that’ll help against AI, to put it lightly.
To be clear, I recognize that your description is accurate for a significant portion of people. But it’s not close to the majority, and movement towards making it the majority has historically demanded changing the enforceable rules in a way that would reliably constrain the aggregate agency of the high dimensional control system steering the economy. When we have a sufficiently much more powerful one of those is when we expect failure, and right now it doesn’t seem to me that there’s any movement on a solution to that. We can talk about “oh we need something better than capitalism” but the problem with the stock market is simply that it’s enforceable prediction, thereby sucking up enough air from the room that a majority of people do not get the benefits you’re describing. If they did, then you’re right, it would be fine!
I mean, also there’s this, but somehow I expect that that won’t stick around long after robots are enough cheaper than humans
I think we’re talking past each other a bit. It’s absolutely true that the vast majority historically and, to a lesser extent, in modern times, are pretty constrained in their choices. This constraint is HIGHLY correlated with distance from participation in voluntary trade (of labor or resources).
I think the disconnect is the word “capitalism”—when you talk about stock markets and price discovery, that says to me you’re thinking of a small part of the system. I fully agree that there are a lot of really unpleasant equilibra with the scale and optimization pressure of the current legible financial world, and I’d love to undo a lot of it. But the underlying concept of enforced and agreed property rights and individual human decisions is important to me, and seems to be the thing that gets destroyed first when people decry capitalism.
Ok, it sounds, even to me, like “The heads. You’re looking at the heads. Sometimes he goes too far. He’s the first one to admit it.” But really, I STRONGLY expect that I am experiencing peak human freedom RIGHT NOW (well, 20 years ago, but it’s been rather flat for me and my cultural peers for a century, even if somewhat declining recently), and capitalism (small-c, individual decisions and striving, backed by financial aggregation with fairly broad participation) has been a huge driver of that. I don’t see any alternatives that preserve the individuality of even a significant subset of humanity.
If property rights to the stars are distributed prior to this, why does this competition cause issues? Maybe you basically agree here, but think it’s unlikely property will be distributed like this.
Separately, for competitive dynamics with reasonable rule of law and alignment ~solved, why do you think the strategy stealing assumption won’t apply? (There are a bunch of possible objections here, just wondering what your’s is. Personally I think strategy stealing is probably fine if the altruistic actors care about the long run and are strategic.)
If the situation was something like “current people, weighted by wealth, deliberate for a while on what to do with our resources” then I agree that’s probably like 5 − 10 times worse than the best approach (which is still a huge haircut) but not clearly catastrophic. But it’s not clear to me that’s what the default outcome of competitive dynamics would look like—sufficiently competitive dynamics could force out altruistic actors if they get outcompeted by non-altruistic actors.
I think one crux between you and I, at least, is that you see this as a considered division of how to divide resources, and I see it as an equilibrium consensus/acceptance of what property rights to enforce in maintenance, creation, and especially transfer of control/usage of resources. You think of static division, I think of equilibria and motion. Both are valid, but experience and resource use is ongoing and it must be accounted for.
I’m happy that the modern world generally approves of self-ownership: a given individual gets to choose what to do (within limits, but it’s nowhere near the case that my body and mind are part of the resources allocated by whatever mechanism is being considered). It’s generally considered an alignment failure if individual will is just a resource that the AI manages. Physical resources (and behavioral resources, which are a sale of the results of some human efforts, a distinct resource from the mind performing the action) are generally owned by someone, and they trade some results to get the results of other people’s resources (including their labor and thought-output).
There could be a side-mechanism for some amount of resources just for existing, but it’s unlikely that it can be the primary transfer/allocation mechanism, as long as individuals have independent and conflicting desires. Current valuable self-owned products (office work, software design, etc.) probably reduces in value a lot. If all human output becomes valueless (in the “tradable for other desired things or activities” sense of valuable), I don’t think current humans will continue to exist.
Wirehead utopia (including real-world “all desires fulfilled without effort or trade”) doesn’t sound appealing or workable for what I know of my own and general human psychology.
for most people, this is just the right to sell their body to the machine. better than being forced at gunpoint, but being forced to by an empty fridge is not that much better, especially with monopoly accumulation as the default outcome. I agree that being able to mark ones’ selfhood boundaries with property contracts is generally good, but the ability to massively expand ones’ property contracts to exclude others from resource access is effectively a sort of scalping—sucking up resources so as to participate in an emergent cabal of resource withholding. In other words,
The core argument that there’s something critically wrong with capitalism is that the stock market has been an intelligence aggregation system for a long time and has a strong tendency to suck up the air in the system.
Utopia would need to involve a load balancing system that can prevent sucking-up-the-air type resource control imbalancing, so as to prevent
I think this is a big point of disagreement. For most people, there’s some amount of time/energy that’s sold to the machine, and it’s NOWHERE EVEN CLOSE to selling their actual asset (body and mind). There’s a LOT of leisure time, and a LOT of freedom even within work hours, and the choice to do something different tomorrow. It may not be as rewarding, but it’s available and the ability to make those decisions has not been sold or taken.
yeah like, above a certain level of economic power that’s true, but the overwhelming majority of humans are below that level, and AI is expected to raise that waterline. it’s kind of the primary failure mode I expect.
I mean, the 40 hour work week movement did help a lot. But it was an instance of a large push of organizing to demand constraint on what the aggregate intelligence (which at the time was the stock market—which is a trade market of police-enforceable ownership contracts), could demand of people who were not highly empowered. And it involved leveling a lopsided playing field by things that one side considered dirty tricks, such as strikes. I don’t think that’ll help against AI, to put it lightly.
To be clear, I recognize that your description is accurate for a significant portion of people. But it’s not close to the majority, and movement towards making it the majority has historically demanded changing the enforceable rules in a way that would reliably constrain the aggregate agency of the high dimensional control system steering the economy. When we have a sufficiently much more powerful one of those is when we expect failure, and right now it doesn’t seem to me that there’s any movement on a solution to that. We can talk about “oh we need something better than capitalism” but the problem with the stock market is simply that it’s enforceable prediction, thereby sucking up enough air from the room that a majority of people do not get the benefits you’re describing. If they did, then you’re right, it would be fine!
I mean, also there’s this, but somehow I expect that that won’t stick around long after robots are enough cheaper than humans
I think we’re talking past each other a bit. It’s absolutely true that the vast majority historically and, to a lesser extent, in modern times, are pretty constrained in their choices. This constraint is HIGHLY correlated with distance from participation in voluntary trade (of labor or resources).
I think the disconnect is the word “capitalism”—when you talk about stock markets and price discovery, that says to me you’re thinking of a small part of the system. I fully agree that there are a lot of really unpleasant equilibra with the scale and optimization pressure of the current legible financial world, and I’d love to undo a lot of it. But the underlying concept of enforced and agreed property rights and individual human decisions is important to me, and seems to be the thing that gets destroyed first when people decry capitalism.
Ok, it sounds, even to me, like “The heads. You’re looking at the heads. Sometimes he goes too far. He’s the first one to admit it.” But really, I STRONGLY expect that I am experiencing peak human freedom RIGHT NOW (well, 20 years ago, but it’s been rather flat for me and my cultural peers for a century, even if somewhat declining recently), and capitalism (small-c, individual decisions and striving, backed by financial aggregation with fairly broad participation) has been a huge driver of that. I don’t see any alternatives that preserve the individuality of even a significant subset of humanity.
If property rights to the stars are distributed prior to this, why does this competition cause issues? Maybe you basically agree here, but think it’s unlikely property will be distributed like this.
Separately, for competitive dynamics with reasonable rule of law and alignment ~solved, why do you think the strategy stealing assumption won’t apply? (There are a bunch of possible objections here, just wondering what your’s is. Personally I think strategy stealing is probably fine if the altruistic actors care about the long run and are strategic.)