“Workers regularly trade with billionaires and earn more than $77 in wages, despite vast differences in wealth.”
Yes, because the worker has something the billionaire wants (their labor) and so is able to sell it. Yudkowsky’s point about trying to sell an Oreo for $77 is that a billionaire isn’t automatically going to want to buy something off you if they don’t care about it (and neither would an ASI).
”I’m simply arguing against the point that smart AIs will automatically turn violent and steal from agents who are less smart than they are, unless they’re value aligned. This is a claim that I don’t think has been established with any reasonable degree of rigor.”
I completely agree but I’m not sure anyone is arguing that smart AIs would immediately turn violent unless it was in their strategic interest.
Yudkowsky’s point about trying to sell an Oreo for $77 is that a billionaire isn’t automatically going to want to buy something off you if they don’t care about it (and neither would an ASI).
I thought Yudkowsky’s point was that the billionaire won’t give you $77 for an Oreo because they could get an Oreo for less than $77 via other means. But people don’t just have an Oreo to sell you. My point in that sentence was to bring up that workers routinely have things of value that they can sell for well over $77, even to billionaires. Similarly, I claim that Yudkowsky did not adequately show that humans won’t have things of substantial value that they can sell to future AIs.
I’m not sure anyone is arguing that smart AIs would immediately turn violent unless it was in their strategic interest
The claim I am disputing is precisely that it will be in the strategic interest of unaligned AIs to turn violent and steal from agents that are less smart than them. In that sense, I am directly countering a claim that people in these discussions routinely make.
“Workers regularly trade with billionaires and earn more than $77 in wages, despite vast differences in wealth.”
Yes, because the worker has something the billionaire wants (their labor) and so is able to sell it. Yudkowsky’s point about trying to sell an Oreo for $77 is that a billionaire isn’t automatically going to want to buy something off you if they don’t care about it (and neither would an ASI).
”I’m simply arguing against the point that smart AIs will automatically turn violent and steal from agents who are less smart than they are, unless they’re value aligned. This is a claim that I don’t think has been established with any reasonable degree of rigor.”
I completely agree but I’m not sure anyone is arguing that smart AIs would immediately turn violent unless it was in their strategic interest.
I thought Yudkowsky’s point was that the billionaire won’t give you $77 for an Oreo because they could get an Oreo for less than $77 via other means. But people don’t just have an Oreo to sell you. My point in that sentence was to bring up that workers routinely have things of value that they can sell for well over $77, even to billionaires. Similarly, I claim that Yudkowsky did not adequately show that humans won’t have things of substantial value that they can sell to future AIs.
The claim I am disputing is precisely that it will be in the strategic interest of unaligned AIs to turn violent and steal from agents that are less smart than them. In that sense, I am directly countering a claim that people in these discussions routinely make.