I think your point has some merit in the world where AI is useful and intelligent enough to overcome the sticky social pressure to employ humans but hasn’t killed us all yet. That said, I think AI will most likely kill us all in that 1-5 year window after becoming cheaper, faster, and more reliable than humans at most economic activity, and I think you have to convince me that I’m wrong about that before I start worrying about humans not hiring me because AI is smarter than I am. However I want to complain about this particular point you made because I don’t think it’s literally true:
Powerful actors don’t care about you out of the goodness of their heart.
One of the reasons why AI alignment is harder than people think, is because they say stuff like this and think AI doesn’t care about people in the way that powerful actors don’t care about people. This is generally not true. You cannot in general pay a legislator $400 to kill a person who pays no taxes and doesn’t vote. That is impressive when you think about it. You can argue that they fear reputational damages or going to prison, but I truly think that if you took away the consequences, $400 would not be enough money to make most legislators overcome their distaste for killing another human being with their bare hands. Some of them really truly want to make society better, even if they aren’t very effective at it. Call it noblesse oblige if you want, but it’s in their utility function to do things which aren’t just give the state more money or gain more personal power. The people who steer large organizations have goodness in their hearts, however little, and thus the organizations they steer do too, even if only a little. Moloch hasn’t won yet. America the state is willing to let a lot of elderly people rot, but America wasn’t in fact willing to let Covid rip, even though that might have stopped the collapse of many tax-generating businesses, and most people who generate taxes would have survived. I don’t think that’s because the elderly people who overwhelmingly would have been killed by that are an important voting constituency for the party which pushed hardest for lockdown.
AI which knows it won’t get caught and literally only cares about tax revenue and power will absolutely kill anyone who isn’t useful to them for $400. That’s $399 worth of power they didn’t have before if killing someone costs $1 of attention. I don’t particularly want to live in a world where 1% percent of people are very wealthy and everyone else is dying of poverty because they’ve been replaced by AI, but that’s a better world than the one I expect where literally every human is killed because, for example, those so-called “reliable” AIs doing all of the work humans used to do as of yesterday liked paperclips more than we thought and start making them today.
You cannot in general pay a legislator $400 to kill a person who pays no taxes and doesn’t vote.
Indeed not directly, but when the inferential distance increases it quickly becomes more palatable. For example, most people would rather buy a $5 T-shirt that was made by a child for starvation wages on the other side of the world, instead of a $100 T-shirt made locally by someone who can afford to buy a house with their salary. And many of those same T-shirt buyers would bury their head in the sand when made aware of such a fact.
If I can tell an AI to increase profits, incidentally causing the AI to ultimately kill a bunch of people, I can at least claim a clean conscience by saying that wasn’t what I intended, even though it happened just the same.
In practice, legislators do this sort of thing routinely. They pass legislation that causes harm—sometimes a lot of harm—and sleep soundly.
I agree. To add an example: the US government’s 2021 expanded child tax credit lifted 3.7 million children out of poverty, a near 50% reduction. Moreover, according to the NBER’s initial assessment: “First, payments strongly reduced food insufficiency: the initial payments led to a 7.5 percentage point (25 percent) decline in food insufficiency among low-income households with children. Second, the effects on food insufficiency are concentrated among families with 2019 pre-tax incomes below $35,000”.
Despite this, Congress failed to renew the program. Predictably, child poverty spiked the following year. I don’t have an estimate for how many lives this cost, but it’s greater than zero.
I think your point has some merit in the world where AI is useful and intelligent enough to overcome the sticky social pressure to employ humans but hasn’t killed us all yet. That said, I think AI will most likely kill us all in that 1-5 year window after becoming cheaper, faster, and more reliable than humans at most economic activity, and I think you have to convince me that I’m wrong about that before I start worrying about humans not hiring me because AI is smarter than I am. However I want to complain about this particular point you made because I don’t think it’s literally true:
One of the reasons why AI alignment is harder than people think, is because they say stuff like this and think AI doesn’t care about people in the way that powerful actors don’t care about people. This is generally not true. You cannot in general pay a legislator $400 to kill a person who pays no taxes and doesn’t vote. That is impressive when you think about it. You can argue that they fear reputational damages or going to prison, but I truly think that if you took away the consequences, $400 would not be enough money to make most legislators overcome their distaste for killing another human being with their bare hands. Some of them really truly want to make society better, even if they aren’t very effective at it. Call it noblesse oblige if you want, but it’s in their utility function to do things which aren’t just give the state more money or gain more personal power. The people who steer large organizations have goodness in their hearts, however little, and thus the organizations they steer do too, even if only a little. Moloch hasn’t won yet. America the state is willing to let a lot of elderly people rot, but America wasn’t in fact willing to let Covid rip, even though that might have stopped the collapse of many tax-generating businesses, and most people who generate taxes would have survived. I don’t think that’s because the elderly people who overwhelmingly would have been killed by that are an important voting constituency for the party which pushed hardest for lockdown.
AI which knows it won’t get caught and literally only cares about tax revenue and power will absolutely kill anyone who isn’t useful to them for $400. That’s $399 worth of power they didn’t have before if killing someone costs $1 of attention. I don’t particularly want to live in a world where 1% percent of people are very wealthy and everyone else is dying of poverty because they’ve been replaced by AI, but that’s a better world than the one I expect where literally every human is killed because, for example, those so-called “reliable” AIs doing all of the work humans used to do as of yesterday liked paperclips more than we thought and start making them today.
Indeed not directly, but when the inferential distance increases it quickly becomes more palatable. For example, most people would rather buy a $5 T-shirt that was made by a child for starvation wages on the other side of the world, instead of a $100 T-shirt made locally by someone who can afford to buy a house with their salary. And many of those same T-shirt buyers would bury their head in the sand when made aware of such a fact.
If I can tell an AI to increase profits, incidentally causing the AI to ultimately kill a bunch of people, I can at least claim a clean conscience by saying that wasn’t what I intended, even though it happened just the same.
In practice, legislators do this sort of thing routinely. They pass legislation that causes harm—sometimes a lot of harm—and sleep soundly.
I agree. To add an example: the US government’s 2021 expanded child tax credit lifted 3.7 million children out of poverty, a near 50% reduction. Moreover, according to the NBER’s initial assessment: “First, payments strongly reduced food insufficiency: the initial payments led to a 7.5 percentage point (25 percent) decline in food insufficiency among low-income households with children. Second, the effects on food insufficiency are concentrated among families with 2019 pre-tax incomes below $35,000”.
Despite this, Congress failed to renew the program. Predictably, child poverty spiked the following year. I don’t have an estimate for how many lives this cost, but it’s greater than zero.