“You wouldn’t make that kind of mistake about an AI” is a good line. Almost no one else thinks of the state as an eldritch horror, or even refers to it as such in jest, like they do about AI systems. They should.
“L’état c’est nous” though? (The state, it is us.)
I’m pretty sure I am not an eldritch horror and I suspect you aren’t either, Shankar! Does the “eldritch horror part” arises from our composition? Is so, why and how? Maybe it is an aspect of humans that emerges somehow from a large number of humans?
“L’état ce ne sont que des bureaucrates humains” is another possibility (the state, it is merely some human bureaucrats) who I fully admit might not be saints, and might not be geniuses, but at least they are humans, operating at human speeds, with human common sense and the moral intuitions that arise in a human limbic system in the loop.
AGI really will be fundamentally different, in a statistical, material, mechanical, temporal, and algorithmic sense.
Scenario C, with an approach where “the regulation is deeply conservative and in every case of ambiguity bans things rather than permits them” seemed the most promising to me, and the OP’s central problem there is only limited by “international agreement”… which existing nation states know how to at least sort of do.
Calling it “eldritch” is mere rhetorical flourish to evoke Lovecraft; of course it’s not literally paranormal.
Asking which individual is responsible for the evil of the state is like asking which transistor in the AGI is misaligned. That kind of reductionism obviously won’t get you a reasonable answer, and you know it.
The problem is the incorrigibility of the system; it’s the same idea as Scott Alexander’s Meditations on Moloch: ultimately, it’s a coordination problem that giving a name to helps reify in the human mind. In this context, I like quoting 1 Samuel 8:11–18, slightly paraphrased.
Upon consideration, a lighthearted example might be better than Cabaret at showing how such a system can get out of control: Daylight Saving Time. The US government can’t get rid of it, and even I can’t find a way to blame that on bureaucrats.
As for “the state, it is us,” I guess I don’t understand what you mean. Sure, in the same sense the serfs are the estate; the conscripts, the army; and the slaves, the plantation, but that’s not really what one means when denouncing the state. But if it’s a way of transferring moral culpability for the state’s actions, I reject that entirely: I do not blame a slave for his submission.
I don’t think the moral intuitions of the people involved are really relevant, for the reasons above, but more importantly, you say that looking backwards through human history, the distribution is not fundamentally unpredictable: that’s not how predictions work! To people who had never seen a state before, I claim it would have indeed have been alien to see, for example, fishermen killing whales for neither meat nor blubber, but to fill quotas.
If AGI is indeed as “fundamentally different” as you worry from anything that came before it, I do not consider it as unlikely to be better than the state as it exists today as you do.
Scenario C seemed the most promising to me too, but for different reasons.
When you say “I do not blame a slave for his submission” regarding Daylight Savings Time, that totally works in the second frame where “l’état ce ne sont que des bureaucrates humains”.
You’re identifying me, and you, and all the other “slaves” as “mere transistors in society”.
I dunno about you, but I grew up in a small (unincorporated) town, that was run by the Chamber of Commerce and Rotary Club. My first check as pay for labor was as a soccer referee when I was 12, reffing seven year olds. There was a Deputy of the County Sheriff, but he was not the state. He was very mindful around the communitarian influencers of the town.
Of my town, it made total sense to me to imagine “l’état c’est nous”.
Whether you were Democrat, Green, Republican, or Libertarian in your party registration were were all democrats, notpeasants. Yes?
In my town, PTA moms would make sure the Deputy got a bad guy if the gossip network noticed that there was a schizophrenic building a machine gun in his garage (or whatever).
I road my bike around, and picked wild fruit from overgrown orchards.
A chain tried to open a Kwik-E-Mart in town, and they didn’t donate to the PTA’s Halloween Carnival, and so I wasn’t allowed to shop there, as a child, for years until they started donating, which they eventually did.
I don’t think the moral intuitions of the people involved are really relevant, for the reasons above
I can’t imagine any polity actually working where the rulers (and in a democratic republic that’s the voters) did NOT have moral intuitions engaged with their regulation of themselves and their neighbors.
I can imagine people’s moral intuitions not connecting to the successful operation of their own government if the government is just evil, and anarchism is correct… but I don’t think the government is evil and therefore I don’t think anarchism is correct!
And if I did think that, then I’d expect it to be relatively clear to show this to nearly all “right thinking people” using reasonable person standards of explanation, and then “we” could just notice that, and overthrow the evil government… right?
As I see it, Russia and China are places where the common man is tragically disengaged from governance.
And because they do not have an autonomous citizenry, they lack people who have trust in their own ability to engage in civic reasoning towards civic ends.
There is only one real “citizen” in each of those countries: Putin and Xi respectively… who need not recognize anyone else. (Note: this actually makes securing AI in those countries easier! Convince “the one guy” and… I think then it will basically happen!?! (From their perspective, I bet we seem like the chaos monkeys.))
Now, I am not personally a neo-con right now.
My current posteriors say: maybe not all currently existing cultures are capable of healthy democratic self-rule.
However, when I think of “straightforward” solutions to AGI regulation my first thought is to set up a Global Parliament of 71 representatives, with proportional representation and quadratic monetary voting as a funding mechanism. I know there are more countries than 71 on the whole planet, but South American only has ~425 million people and ~2 languages.
Make the new thing a “House of Commons” that lets the “United Nations” function as a sort of auxiliary “Senate” or “House of Lords” or whatever (for ceremonial bootstrapping)?
Have that group hold elections over and over until they can find a Prime Minister of Earth who fulfills the Condorcet Criterion.
I bet that PM would be in favor of “AI-not-killing-everyone-ism” because “AI-killing-everyone-ism” seems like a pretty funky platform!
Personally, I would be open to paying every human who can speak (but not every AI and not every orca and not every african gray parrot) a silver dime to participate in voting.
I’d make the first vote cost (in USD?) cost $1, then $2.71, $7.39, $20.09, $54.60, $148.41 and so on with powers of “e”.
People would be able to give the dime back to get more votes… but personally I’d keep my dime as a “globally patriotic” keepsake and just pay for my votes in fiat.
I think America and Japan and Korea and Europe and so on have enough rich people that if there’s a couple billion idiots clogging up the voting system… those idiots will be poor enough to lose out in the final vote count? And I honestly thing the idiots would wise up pretty fast! <3
If we could finish the “first try” before the end of 2023, that would be amazing! Then do another in 2024. Then do it every year for 10 years, just to find the balances, and learn the skills!
Then my representative in Earth’s parliament could ask Earth’s Prime Minister, each year, what the fuck was up with AI, and maybe the PM would say (into the cameras in the Chamber) that the global government can’t use all that quadratic voting money to come up with ANY way to work with the pre-existing westphalian states to globally prevent humans from getting murdered by a scary AI in any way at all ever...
...and then maybe we deserve to die?
Or (I think to myself) if that’s what YOU all end up electing, then maybe YOU ALL deserve to die? I don’t wanna die, see? I just want nice normal things, like clean neighborhoods and happy kids n’stuff :-)
So at that point imaginary point, where global governance has actually been tried and it didn’t work… I might be ready to say that “my right to self defense is more important than the rest of humanity’s right to the consent of the governed”…
...and then I’d start looking for someone who has a solution for “the problem of literally everyone on earth maybe getting killed in a small number of years by a bunch of poorly coded machines” who could credibly say “l’etat c’est moi” like in the bad old days?
I don’t know basically anything about Napoleon the Eighth, but if he is an adequate focal point who can read a credible plan for dealing with AI off of a teleprompter...
...then maybe I’d give up on democracy, and bend the knee to him, and hope desperately that his “Butlerian” War is relatively peaceful and fast and successful???
I really reallyreally hope it doesn’t get to that point, but if it does… well… I’m pretty sure I have a right to self defense???
And maybe that right (and the similar rights of everyone else who wants to be safe from AI) trumps the moral requirement to secure the consent of governed people?
But I would expect any unelected monarch to still take deontics strongly into account, even if granted insane powers by a crazy situation. The people who swear their sword to such a ruler would care about honor, I’m pretty sure...
Maybe you can explain to me, Shankar, how you complete the sentence “l’etat c’est <entity>” such that the <entity> seems to you like an entity that is at least vaguely person-shaped and able to track its own interests, and pursue its real interests somewhat effectively.
(And if there is no such person-shaped thing in charge of anything, then who the fuck are we paying taxes to? If there is no at-least-semi-benevolent person-shaped entity in that “Rulership Slot”, doesn’t that mean we could (and maybe should?) all just agree, on the internet, to simultaneously all not pay taxes, and then we’d all get away with it, and then we could sing songs about it or something?)
This is all intensely practical from my perspective.
Its the same “mental muscles” that hold a marriage together, or keep seven year olds from hurting each other while they play soccer, or let a group house elect a kitchen manager who buys food for the whole house that makes the vegans and the carnists happy, because everyone has a right to eat food they’ve paid for, and collective buying is a good way to get good deals for low prices!
If you’re not modeling government as a best-effort pretty-OK “buyers club for public services” run by people who care about the people they have a formal duty to act on behalf of… why aren’t you burning cop cars and shooting tax collectors?
And if you do think the government is anything other than a bunch of evil mindless parasites, please join with me and get that non-evil government to do its fucking job and credibly and reliably and systematically protect humans from robot-caused extinction.
I read your perspective that you’ve elaborated on at some considerable length, and it’s more than a little frustrating that you get so close to understanding mine, that you describe what seems like a perfectly reasonable model of my views, and then go “surely not,” so I shall be a little terse; that should leave less room for well-meaning misinterpretation.
if the government is just evil, and anarchism is correct
Yes.
I’d expect it to be relatively clear to show this to nearly all “right thinking people” using reasonable person standards of explanation, and then “we” could just notice that, and overthrow the evil government… right?
I wouldn’t. I didn’t believe it until only recently (I used to be a minarchist), so I see just how difficult it is to show this.
Also,“overthrow the evil government” sounds like “just switch off the misaligned AGI.”
there is no at-least-semi-benevolent person-shaped entity in that “Rulership Slot”
There isn’t.
should all just agree, on the internet, to simultaneously all not pay taxes,
We should.
if you do think the government is anything other than a bunch of evil mindless parasites
I don’t. (Well, okay, maybe the county sheriffs of some small towns. But nobody relevant.)
how you complete the sentence “l’etat c’est <entity>”
If you insist that I complete this French sentence you keep riffing on, “l’etat c’est Moloch.”
why aren’t you burning cop cars and shooting tax collectors?
This is an eminently reasonable question I ask myself everyday: call it cowardice, but I don’t think it will accomplish anything, and I’d rather be alive than dead.
As a side note, this fundamental misunderstanding reminds of how Bioshock became so beloved by libertarians: the intended response to the “no gods or kings, only man” message was “Ah, I see how good and necessary the state is,” not “No king, no king, la la la la la la.”
I beg the tolerance of anyone who sees these two very long comments.
I personally found it useful to learn “yet another of my interlocutors who seems to be opposed to AI regulations has just turned out to just be basically an anarchist at heart”.
Also, Shankar and I have started DMing a bunch, to look for cruxes, because I really want to figure out how Anarchist Souls work, and he’s willing to seek common epistemic ground, and so hopefully I’ll be able to learn something in private, and me and Shankar can do some “adversarial collaboration” (or whatever), and eventually we might post something in public that lists our agreements and “still unresolved cruxes”… or something? <3
In the meantime, please don’t downvote him (or me) too harshly! I don’t think we will be “polluting the signal/noise commons of LW” much more until the private conversation resolves? Hopefully? :-)
“You wouldn’t make that kind of mistake about an AI” is a good line. Almost no one else thinks of the state as an eldritch horror, or even refers to it as such in jest, like they do about AI systems. They should.
“L’état c’est nous” though? (The state, it is us.)
I’m pretty sure I am not an eldritch horror and I suspect you aren’t either, Shankar! Does the “eldritch horror part” arises from our composition? Is so, why and how? Maybe it is an aspect of humans that emerges somehow from a large number of humans?
“L’état ce ne sont que des bureaucrates humains” is another possibility (the state, it is merely some human bureaucrats) who I fully admit might not be saints, and might not be geniuses, but at least they are humans, operating at human speeds, with human common sense and the moral intuitions that arise in a human limbic system in the loop.
We have a lot of data on what instances from the class of human governments look like! The distribution is not fundamentally alien and unpredictable.
AGI really will be fundamentally different, in a statistical, material, mechanical, temporal, and algorithmic sense.
Scenario C, with an approach where “the regulation is deeply conservative and in every case of ambiguity bans things rather than permits them” seemed the most promising to me, and the OP’s central problem there is only limited by “international agreement”… which existing nation states know how to at least sort of do.
Calling it “eldritch” is mere rhetorical flourish to evoke Lovecraft; of course it’s not literally paranormal.
Asking which individual is responsible for the evil of the state is like asking which transistor in the AGI is misaligned. That kind of reductionism obviously won’t get you a reasonable answer, and you know it.
The problem is the incorrigibility of the system; it’s the same idea as Scott Alexander’s Meditations on Moloch: ultimately, it’s a coordination problem that giving a name to helps reify in the human mind. In this context, I like quoting 1 Samuel 8:11–18, slightly paraphrased.
Upon consideration, a lighthearted example might be better than Cabaret at showing how such a system can get out of control: Daylight Saving Time. The US government can’t get rid of it, and even I can’t find a way to blame that on bureaucrats.
As for “the state, it is us,” I guess I don’t understand what you mean. Sure, in the same sense the serfs are the estate; the conscripts, the army; and the slaves, the plantation, but that’s not really what one means when denouncing the state. But if it’s a way of transferring moral culpability for the state’s actions, I reject that entirely: I do not blame a slave for his submission.
I don’t think the moral intuitions of the people involved are really relevant, for the reasons above, but more importantly, you say that looking backwards through human history, the distribution is not fundamentally unpredictable: that’s not how predictions work! To people who had never seen a state before, I claim it would have indeed have been alien to see, for example, fishermen killing whales for neither meat nor blubber, but to fill quotas.
If AGI is indeed as “fundamentally different” as you worry from anything that came before it, I do not consider it as unlikely to be better than the state as it exists today as you do.
Scenario C seemed the most promising to me too, but for different reasons.
When you say “I do not blame a slave for his submission” regarding Daylight Savings Time, that totally works in the second frame where “l’état ce ne sont que des bureaucrates humains”.
You’re identifying me, and you, and all the other “slaves” as “mere transistors in society”.
I dunno about you, but I grew up in a small (unincorporated) town, that was run by the Chamber of Commerce and Rotary Club. My first check as pay for labor was as a soccer referee when I was 12, reffing seven year olds. There was a Deputy of the County Sheriff, but he was not the state. He was very mindful around the communitarian influencers of the town.
Of my town, it made total sense to me to imagine “l’état c’est nous”.
Whether you were Democrat, Green, Republican, or Libertarian in your party registration were were all democrats, not peasants. Yes?
In my town, PTA moms would make sure the Deputy got a bad guy if the gossip network noticed that there was a schizophrenic building a machine gun in his garage (or whatever).
I road my bike around, and picked wild fruit from overgrown orchards.
A chain tried to open a Kwik-E-Mart in town, and they didn’t donate to the PTA’s Halloween Carnival, and so I wasn’t allowed to shop there, as a child, for years until they started donating, which they eventually did.
I can’t imagine any polity actually working where the rulers (and in a democratic republic that’s the voters) did NOT have moral intuitions engaged with their regulation of themselves and their neighbors.
I can imagine people’s moral intuitions not connecting to the successful operation of their own government if the government is just evil, and anarchism is correct… but I don’t think the government is evil and therefore I don’t think anarchism is correct!
And if I did think that, then I’d expect it to be relatively clear to show this to nearly all “right thinking people” using reasonable person standards of explanation, and then “we” could just notice that, and overthrow the evil government… right?
As I see it, Russia and China are places where the common man is tragically disengaged from governance.
And because they do not have an autonomous citizenry, they lack people who have trust in their own ability to engage in civic reasoning towards civic ends.
There is only one real “citizen” in each of those countries: Putin and Xi respectively… who need not recognize anyone else. (Note: this actually makes securing AI in those countries easier! Convince “the one guy” and… I think then it will basically happen!?! (From their perspective, I bet we seem like the chaos monkeys.))
Now, I am not personally a neo-con right now.
My current posteriors say: maybe not all currently existing cultures are capable of healthy democratic self-rule.
However, when I think of “straightforward” solutions to AGI regulation my first thought is to set up a Global Parliament of 71 representatives, with proportional representation and quadratic monetary voting as a funding mechanism. I know there are more countries than 71 on the whole planet, but South American only has ~425 million people and ~2 languages.
Make the new thing a “House of Commons” that lets the “United Nations” function as a sort of auxiliary “Senate” or “House of Lords” or whatever (for ceremonial bootstrapping)?
Have that group hold elections over and over until they can find a Prime Minister of Earth who fulfills the Condorcet Criterion.
I bet that PM would be in favor of “AI-not-killing-everyone-ism” because “AI-killing-everyone-ism” seems like a pretty funky platform!
Personally, I would be open to paying every human who can speak (but not every AI and not every orca and not every african gray parrot) a silver dime to participate in voting.
I’d make the first vote cost (in USD?) cost $1, then $2.71, $7.39, $20.09, $54.60, $148.41 and so on with powers of “e”.
People would be able to give the dime back to get more votes… but personally I’d keep my dime as a “globally patriotic” keepsake and just pay for my votes in fiat.
I think America and Japan and Korea and Europe and so on have enough rich people that if there’s a couple billion idiots clogging up the voting system… those idiots will be poor enough to lose out in the final vote count? And I honestly thing the idiots would wise up pretty fast! <3
If we could finish the “first try” before the end of 2023, that would be amazing! Then do another in 2024. Then do it every year for 10 years, just to find the balances, and learn the skills!
Then my representative in Earth’s parliament could ask Earth’s Prime Minister, each year, what the fuck was up with AI, and maybe the PM would say (into the cameras in the Chamber) that the global government can’t use all that quadratic voting money to come up with ANY way to work with the pre-existing westphalian states to globally prevent humans from getting murdered by a scary AI in any way at all ever...
...and then maybe we deserve to die?
Or (I think to myself) if that’s what YOU all end up electing, then maybe YOU ALL deserve to die? I don’t wanna die, see? I just want nice normal things, like clean neighborhoods and happy kids n’stuff :-)
So at that point imaginary point, where global governance has actually been tried and it didn’t work… I might be ready to say that “my right to self defense is more important than the rest of humanity’s right to the consent of the governed”…
...and then I’d start looking for someone who has a solution for “the problem of literally everyone on earth maybe getting killed in a small number of years by a bunch of poorly coded machines” who could credibly say “l’etat c’est moi” like in the bad old days?
I don’t know basically anything about Napoleon the Eighth, but if he is an adequate focal point who can read a credible plan for dealing with AI off of a teleprompter...
...then maybe I’d give up on democracy, and bend the knee to him, and hope desperately that his “Butlerian” War is relatively peaceful and fast and successful???
I really really really hope it doesn’t get to that point, but if it does… well… I’m pretty sure I have a right to self defense???
And maybe that right (and the similar rights of everyone else who wants to be safe from AI) trumps the moral requirement to secure the consent of governed people?
But I would expect any unelected monarch to still take deontics strongly into account, even if granted insane powers by a crazy situation. The people who swear their sword to such a ruler would care about honor, I’m pretty sure...
Maybe you can explain to me, Shankar, how you complete the sentence “l’etat c’est <entity>” such that the <entity> seems to you like an entity that is at least vaguely person-shaped and able to track its own interests, and pursue its real interests somewhat effectively.
(And if there is no such person-shaped thing in charge of anything, then who the fuck are we paying taxes to? If there is no at-least-semi-benevolent person-shaped entity in that “Rulership Slot”, doesn’t that mean we could (and maybe should?) all just agree, on the internet, to simultaneously all not pay taxes, and then we’d all get away with it, and then we could sing songs about it or something?)
This is all intensely practical from my perspective.
Its the same “mental muscles” that hold a marriage together, or keep seven year olds from hurting each other while they play soccer, or let a group house elect a kitchen manager who buys food for the whole house that makes the vegans and the carnists happy, because everyone has a right to eat food they’ve paid for, and collective buying is a good way to get good deals for low prices!
If you’re not modeling government as a best-effort pretty-OK “buyers club for public services” run by people who care about the people they have a formal duty to act on behalf of… why aren’t you burning cop cars and shooting tax collectors?
And if you do think the government is anything other than a bunch of evil mindless parasites, please join with me and get that non-evil government to do its fucking job and credibly and reliably and systematically protect humans from robot-caused extinction.
I read your perspective that you’ve elaborated on at some considerable length, and it’s more than a little frustrating that you get so close to understanding mine, that you describe what seems like a perfectly reasonable model of my views, and then go “surely not,” so I shall be a little terse; that should leave less room for well-meaning misinterpretation.
Yes.
I wouldn’t. I didn’t believe it until only recently (I used to be a minarchist), so I see just how difficult it is to show this.
Also, “overthrow the evil government” sounds like “just switch off the misaligned AGI.”
There isn’t.
We should.
I don’t. (Well, okay, maybe the county sheriffs of some small towns. But nobody relevant.)
If you insist that I complete this French sentence you keep riffing on, “l’etat c’est Moloch.”
This is an eminently reasonable question I ask myself everyday: call it cowardice, but I don’t think it will accomplish anything, and I’d rather be alive than dead.
As a side note, this fundamental misunderstanding reminds of how Bioshock became so beloved by libertarians: the intended response to the “no gods or kings, only man” message was “Ah, I see how good and necessary the state is,” not “No king, no king, la la la la la la.”
I beg the tolerance of anyone who sees these two very long comments.
I personally found it useful to learn “yet another of my interlocutors who seems to be opposed to AI regulations has just turned out to just be basically an anarchist at heart”.
Also, Shankar and I have started DMing a bunch, to look for cruxes, because I really want to figure out how Anarchist Souls work, and he’s willing to seek common epistemic ground, and so hopefully I’ll be able to learn something in private, and me and Shankar can do some “adversarial collaboration” (or whatever), and eventually we might post something in public that lists our agreements and “still unresolved cruxes”… or something? <3
In the meantime, please don’t downvote him (or me) too harshly! I don’t think we will be “polluting the signal/noise commons of LW” much more until the private conversation resolves? Hopefully? :-)
The state is largely run by people who seek power and fame. That is importantly different from most of us.