I’ve been thinking about some of the issues with CEV. It’s come up a few times that humanity might not have a coherent, non-contradictory set of values. And the question of how to come up with some set of values that best represents everyone.
It occurs to me that this might be a problem mathematicians have already solved, or at least given a lot of thought. In the form of voting systems. Voting is a very similar problem. You have a bunch of people you want to represent fairly, and you need to select a leader that best represents their interests.
My favorite alternative voting system is the Condorcet Method. Basically it compares each candidate in a 1v1 election, and selects the candidate that would have won every single election.
It is possible for there not to be a Condorcet winner. If the population has circular preferences. Candidate A > Candidate B > C > A… Like a rock paper scissors thing.
To solve this there are a number of methods developed to select the best compromise. My favorite is Minimax. It selects the candidate who’s greatest loss is the least bad. I think that’s the most desirable way to pick a winner, and it’s also super simple.
There are some differences. Instead of a leader, we want the best set of values and policies for the AI to follow. And there might not be a finite set of candidates, but an infinite number of possibilities. And actually voting might be impractical. Instead an AI might have to predict what you would have voted, if you knew all the arguments and had much time to think about it and come to a conclusion. But I think it can still be modeled as a voting problem.
Now this isn’t actually something we need to figure out now. If we somehow had an FAI, we could probably just ask it to come up with the most fair way of representing everyone’s values. We probably don’t need to hardcode these details.
The bigger issue is why would the person or group building the FAI even bother to do this? They could just take their own CEV and ignore everyone elses. And they have every incentive to do this. It might even be significantly simpler than trying to do a full CEV of humanity. So even if we do solve FAI, humanity is probably still screwed.
EDIT: After giving it some more thought, I’m not sure voting systems are actually desirable. The whole point of voting is that people can’t be trusted to just specify their utility functions. The perfect voting system would be for each person to give a number to each candidate based on how much utility they’d get from them being elected. But that’s extremely susceptible to tactical voting.
However with FAI, it’s possible we could come up with some way of keeping people honest, or peering into their brains and getting their true value function. That adds a great deal of complexity though. And it requires trusting the AI to do a complex, arbitrary, and subjective task. Which means you must have already solved FAI.
If I were God of the World, I would model the problem as more of a River Crossing Puzzle. How do you get things moving along when everyone on the boat wants to kill each other? Segregation! Resettling humanity mapped over a giant Venn diagram is trivial once we are all uploaded, but it also runs into ethical problems; just as voting and enacting the will of the majority (or some version thereof) is problematic, so is setting up the world so that the oppressor and the oppressee will never be allowed meet. However, in my experience people are much happier with rules like “you can’t go there” and much less happier with rules like “you have to do what that guy wants”. This is probably due to our longstanding tradition of private property.
This makes some assumptions as to what the next world will look like, but I think that it is a likely outcome—it is always much easier to send the kids to their rooms than to hold a family court, and I think a cost/benefit analysis would almost surely show that it is not worth trying to sort out all human problems as one big happy group.
Of course, this assumes that we don’t do something crazy like include democracy and unity of the human race as terminal values.
The local population consists of 80% blue people and 20% orange people. For some reason, the blue people dislike orange people. A blue leader arises who says “We must kill all the orange people and take their stuff!” Well, it’s an issue, and how do people properly decide on a policy? By voting, of course. Everyone votes and the policy passes by simple majority. And so the blue people kill all the orange people and take their stuff. The end.
This is exactly the type of problems that mathematicians have tried to solve with different voting schemes. One recent example that has the potential to solve this problem is quadratic vote buying, which takes into account strong preferences of minorities.
This is exactly the type of problems that mathematicians have tried to solve
I am not sure this is a mathematical problem. Generally speaking, giving a minority the veto power trades off minority safety against government ability to do things. In the limit you have decision making by consensus which has obvious problems.
quadratic vote buying
What do you buy votes with? Money? Then it’s an easy way for the blue people to first take orange people’s stuff and then, once the orange people run out of resources to buy votes with, to kill them anyway.
Generally speaking, giving a minority the veto power trades off minority safety against government ability to do things. In the limit you have decision making by consensus which has obvious problems.
That’s precisely why it is a mathematical problem… you need to quantify the tradeoffs, and figure out which voting schemes maximize different value schemes and utility functions. Math can’t SOLVE this problem because it’s a ought problem, not an is problem.
But you can’t answer the ought side of things without first knowing the is side.
In terms of quadratic vote buying, money is only one way to do it, another is to have an artificial or digital currency just for vote buying, for which people get a fixed amount for the year.
I don’t think your concept of it really makes sense in the context of modern government with a police force, international oversight, etc. All voting schemes break down when you assume a base state of anarchy—but assuming there’s already a rule of law in place, you can maximize how effective those laws are (or the politicians who make them) by changing your voting rules.
That’s precisely why it is a mathematical problem… Math can’t SOLVE this problem
Ahem.
in the context of modern government with a police force, international oversight, etc.
I would be quite interested to learn who exerts “international oversight” over, say, USA.
Besides, are you really saying a “modern” government can do no wrong??
assuming there’s already a rule of law in place, you can maximize how effective those laws are
I’m sorry, I’m not talking about the executive function of the government which merely implements the laws, I’m talking about the legislative function which actually makes the laws. There is no assumption of the base state of anarchy.
You claimed this is a mathematical problem, but in the next breath said that math can’t solve it. Then what was the point of claiming it to be a math problem in the first place? Just because dealing with it involves numbers? That does not make it a math problem.
The UN
LOL. Can we please stick a bit closer to the real world?
Would a historical example of what you’re talking about be the legality of slavery?
Actually, the first example that comes to mind is the when the US decided that all Americans who happen to be of Japanese descent and have the misfortune to live on the West Coast need to be rounded up and sent to concentration, err.. internment camps.
I’m not sure that’s a fair problem to ascribe to voting. If >50% of that populace wants to kill the orange folks its going to happen, however they select their leaders. It isn’t voting’s fault that this example is filled with maniacs.
But maybe that’s the correct outcome? If 80% of the population truly believes that some people should die, maybe they should. What higher authority can we appeal to?
I’m not saying I think minorities should die. But I also don’t think the majority thinks that either. So it’s just an absurd hypothetical. You could say the same thing about CEV in general. “We shouldn’t take the utility function of humanity, because what if it’s bad?” Bad according to what? What higher utility function are we using to determine badness? Some individual’s?
I think Condorcet voting is the best way to compromise between a lot of different people’s values. It tends to favor moderates and compromises. Especially the Minimax method i mentioned.
I don’t think this system is great, I just think it’s the best we can possibly do.
“We shouldn’t take the utility function of humanity, because what if it’s bad?” Bad according to what? What higher utility function are we using to determine badness? Some individual’s?
You’ll have to convince me that taking other people’s utility function into account is consistent with my utility function.
It’s not. I literally discussed that in my first comment. If you can become dictator, it’s definitely in your interest to do so. Instead of turning power over to a democracy.
But I would much rather live under a democracy than a dictatorship where I’m not dictator.
But maybe that’s the correct outcome? If 80% of the population truly believes that some people should die, maybe they should. What higher authority can we appeal to?
Really?
So it’s just an absurd hypothetical.
I wonder if you heard the word “genocide” before. Not in the context of hypotheticals, but as a recurring feature of human history.
If 80% of the population has a certain value, how can you say that value is wrong? Statistically you are far more likely to be in that 80%.
And the alternative isn’t “you get to be dictator and have all your values maximized without compromise”. It’s “some random individual is picked from the population and gets his values maximized over everyone else’s.” Democracy of values is far preferable.
I wonder if you heard the word “genocide” before. Not in the context of hypotheticals, but as a recurring feature of human history.
By functioning democracies? With a perfectly rational and informed population?
That’s the important part of CEV, or at least my interpretation of it. The AI predicts what you would decide, if you knew all the relevant information and had plenty of time to think about it. I’m not suggesting a regular democracy where the voters barely know anything.
Indeed, it is not. The question mark at the end might indicate that it is a question.
If 80% of the population has a certain value, how can you say that value is wrong?
I don’t see any problems with this whatsoever. I am not obligated to convert to the values of the majority. What is the issue that you see?
By functioning democracies?
There is a bit of a true Scotsman odor to this question :-) but let me point out my example upthread and ask you whether the Nazi party came to power democratically.
AI predicts what you would decide
At this level you might as well cut to the chase and go straight to “I wish for you to do what I should wish for”. No need to try to tell God… err.. AI how to do it.
I don’t see any problems with this whatsoever. I am not obligated to convert to the values of the majority. What is the issue that you see?
And they aren’t obligated to convert to your values. Not everyone can have their way! Democratic voting is the fairest way of making a decision when people can’t agree.
There is a bit of a true Scotsman odor to this question :-) but let me point out my example upthread and ask you whether the Nazi party came to power democratically.
Yes I know it’s No-True-Scotsman-y, but I really believe that a totally informed population would make very different decisions than an angry mob during a war and depression.
And even your examples are not convincing. Internment during wartime wasn’t anywhere near the level of genocide. And the Nazi election was far from fair:
...the Nazis “unleashed a campaign of violence and terror that dwarfed anything seen so far.” Storm troopers began attacking trade union and Communist Party (KPD) offices and the homes of left-wingers. In the second half of February, the violence was extended to the Social Democrats, with gangs of brownshirts breaking up Social Democrat meetings and beating up their speakers and audiences. Issues of Social Democratic newspapers were banned. Twenty newspapers of the Centre Party, a party of Catholic Germans, were banned in mid-February for criticizing the new government. Government officials known to be Centre Party supporters were dismissed from their offices, and stormtroopers violently attacked party meetings in Westphalia.
Six days before the scheduled election date, the German parliament building was set alight in the Reichstag fire, allegedly by the Dutch Communist Marinus van der Lubbe. This event reduced the popularity of the KPD… This emergency law removed many civil liberties and allowed the arrest of… 4,000 leaders and members of the KPD shortly before the election, suppressing the Communist vote and consolidating the position of the Nazis. The KPD was effectively outlawed...
The resources of big business and the state were thrown behind the Nazis’ campaign to achieve saturation coverage all over Germany. Brownshirts and SS patrolled and marched menacingly through the streets of cities and towns. A “combination of terror, repression and propaganda was mobilized in every… community, large and small, across the land.” To further ensure the outcome of the vote would be a Nazi majority, Nazi organizations “monitored” the vote process. In Prussia 50,000 members of the SS, SA and Stahlhelm were ordered to monitor the votes as deputy sheriffs by acting Interior Minister Hermann Göring.
.
At this level you might as well cut to the chase and go straight to “I wish for you to do what I should wish for”. No need to try to tell God… err.. AI how to do it.
Well I did mention that in my first comment. This is more of an aesthetic thing to talk about. Once we have an AI we can just ask it how to solve this problem.
But I still think it’s somewhat important to think about. Because if we go with your solution, we just get whatever the creator of the AI wants. He becomes supreme dictator of the universe forever, and forces his values on everyone for eternity. I would much rather have CEV or something like it.
Democratic voting is the fairest way of making a decision when people can’t agree.
That sounds like an article of faith.
“Fair” is a very… relative world. Calling something “fair” rarely means more than “I like / approve of it”.
This is more of an aesthetic thing to talk about.
Ah. Well, speaking aesthetically, I find the elevation of mob rule to be the ultimate moral principle ugly and repugnant. Y’know, de gustibus ’n’all...
Well see my edit to my first comment. I’ll paste it here:
After giving it some more thought, I’m not sure voting systems are actually desirable. The whole point of voting is that people can’t be trusted to just specify their utility functions. The perfect voting system would be for each person to give a number to each candidate based on how much utility they’d get from them being elected. But that’s extremely susceptible to tactical voting.
However with FAI, it’s possible we could come up with some way of keeping people honest, or peering into their brains and getting their true value function. That adds a great deal of complexity though. And it requires trusting the AI to do a complex, arbitrary, and subjective task. Which means you must have already solved FAI.
Do you agree that the fairest system would be to combine everyone’s utility functions and maximize them? Of course somehow giving everyone equal weight to avoid utility monsters and other issues. I think these issues can be worked out.
If so, do you agree that voting systems are the best compromise when you can’t just read people’s utility functions? And need to worry about tactical voting? Because that is basically what I was getting at.
If you don’t agree to the above, then I don’t understand your objection. CEV is about somehow finding the best compromise of all humans’ utility functions. About combining them all. All I’m talking about is more concrete methods of doing that.
Do you agree that the fairest system would be to combine everyone’s utility functions and maximize them? Of course somehow giving everyone equal weight to avoid utility monsters and other issues. I think these issues can be worked out.
Anything you can do maximizes some combination of people’s utility functions. So it is trivially true that the fairest system is a system which uses some combination of people’s utility functions. Unless you can first describe how you are going to avoid utility monsters and other perils of utilitarianism, you really haven’t said anything useful.
Do you agree that the fairest system would be to combine everyone’s utility functions and maximize them?
No, I do not. I do not think that humans have coherent utility functions. I don’t think utilities of different people can be meaningfully combined, too.
I’ve been thinking about some of the issues with CEV. It’s come up a few times that humanity might not have a coherent, non-contradictory set of values. And the question of how to come up with some set of values that best represents everyone.
It occurs to me that this might be a problem mathematicians have already solved, or at least given a lot of thought. In the form of voting systems. Voting is a very similar problem. You have a bunch of people you want to represent fairly, and you need to select a leader that best represents their interests.
My favorite alternative voting system is the Condorcet Method. Basically it compares each candidate in a 1v1 election, and selects the candidate that would have won every single election.
It is possible for there not to be a Condorcet winner. If the population has circular preferences. Candidate A > Candidate B > C > A… Like a rock paper scissors thing.
To solve this there are a number of methods developed to select the best compromise. My favorite is Minimax. It selects the candidate who’s greatest loss is the least bad. I think that’s the most desirable way to pick a winner, and it’s also super simple.
There are some differences. Instead of a leader, we want the best set of values and policies for the AI to follow. And there might not be a finite set of candidates, but an infinite number of possibilities. And actually voting might be impractical. Instead an AI might have to predict what you would have voted, if you knew all the arguments and had much time to think about it and come to a conclusion. But I think it can still be modeled as a voting problem.
Now this isn’t actually something we need to figure out now. If we somehow had an FAI, we could probably just ask it to come up with the most fair way of representing everyone’s values. We probably don’t need to hardcode these details.
The bigger issue is why would the person or group building the FAI even bother to do this? They could just take their own CEV and ignore everyone elses. And they have every incentive to do this. It might even be significantly simpler than trying to do a full CEV of humanity. So even if we do solve FAI, humanity is probably still screwed.
EDIT: After giving it some more thought, I’m not sure voting systems are actually desirable. The whole point of voting is that people can’t be trusted to just specify their utility functions. The perfect voting system would be for each person to give a number to each candidate based on how much utility they’d get from them being elected. But that’s extremely susceptible to tactical voting.
However with FAI, it’s possible we could come up with some way of keeping people honest, or peering into their brains and getting their true value function. That adds a great deal of complexity though. And it requires trusting the AI to do a complex, arbitrary, and subjective task. Which means you must have already solved FAI.
If I were God of the World, I would model the problem as more of a River Crossing Puzzle. How do you get things moving along when everyone on the boat wants to kill each other? Segregation! Resettling humanity mapped over a giant Venn diagram is trivial once we are all uploaded, but it also runs into ethical problems; just as voting and enacting the will of the majority (or some version thereof) is problematic, so is setting up the world so that the oppressor and the oppressee will never be allowed meet. However, in my experience people are much happier with rules like “you can’t go there” and much less happier with rules like “you have to do what that guy wants”. This is probably due to our longstanding tradition of private property.
This makes some assumptions as to what the next world will look like, but I think that it is a likely outcome—it is always much easier to send the kids to their rooms than to hold a family court, and I think a cost/benefit analysis would almost surely show that it is not worth trying to sort out all human problems as one big happy group.
Of course, this assumes that we don’t do something crazy like include democracy and unity of the human race as terminal values.
This puts me in mind of Eliezer’s “Failed Utopia #4-2”.
Not quite.
The local population consists of 80% blue people and 20% orange people. For some reason, the blue people dislike orange people. A blue leader arises who says “We must kill all the orange people and take their stuff!” Well, it’s an issue, and how do people properly decide on a policy? By voting, of course. Everyone votes and the policy passes by simple majority. And so the blue people kill all the orange people and take their stuff. The end.
This is exactly the type of problems that mathematicians have tried to solve with different voting schemes. One recent example that has the potential to solve this problem is quadratic vote buying, which takes into account strong preferences of minorities.
I am not sure this is a mathematical problem. Generally speaking, giving a minority the veto power trades off minority safety against government ability to do things. In the limit you have decision making by consensus which has obvious problems.
What do you buy votes with? Money? Then it’s an easy way for the blue people to first take orange people’s stuff and then, once the orange people run out of resources to buy votes with, to kill them anyway.
That’s precisely why it is a mathematical problem… you need to quantify the tradeoffs, and figure out which voting schemes maximize different value schemes and utility functions. Math can’t SOLVE this problem because it’s a ought problem, not an is problem.
But you can’t answer the ought side of things without first knowing the is side.
In terms of quadratic vote buying, money is only one way to do it, another is to have an artificial or digital currency just for vote buying, for which people get a fixed amount for the year.
I don’t think your concept of it really makes sense in the context of modern government with a police force, international oversight, etc. All voting schemes break down when you assume a base state of anarchy—but assuming there’s already a rule of law in place, you can maximize how effective those laws are (or the politicians who make them) by changing your voting rules.
Ahem.
I would be quite interested to learn who exerts “international oversight” over, say, USA.
Besides, are you really saying a “modern” government can do no wrong??
I’m sorry, I’m not talking about the executive function of the government which merely implements the laws, I’m talking about the legislative function which actually makes the laws. There is no assumption of the base state of anarchy.
This isn’t helpful. There’s nothing for me to respond to.
The UN (specifically, other very powerful countries that trade with the US).
Would a historical example of what you’re talking about be the legality of slavery?
Let me unroll my ahem.
You claimed this is a mathematical problem, but in the next breath said that math can’t solve it. Then what was the point of claiming it to be a math problem in the first place? Just because dealing with it involves numbers? That does not make it a math problem.
LOL. Can we please stick a bit closer to the real world?
Actually, the first example that comes to mind is the when the US decided that all Americans who happen to be of Japanese descent and have the misfortune to live on the West Coast need to be rounded up and sent to concentration, err.. internment camps.
Problems can have a mathematical aspect without being completely solvable by math.
I’m not sure that’s a fair problem to ascribe to voting. If >50% of that populace wants to kill the orange folks its going to happen, however they select their leaders. It isn’t voting’s fault that this example is filled with maniacs.
But maybe that’s the correct outcome? If 80% of the population truly believes that some people should die, maybe they should. What higher authority can we appeal to?
I’m not saying I think minorities should die. But I also don’t think the majority thinks that either. So it’s just an absurd hypothetical. You could say the same thing about CEV in general. “We shouldn’t take the utility function of humanity, because what if it’s bad?” Bad according to what? What higher utility function are we using to determine badness? Some individual’s?
I think Condorcet voting is the best way to compromise between a lot of different people’s values. It tends to favor moderates and compromises. Especially the Minimax method i mentioned.
I don’t think this system is great, I just think it’s the best we can possibly do.
You’ll have to convince me that taking other people’s utility function into account is consistent with my utility function.
It’s not. I literally discussed that in my first comment. If you can become dictator, it’s definitely in your interest to do so. Instead of turning power over to a democracy.
But I would much rather live under a democracy than a dictatorship where I’m not dictator.
Really?
I wonder if you heard the word “genocide” before. Not in the context of hypotheticals, but as a recurring feature of human history.
That’s not an argument.
If 80% of the population has a certain value, how can you say that value is wrong? Statistically you are far more likely to be in that 80%.
And the alternative isn’t “you get to be dictator and have all your values maximized without compromise”. It’s “some random individual is picked from the population and gets his values maximized over everyone else’s.” Democracy of values is far preferable.
By functioning democracies? With a perfectly rational and informed population?
That’s the important part of CEV, or at least my interpretation of it. The AI predicts what you would decide, if you knew all the relevant information and had plenty of time to think about it. I’m not suggesting a regular democracy where the voters barely know anything.
Indeed, it is not. The question mark at the end might indicate that it is a question.
I don’t see any problems with this whatsoever. I am not obligated to convert to the values of the majority. What is the issue that you see?
There is a bit of a true Scotsman odor to this question :-) but let me point out my example upthread and ask you whether the Nazi party came to power democratically.
At this level you might as well cut to the chase and go straight to “I wish for you to do what I should wish for”. No need to try to tell God… err.. AI how to do it.
And they aren’t obligated to convert to your values. Not everyone can have their way! Democratic voting is the fairest way of making a decision when people can’t agree.
Yes I know it’s No-True-Scotsman-y, but I really believe that a totally informed population would make very different decisions than an angry mob during a war and depression.
And even your examples are not convincing. Internment during wartime wasn’t anywhere near the level of genocide. And the Nazi election was far from fair:
.
Well I did mention that in my first comment. This is more of an aesthetic thing to talk about. Once we have an AI we can just ask it how to solve this problem.
But I still think it’s somewhat important to think about. Because if we go with your solution, we just get whatever the creator of the AI wants. He becomes supreme dictator of the universe forever, and forces his values on everyone for eternity. I would much rather have CEV or something like it.
That sounds like an article of faith.
“Fair” is a very… relative world. Calling something “fair” rarely means more than “I like / approve of it”.
Ah. Well, speaking aesthetically, I find the elevation of mob rule to be the ultimate moral principle ugly and repugnant. Y’know, de gustibus ’n’all...
I don’t believe I proposed any.
Well see my edit to my first comment. I’ll paste it here:
Do you agree that the fairest system would be to combine everyone’s utility functions and maximize them? Of course somehow giving everyone equal weight to avoid utility monsters and other issues. I think these issues can be worked out.
If so, do you agree that voting systems are the best compromise when you can’t just read people’s utility functions? And need to worry about tactical voting? Because that is basically what I was getting at.
If you don’t agree to the above, then I don’t understand your objection. CEV is about somehow finding the best compromise of all humans’ utility functions. About combining them all. All I’m talking about is more concrete methods of doing that.
Anything you can do maximizes some combination of people’s utility functions. So it is trivially true that the fairest system is a system which uses some combination of people’s utility functions. Unless you can first describe how you are going to avoid utility monsters and other perils of utilitarianism, you really haven’t said anything useful.
No, I do not. I do not think that humans have coherent utility functions. I don’t think utilities of different people can be meaningfully combined, too.
Ah, yes, the famous business plan of the underpants gnomes...
No, I do not. They might be best given some definitions of “best” and given some conditionals, but they are not always best regardless of anything.
What makes you think it is possible?