You say that certain choices make our mutual agency possible, which suggests that other choices make it impossible. But you also say we can’t choose to deny our own agency, which suggests that our choices don’t affect whether our mutual agency exists or doesn’t. I’m not really sure what “mutual agency” is in this context, though.
If it helps, I agree that in the situation you describe, it’s important that we both be willing to stick to whatever agreement we make; without that, the agreements have no value. Whether that willingness derives from us valuing peaceful coexistence, or us valuing our reputations as word-keepers, or us believing that there’s a powerful third party likely to punish noncompliance, or whatever, doesn’t change the importance.
While writing a reply I realized that I was arguing a completely different point than I originally made. :-( And that’s bad.
While writing a (german) wikibook on rationality I came to the conclusion that in addition to epistemological and instrumental rationality there should be a separate notion of “discourse rationality”, that is, how to lead a rational discussion. And this is not it.
Now I’m not sure to what point to go back to start again.
Edit:
Ok, got it, it was my previous post that was confused.
What you were asking is, if I say, I have a rule that you do not make white people suffer the presence of black people and you come along and say that my rule is wrong, we should in fact welcome anybody’s participation in our society no matter the color, how do we decide who’s right?
And I am not entirely sure that the rules’ actual ability to enable peaceful coexistence is the correct way to decide, since it is conceivable that segregation of the races might actually be more peaceful, like if we could build an impenetrable wall in northern Ireland to separate the religious groups. But I would say that by doing so we are solving the wrong problem.
Instead I would base my decision on an idealized concept of an agent. And with this idealized concept it is clear that color or identification as belonging to a certain tribe does not affect agency and therefore cannot be morally relevant. But a right to life, liberty, property, etc. does in fact affect agency because it defines the things an agent can make choices about, so these are moral questions. So in the above scenario I would have to agree that your rule is the better one, but not because of its consequences but because of its properties.
This notion of “agency” is doing a lot of work in this account, and I have to admit I don’t really understand it.
I understand that it’s not the same thing as preference, and it’s not the same thing as volition, and it’s not related to things like ethnicity or nationality or upbringing and thus is not the same thing as values (which do depend on those things). But those are all negative statements, which are only marginally helpful.
Approaching it the other way: R2 is better than R1, you say, because of properties of R2… which are based on an idealized concept of an agent. All we know about R2 is that it meets the “minimal set of rules that make peaceful coexistence possible” standard better than R1 does. So those two things are presumably related in some way… but I don’t grasp the relation.
All we know about R2 is that it meets the “minimal set of rules that make peaceful coexistence possible” standard better than R1 does.
It’s difficult to think about that without an example. Ideally the reason why you don’t do certain things to other people should imply what those things are.
This notion of “agency” is doing a lot of work in this account, …
Yes, I was using agency to replace “being human”. I think we are moral because we recognize other people as humans like ourselves and use the same brain circuits to model what is happening to them that we use if it were happening to us, thus comes the golden rule morality.
From that I was thinking, is there something that makes us humans special which could actually justify such an approach? And I came up with agency, which I guess is the ability to make a conscious choice of action. So there are three parts to agency. A set of actions, the ability to choose not just based upon the current state of the world but based on preferences of expected consequences, and thirdly consciousness as ability to make meta-level choices. Maybe there needs to be something about learning there as well.
If I think about this notion of agency and try to come up with moral ground rules that are suggested by it, I come up with “life, liberty, and the pursuit of happiness” what is of course not a set of rules, but it’s a more or less direct translation of my three parts of agency into a moral-ish language. How to get from there to a specific set of rules is something I don’t know but think should be possible. And how to choose if different sets of rules would satisfy the purpose is also something I don’t know. It could be that in this case it doesn’t actually matter.
So, you’ve described a human preference for having the things we’d want to happen to us also happen to systems we recognize as sufficiently like ourselves. Call that preference P.
A preference utilitarian would say that the moral value of a choice is proportional to the degree to which P is satisfied by that choice. (All else being equal.)
If I’ve understood you correctly, you reject preference utilitarianism as a moral framework. Instead, you suggest a deontological framework based on “agency.” And agency is a concept you came up with to encapsulate whatever properties humans have that “justify” preferring P.
Have I followed you so far?
OK. Can you say more about how a preference is justified?
For example, you conclude that humans are justified in preferring P on the basis of various attributes of humans (the ability to take action based on expected consequences, the ability to make “meta-level” choices, “a set of actions,” and maybe something about learning). I infer you believe we’re _un_justified in preferring P on the basis of other attributes (say, skin color, or height above sea level, or tendency to slaughter other humans).
Is that right?
How did you arrive at those particular attributes?
So, you’ve described a human preference for having the things we’d want to happen to us also happen to systems we recognize as sufficiently like ourselves. Call that preference P.
A preference utilitarian would say that the moral value of a choice is proportional to the degree to which P is satisfied by that choice. (All else being equal.)
I was starting from my own intuitions about my moral preferences. But if you stop at treating morality as a preference you run into problems when people don’t share these preferences. A common variation might for example be that people believe it is good for the strong to prey on the weak. But with morality being an interpersonal thing, any morality must account for differences in preferences and therefore cannot be based in preferences. That’s why I reject preference utilitarianism.
And agency is a concept you came up with to encapsulate whatever properties humans have that “justify” preferring P.
My agency based morality does justify my moral preferences, but it doesn’t “just” justify my moral preferences. I only have my moral preferences as a starting point. From that I construct an abstract moral framework, check if that abstract framework satisfies conditions of consistency and plausibility, and after I’ve been convinced it does, use it to justify or adjust my moral preferences.
Other people might come to different conclusions using this process, but since now our moral framework is removed from mere preferences we can use properties of the frameworks in question to try and integrate them or decide between them. A preference utilitarian would have to resort to some unjustified selection method like majority vote.
So how do I come up with the properties of a moral framework that make it better than an other? I don’t know yet. I would suggest that minimalism is a good property. With a non minimal framework people could always ask, “why should we adopt this policy?”. With a minimal framework it’s either adopt all of it or don’t adopt it at all. I also justify agency as the primary motivation since our agency is what creates the problem in the first place. Without choice we have no use for morality. Without deliberation we couldn’t follow it. Without metalevel reasoning we couldn’t adopt it, etc. Short, agency is the very thing that creates a solvable problem of morality and thus is the best place to solve it. If we start to argue that point, then we are coming to a point where we run into gödelian incompleteness.
You keep tossing the word “justified” around, and I am increasingly unclear on how the work that you want that word to do is getting done.
For example: I agree with you that a preference utilitarian needs some mechanism for resolving situations where preferences conflict, but I’m not sure on what basis you conclude that such a mechanism must be unjustified, nor on what basis you conclude that your agency-based moral frameworks support a more justifiable method for integrating or deciding between different people’s conflicting framework-based-conclusions.
I find your “without X we wouldn’t have a problem and therefore X is the solution” argument unconvincing. Mostly it sounds to me like you’ve decided that your framework is cool, and now you’re looking for arguments to support it.
but I’m not sure on what basis you conclude that such a mechanism must be unjustified
I was thinking it needs to be separately justified and is not justified from the principle of preference utilitarianism.
I find your “without X we wouldn’t have a problem and therefore X is the solution” argument unconvincing.
It’s a basic principle of engineering to solve a problem where it occurs. I think we’ve reached the point where I am not prepared to argue any further and don’t think it would be fruitful to try. I thank you for the challenge.
Mostly it sounds to me like you’ve decided that your framework is cool, and now you’re looking for arguments to support it.
That might be the case but I don’t think it likely. I am an asshole enough to do what I want even without moral justification and I am a cynic enough not to expect anything else from other people. I was writing my original comment merely as an additional comment to the morality debate on Less Wrong because I believe that if Eliezer would create his FAI tomorrow it wouldn’t be friendly towards me. The rest was just trying to answer your questions because I really think they helped me to think it through.
I also am confused by what you’re saying here.
You say that certain choices make our mutual agency possible, which suggests that other choices make it impossible. But you also say we can’t choose to deny our own agency, which suggests that our choices don’t affect whether our mutual agency exists or doesn’t. I’m not really sure what “mutual agency” is in this context, though.
If it helps, I agree that in the situation you describe, it’s important that we both be willing to stick to whatever agreement we make; without that, the agreements have no value. Whether that willingness derives from us valuing peaceful coexistence, or us valuing our reputations as word-keepers, or us believing that there’s a powerful third party likely to punish noncompliance, or whatever, doesn’t change the importance.
While writing a reply I realized that I was arguing a completely different point than I originally made. :-( And that’s bad.
While writing a (german) wikibook on rationality I came to the conclusion that in addition to epistemological and instrumental rationality there should be a separate notion of “discourse rationality”, that is, how to lead a rational discussion. And this is not it.
Now I’m not sure to what point to go back to start again.
Edit:
Ok, got it, it was my previous post that was confused.
What you were asking is, if I say, I have a rule that you do not make white people suffer the presence of black people and you come along and say that my rule is wrong, we should in fact welcome anybody’s participation in our society no matter the color, how do we decide who’s right?
And I am not entirely sure that the rules’ actual ability to enable peaceful coexistence is the correct way to decide, since it is conceivable that segregation of the races might actually be more peaceful, like if we could build an impenetrable wall in northern Ireland to separate the religious groups. But I would say that by doing so we are solving the wrong problem.
Instead I would base my decision on an idealized concept of an agent. And with this idealized concept it is clear that color or identification as belonging to a certain tribe does not affect agency and therefore cannot be morally relevant. But a right to life, liberty, property, etc. does in fact affect agency because it defines the things an agent can make choices about, so these are moral questions. So in the above scenario I would have to agree that your rule is the better one, but not because of its consequences but because of its properties.
This notion of “agency” is doing a lot of work in this account, and I have to admit I don’t really understand it.
I understand that it’s not the same thing as preference, and it’s not the same thing as volition, and it’s not related to things like ethnicity or nationality or upbringing and thus is not the same thing as values (which do depend on those things). But those are all negative statements, which are only marginally helpful.
Approaching it the other way: R2 is better than R1, you say, because of properties of R2… which are based on an idealized concept of an agent. All we know about R2 is that it meets the “minimal set of rules that make peaceful coexistence possible” standard better than R1 does. So those two things are presumably related in some way… but I don’t grasp the relation.
I’m still pretty confused, here.
It’s difficult to think about that without an example. Ideally the reason why you don’t do certain things to other people should imply what those things are.
Yes, I was using agency to replace “being human”. I think we are moral because we recognize other people as humans like ourselves and use the same brain circuits to model what is happening to them that we use if it were happening to us, thus comes the golden rule morality.
From that I was thinking, is there something that makes us humans special which could actually justify such an approach? And I came up with agency, which I guess is the ability to make a conscious choice of action. So there are three parts to agency. A set of actions, the ability to choose not just based upon the current state of the world but based on preferences of expected consequences, and thirdly consciousness as ability to make meta-level choices. Maybe there needs to be something about learning there as well.
If I think about this notion of agency and try to come up with moral ground rules that are suggested by it, I come up with “life, liberty, and the pursuit of happiness” what is of course not a set of rules, but it’s a more or less direct translation of my three parts of agency into a moral-ish language. How to get from there to a specific set of rules is something I don’t know but think should be possible. And how to choose if different sets of rules would satisfy the purpose is also something I don’t know. It could be that in this case it doesn’t actually matter.
So, you’ve described a human preference for having the things we’d want to happen to us also happen to systems we recognize as sufficiently like ourselves. Call that preference P.
A preference utilitarian would say that the moral value of a choice is proportional to the degree to which P is satisfied by that choice. (All else being equal.)
If I’ve understood you correctly, you reject preference utilitarianism as a moral framework. Instead, you suggest a deontological framework based on “agency.” And agency is a concept you came up with to encapsulate whatever properties humans have that “justify” preferring P.
Have I followed you so far?
OK. Can you say more about how a preference is justified?
For example, you conclude that humans are justified in preferring P on the basis of various attributes of humans (the ability to take action based on expected consequences, the ability to make “meta-level” choices, “a set of actions,” and maybe something about learning). I infer you believe we’re _un_justified in preferring P on the basis of other attributes (say, skin color, or height above sea level, or tendency to slaughter other humans).
Is that right?
How did you arrive at those particular attributes?
I was starting from my own intuitions about my moral preferences. But if you stop at treating morality as a preference you run into problems when people don’t share these preferences. A common variation might for example be that people believe it is good for the strong to prey on the weak. But with morality being an interpersonal thing, any morality must account for differences in preferences and therefore cannot be based in preferences. That’s why I reject preference utilitarianism.
My agency based morality does justify my moral preferences, but it doesn’t “just” justify my moral preferences. I only have my moral preferences as a starting point. From that I construct an abstract moral framework, check if that abstract framework satisfies conditions of consistency and plausibility, and after I’ve been convinced it does, use it to justify or adjust my moral preferences.
Other people might come to different conclusions using this process, but since now our moral framework is removed from mere preferences we can use properties of the frameworks in question to try and integrate them or decide between them. A preference utilitarian would have to resort to some unjustified selection method like majority vote.
So how do I come up with the properties of a moral framework that make it better than an other? I don’t know yet. I would suggest that minimalism is a good property. With a non minimal framework people could always ask, “why should we adopt this policy?”. With a minimal framework it’s either adopt all of it or don’t adopt it at all. I also justify agency as the primary motivation since our agency is what creates the problem in the first place. Without choice we have no use for morality. Without deliberation we couldn’t follow it. Without metalevel reasoning we couldn’t adopt it, etc. Short, agency is the very thing that creates a solvable problem of morality and thus is the best place to solve it. If we start to argue that point, then we are coming to a point where we run into gödelian incompleteness.
You keep tossing the word “justified” around, and I am increasingly unclear on how the work that you want that word to do is getting done.
For example: I agree with you that a preference utilitarian needs some mechanism for resolving situations where preferences conflict, but I’m not sure on what basis you conclude that such a mechanism must be unjustified, nor on what basis you conclude that your agency-based moral frameworks support a more justifiable method for integrating or deciding between different people’s conflicting framework-based-conclusions.
I find your “without X we wouldn’t have a problem and therefore X is the solution” argument unconvincing. Mostly it sounds to me like you’ve decided that your framework is cool, and now you’re looking for arguments to support it.
I was thinking it needs to be separately justified and is not justified from the principle of preference utilitarianism.
It’s a basic principle of engineering to solve a problem where it occurs. I think we’ve reached the point where I am not prepared to argue any further and don’t think it would be fruitful to try. I thank you for the challenge.
That might be the case but I don’t think it likely. I am an asshole enough to do what I want even without moral justification and I am a cynic enough not to expect anything else from other people. I was writing my original comment merely as an additional comment to the morality debate on Less Wrong because I believe that if Eliezer would create his FAI tomorrow it wouldn’t be friendly towards me. The rest was just trying to answer your questions because I really think they helped me to think it through.