″...then there’s the idea that rationalists should be able to (a) solve group coordination problems, (b) care a lot about other people and (c) win...”
Why should rationalists necessarily care a lot about other people? If we are to avoid circular altruism and the nefarious effects of other-optimizing, the best amount of caring might be less than “a lot.”
Additionally, caring about other people in the sense of seeking emotional gratification primarily in tribe-like social rituals may be truly inimical to dedicating one’s life to theoretical physics, math, or any other far-thinking discipline.
Caring about other people may entail involvement in politics, and local politics can be just as mind-killing as national politics.
Sorry to answer a 5 year old post, but apparently people read these things. You asked “Why should rationalists necessarily care a lot about other people,” but all the post said was that they should be able to.
Why should rationalists necessarily care a lot about other people?
They shouldn’t, particularly. End goals are not a part of rationality, rationality exists to achieve them.
However, many end goals can be more easily achieved by getting help from others. If your end goals are like this, it’s rational for you to solve group coordination problems and care about other people.
I don’t think that b is necessarily an immediate entailment of rationality, but a condition that can be met simultaneously with a and c. The post presents a situation where c is satisficed only through a and b. (It does not take much finagling to suppose that a lonesome mountain man existence in a world ruled by barbarians is inferior in fuzziness and utilons relative to the expectation of the world where ab and c are held to be true.)
Good point Bacon. I’ve been wondering where the implicit assumption that rational agents have an altruistic agenda came from. The assumption seems to permeate a rather large number of posts.
When Omega offers to save lives, why do I care? To be perfectly honest, my own utility function suggests that those extra billions are a liability to my interests.
When I realise that my altruistic notions are in conflict with my instinctive drive for status and influence, why do I “need to move in the direction of joining groups more easily, even in the face of annoyances and apparent unresponsiveness”? If anything it seems somewhat more rational to acknowledge the drive for status and self-interest as the key component and satisfy those criteria more effectively.
This isn’t to say I don’t have an altruistic agenda that I pursue. It is just that I don’t see that agenda itself as ‘rational’ at all. It is somewhere between merely arbitrary and ‘slightly irrational’.
With that caveat, this summary and plenty of the posts contained within are damn useful!
“With that caveat, this summary and plenty of the posts contained within are damn useful!”
I resoundingly agree.
That said, Eliezer is attempting to leverage the sentiments we now call “altruistic” into efficient other-optimizing. What if all people are really after is warm fuzzies? Mightn’t they then shrink from the prospect of optimally helping others?
Hobbes gives us several possible reasons for altruism, none of which seem to be conducive to effective helping:
“When the transferring of right is not mutual, but one of the parties transferreth in hope to gain thereby friendship or service from another, or from his friends; or in hope to gain the reputation of charity, or magnanimity; or to deliver his mind from the pain of compassion [self-haters give more?]; or in hope of reward in heaven; this is not contract, but gift, free gift, grace: which words signify one and the same thing.”
There is also the problem of epistemic limitations around other-optimizing. Charity might remove more utilons from the giver than it bestows upon the receiver, if only because it’s difficult to know what other people need and easier to know what oneself needs.
“Mightn’t” we shrink from optimal helping? “Might” charity be usually an imbalance of utilons?
Yes, we might, it might.
These are important considerations—I don’t mean to denigrate clear thinking. But to lie content with hypothetical reasons why something wouldn’t work, due to a common hidden laziness of most humans but which we can convince ourselves is due to more noble and reasonable reasons, is to completely miss the most crucial point of this entire Sequence: actually doing something, testing.
I think it’s safe to say that the natural inclination of most humans isn’t initiating large projects with high but uncertain reward. It’s to “just get by”, a fact which I must thank you, good sir, for illustrating.… it was intentional, right?
″...then there’s the idea that rationalists should be able to (a) solve group coordination problems, (b) care a lot about other people and (c) win...”
Why should rationalists necessarily care a lot about other people? If we are to avoid circular altruism and the nefarious effects of other-optimizing, the best amount of caring might be less than “a lot.”
Additionally, caring about other people in the sense of seeking emotional gratification primarily in tribe-like social rituals may be truly inimical to dedicating one’s life to theoretical physics, math, or any other far-thinking discipline.
Caring about other people may entail involvement in politics, and local politics can be just as mind-killing as national politics.
Sorry to answer a 5 year old post, but apparently people read these things. You asked “Why should rationalists necessarily care a lot about other people,” but all the post said was that they should be able to.
They shouldn’t, particularly. End goals are not a part of rationality, rationality exists to achieve them.
However, many end goals can be more easily achieved by getting help from others. If your end goals are like this, it’s rational for you to solve group coordination problems and care about other people.
I don’t think that b is necessarily an immediate entailment of rationality, but a condition that can be met simultaneously with a and c. The post presents a situation where c is satisficed only through a and b. (It does not take much finagling to suppose that a lonesome mountain man existence in a world ruled by barbarians is inferior in fuzziness and utilons relative to the expectation of the world where a b and c are held to be true.)
Good point Bacon. I’ve been wondering where the implicit assumption that rational agents have an altruistic agenda came from. The assumption seems to permeate a rather large number of posts.
When Omega offers to save lives, why do I care? To be perfectly honest, my own utility function suggests that those extra billions are a liability to my interests.
When I realise that my altruistic notions are in conflict with my instinctive drive for status and influence, why do I “need to move in the direction of joining groups more easily, even in the face of annoyances and apparent unresponsiveness”? If anything it seems somewhat more rational to acknowledge the drive for status and self-interest as the key component and satisfy those criteria more effectively.
This isn’t to say I don’t have an altruistic agenda that I pursue. It is just that I don’t see that agenda itself as ‘rational’ at all. It is somewhere between merely arbitrary and ‘slightly irrational’.
With that caveat, this summary and plenty of the posts contained within are damn useful!
“With that caveat, this summary and plenty of the posts contained within are damn useful!”
I resoundingly agree.
That said, Eliezer is attempting to leverage the sentiments we now call “altruistic” into efficient other-optimizing. What if all people are really after is warm fuzzies? Mightn’t they then shrink from the prospect of optimally helping others?
Hobbes gives us several possible reasons for altruism, none of which seem to be conducive to effective helping:
“When the transferring of right is not mutual, but one of the parties transferreth in hope to gain thereby friendship or service from another, or from his friends; or in hope to gain the reputation of charity, or magnanimity; or to deliver his mind from the pain of compassion [self-haters give more?]; or in hope of reward in heaven; this is not contract, but gift, free gift, grace: which words signify one and the same thing.”
There is also the problem of epistemic limitations around other-optimizing. Charity might remove more utilons from the giver than it bestows upon the receiver, if only because it’s difficult to know what other people need and easier to know what oneself needs.
“Mightn’t” we shrink from optimal helping? “Might” charity be usually an imbalance of utilons?
Yes, we might, it might.
These are important considerations—I don’t mean to denigrate clear thinking. But to lie content with hypothetical reasons why something wouldn’t work, due to a common hidden laziness of most humans but which we can convince ourselves is due to more noble and reasonable reasons, is to completely miss the most crucial point of this entire Sequence: actually doing something, testing.
I think it’s safe to say that the natural inclination of most humans isn’t initiating large projects with high but uncertain reward. It’s to “just get by”, a fact which I must thank you, good sir, for illustrating.… it was intentional, right?