There is regular structure in human values that can be learned without requiring detailed knowledge of physics, anatomy, or AI programming. [pollid:1091]
Human values are so fragile that it would require a superintelligence to capture them with anything close to adequate fidelity.[pollid:1092]
Humans are capable of pre-digesting parts of the human values problem domain. [pollid:1093]
Successful techniques for value discovery of non-humans, (e.g. artificial agents, non-human animals, human institutions) would meaningfully translate into tools for learning human values. [pollid:1094]
Value learning isn’t adequately being researched by commercial interests who want to use it to sell you things. [pollid:1095]
Practice teaching non-superintelligent machines to respect human values will improve our ability to specify a Friendly utility function for any potential superintelligence.[pollid:1096]
Something other than AI will cause human extinction sometime in the next 100 years.[pollid:1097]
All other things being equal, an additional researcher working on value learning is more valuable than one working on corrigibility, Vingean reflection, or some other portion of the FAI problem. [pollid:1098]
leplen
The case for value learning
Testing [pollid:1090]
I’m working through the udacity deep learning course right now, and I’m always trying to learn more things on the MIRI research guide. I’m in a fairly different timezone, but my schedule is pretty flexible. Maybe we can work something out?
This raises a really interesting point that I wanted to include in the top level post, but couldn’t find a place for. It seems plausible/likely that human savants are implementing arithmetic using different, and much more efficient algorithms than those used by neurotypical humans. This was actually one of the examples I considered in support of the argument that neurons can’t be the underlying reason humans struggle so much with math.
This is a really broad definition of math. There is regular structure in kinetic tasks like throwing a ball through a hoop. There’s also regular structure in tasks like natural language processing. One way to describe that regular structure is through a mathematical representation of it, but I don’t know that I consider basketball ability to be reliant on mathematical ability. Would you describe all forms of pattern matching as mathematical in nature? Is the fact that you can read and understand this sentence also evidence that you are good at math?
Could a digital intelligence be bad at math?
It’s the average({4-2}/2), rather than the sum, since the altruistic agent is interested in maximizing the average utility.
The tribal limitations on altruism that you allude to are definitely one of the tendencies that much of our cultural advice on altruism targets. In many ways the expanding circle of trust, from individuals, to families, to tribes, to cities, to nation states, etc. has been one of the fundamental enablers of human civilization.
I’m less sure about the hard trade-off that you describe. I have a lot of experience being a member of small groups that have altruism towards non-group members as an explicit goal. In that scenario, helping strangers also helps in-group members achieve their goals. I don’t think large-group altruism precludes you from belonging to small in-groups, since very few in-groups demand any sort of absolute loyalty. While full effort in-group altruism, including things like consciously developing new skills to better assist your other group members would absolutely represent a hard trade-off with altruism on a larger scale, people appear to be very capable of belonging to a large number of different in-groups.
This implies that the actual level of commitment required to be a part of most in-groups is rather low, and the socially normative level of altruism is even lower. Belonging to a close-knit in-group with a particularly needy member, (e.g. having a partially disabled parent, spouse, or child) may shift the calculus somewhat, but for most in-groups being a member in good-standing has relatively undemanding requirements. Examining my own motivations it seems that for many of the groups that I participate in most of the work that I do to fulfilling expectations and helping others within those group is more directly driven by my desire for social validation than my selfless perception of the intrinsic value of the other group members.
Conflicting advice on altruism
Fiction is written from inside the head of the characters. Fiction books are books about making choices, about taking actions and seeing how they play out, and the characters don’t already know the answers when they’re making their decisions. Fiction books often seem to most closely resemble the problems that I face in my life.
Books that have people succeed for the wrong reasons I can put down, but watching people make good choices over and over and over again seems like a really useful thing. Books are a really cheap way to get some of the intuitive advantages of additional life experience. You have to be a little careful to pick authors that don’t teach you the wrong lessons, but in general I haven’t found a lot of histories or biographies that really try to tackle the problem of what it’s like to make choices from the inside in an adequate way. If you’ve read lots of historically accurate works that do manage to give easily digested advice on how to make good decisions, I’d love to see your reading list.
On a very basic level, I am an algorithm receiving a stream of sensory data.
So, do you trust that sensory data? You mention reality, presumably you allow that objective reality which generates the stream of your sensory data exists. If you test your models by sensory data, then that sensory data is your “facts”—something that is your criterion for whether a model is good or not.
I am also not sure how do you deal with surprises. Does sensory data always wins over models? Or sometimes you’d be willing to say that you don’t believe your own eyes?
I don’t understand what you mean by trust. Trust has very little to do with it. I work within the model that the sensory data is meaningful, that life as I experience it is meaningful. It isn’t obvious to me that either of those things are true any more than the parallel postulate is obvious to me. They are axioms.
If my eyes right now are saying something different than my eyes normally tell me, then I will tend to distrust my eyes right now in favor of believing what I remember my eyes telling me. I don’t think that’s the same as saying I don’t believe my eyes.
group selection
When you said “more closely linked to genetic self-interest than to personal self-interest” did you mean the genetic self-interest of the entire species or did you mean something along the lines of Dawkins’ Selfish Gene? I read you as arguing for interests of the population gene pool. If you are talking about selfish genes then I don’t see any difference between “genetic self-interest” and “personal self-interest”.
The idea of the genetic self-interest of an entire species is more or less incoherent. Genetic self-interest involves genes making more copies of themselves. Personal self-interest involves persons making decisions that they think will bring them happiness, utility, what have you. To reiterate my earlier statement “the ability of individual members of that species to plan in such a way as to maximize their own well-being.”
is a series of appeals of to authority
Kinda, but the important thing is that you can go and check. In your worldview, how do you go and check yourself? Or are “streams of sensory data” sufficiently syncronised between everyone?
And I go look for review articles that support the quote that people care about social status. But if you don’t consider expert opinion to be evidence, then you have to go back and reinvent human knowledge from the ground up every time you try and learn anything.
I can always go look for more related data if I have questions about a model. I can read more literature. I can make observations.
Fact just isn’t an epistemological category that I have, and it’s not one that I find useful. There are only models.
So how you choose between different models, then? If there are no facts, what are your criteria? Why is the model of lizard overlords ruling the Earth any worse than any other model?
You use expressions like “because it’s always been true in the past”, but what do you mean by “true”?
My primary criterion is consistency. On a very basic level, I am an algorithm receiving a stream of sensory data. I make models to predict what I think that sensory data will look like in the future based on regularities I detect/have detected in the past. Models that capture consistent features of the data go on to correctly control anticipation and are good models, but they’re all models. The only thing I have in my head is the map. I don’t have access to the territory.
And yet I believe with perfect sincerity that, in generals my maps correspond to reality. I call that correspondence truth. I don’t understand the separation you seem to be attempting to make between facts and models or models and reality.
aspect of the climate system that consistently and frequently chnages between glacial and near-interglacial conditions in periods of less than a decade, and on occassion as rapidly as three years
I am not sure this interpretation of the data surivived—see e.g. this:
Neat. Thanks.
The article you link seems to go out of its way to not be seen as challenging my basic claim, e.g. “Having said this, it should be reemphasised that ice-core chemistry does show extremely rapid changes during climate transitions. The reduction in [Ca] between stadial to interstadial conditions during D-O 3 in the GRIP ice-core occurred in two discrete steps totalling just 5 years [Fuhrer et al., 1999].”
Indeed it the success of the human species that I would cite as evidence for my assertion that human behavior is more closely linked to genetic self-interest than to personal self-interest. Cultural and social success is a huge factor in genetic self-interest.
I haven’t been following the subject closely, but didn’t the idea of group selection ran into significant difficulties? My impression is that nowadays it’s not considered to be a major evolution mechanism, though I haven’t looked carefully and will accept corrections.
I’m not sure how group selection is related to material you’re quoting. Cultural success and social success refer to the success of an individual within a culture/society, not to the success of cultures and societies.
If you don’t consider the opinions of experts evidence, what qualifies?
Opinions are not evidence, they are opinions. Argument to authority is, notably, a fallacy. I call things which qualify “facts”.
I mean, it’s sort of a fallacy. At the same time when I’m sick, I go to a doctor and get her medical opinion and treat it as evidence. I’m not an expert on the things that humans value. I don’t have the time or energy to background to perform experiments and evaluate statistical and experimental methods. Even trusting peer review and relying on published literature is a series of appeals of to authority.
it’s not obvious to me that children are a good investment
I think you’re engaging in nirvana fallacy. Children are not a good investment compared to what?
Again—let’s take a medieval European peasant. He has no ability to accumulate capital because he’s poor, because his lord will just take his money if he notices it, and because once in a while an army passes through and basically grabs everything that isn’t nailed down. He doesn’t have any apprentices because peasants don’t have apprentices (and apprentices leave once they learn the craft, anyway). He certainly has friends, but even his friends will feed their family before him when the next famine comes. So, what kind of investments into a non-starving old age should he make?
He can buy jars of salt and bury them. His children, if they survive, may feed their own children rather than him in the next famine. A network of friends and a high standing in the community are at least as valuable to him as investing resources in birthing and raising children who probably won’t see adulthood. He can become an active and respected member of the church. The church is probably a better bet overall since there’s a decent chance his own kids will die, but the church will probably survive.
I’m not an expert on 14th century investment opportunities, I just find the idea that children are clearly the best selfish investment incredible. If children are such a good investment, why did we need a modest proposal? And why are the rich, who retirements are not in doubt, so desirous of children? Why does king Priam need 50 sons? He’s the king of a city. What fears does he have about retirement?
I don’t know that we have access to facts. Everything is interpreted. Everything is a model.
OK. There were 3,932,181 births in the US in 2013 giving the birth rate of 12.4 / 1000 population (source). Tell me what kind of model is that, which theory does this piece of information critically depends on.
The ones digit of that number is almost certainly wrong and I’m not particularly confident about the next two. Believing that number relies on an enormous number of assumptions about the bureaucracy that generated it. Now my model of the world tells me that the bureaucratic system that calculates the birth rate in the U.S. is fairly trustworthy, compared to say the system that manages elections in Russia, but that trust is totally a function of my model of the world. The data you gather depends on your methodology. Some methods may be better established and may have more evidence in support of them, and the data they gather may really seem reliable, but we also thought that the earth was standing still for a very long time.
Fact just isn’t an epistemological category that I have, and it’s not one that I find useful. There are only models. Some models are more descriptive and better than others, some are more supported by evidence. But there aren’t facts, there are no fixed points that I’m 100% sure are true. I consider my knowledge that 2+2=4 to be close to certain as anything just about anything else I believe, but I hesitate to call it a fact. I have that belief because it’s always been true in the past and my brain has learned that induction is reliable. I could be convinced that 2+2=3, and if you believe something only because you have evidence to support it, then you must have a model that translates between the evidence and the belief.
Because the rate of climate change during the Pleistocene would have made long term forecasting difficult.
Huh? Can you, um, provide some links?
I’m hardly an expert on this, but searching for Pleistocene climate variation gives results like this:
“In addition to the well known millennium-scale stadial and interstadial periods, and the previously recognized century-scale climate events that occur during the Allerod and Bolling periods, we detect a still higher frequency of variability associated with abrupt climate change.”
“The seasonal time resolution of the ECM record portrays as aspect of the climate system that consistently and frequently chnages between glacial and near-interglacial conditions in periods of less than a decade, and on occassion as rapidly as three years.”
Climate Change: Natural climate change: proxy-climate data
The idea that people aren’t, by nature, optimal decision makers is one of the core ideas of LW.
We’re not talking about optimal decisions. We’re talking about not screwing up. Humans are the most successful species on this planet—they are capable of not screwing up sufficiently well.
We are specifically talking about the claim, “Would you seriously argue that people choose to have children as a reasonably optimal selfish way of guaranteeing that they continue to have enough to eat once they’re no longer capable of working?”
I am not making the argument that there are no advantages to having and raising children from a retirement perspective. I am making the argument that it is unlikely that people choose to have children in order to obtain those advantages. I am making the argument that the decline in birthrate in unlikely to be due to people adjusting the number of children they have as part of a retirement plan. The success of a species has very little to do with the ability of individual members of that species to plan in such a way as to maximize their own well-being. Ants are collectively one of the most successful organisms in the world, but they certainly don’t engage in long term planning.
Indeed it the success of the human species that I would cite as evidence for my assertion that human behavior is more closely linked to genetic self-interest than to personal self-interest. Cultural and social success is a huge factor in genetic self-interest. There’s a reason that humans have large brains and devote so many resources to processing social relationships and facial cues. We have equipment for obeying social mandates. We understand them intuitively. We don’t have have intuitive equipment for making long-term predictions, since that was selected for.
status … it seems to be the thing that people care most about after short term economic incentives
Evidence please. People certainly care about status, but I don’t think that people always care about money first, status second, and everything else after that.
I consider the word of a Nobel Prize-winning game theorist and economist to qualify as “evidence” on the topic of aggregate human behavior. If you don’t consider the opinions of experts evidence, what qualifies?
I don’t understand why you think that human allegiances have to be founded on the nuclear family.
They don’t have to be, but I think that empirical evidence points to family ties binding more tight than others.
Okay, but that doesn’t necessarily matter. The ties don’t have to be tight, they just have to be adequate. Also, the parent->child bond is typically tighter than the “child->parent” bond. But even if we add an uncertainty cost to forming non-parent child relationships, it’s not obvious to me that children are a good investment. Children die. Children turn out to be non-productive. Children require lots of resources. Even if my teenage apprentice may be less likely to support me, he’s still way cheaper to build a bond with and way more likely to survive to adulthood. I don’t see any good reason to birth children rather than recruit apprentices.
I’m not sure what you mean by fact.
I mean an observable and testable chunk of empirical reality. Not a theory, not an explanation, not a model.
I don’t know that we have access to facts. Everything is interpreted. Everything is a model. Fact isn’t a separate epistemological category. There are things we agree on, even things most people agree on, but I’m not sure what hard and fast distinction you could draw between facts and theories.
You are claiming that humans have evolved the psychological capacity to make decades long judgments in a reasonably optimal way
That seems pretty obvious to me. What, you think no one ever saves for retirement? Why do you believe that to be false?
Because humans engage in hyperbolic discounting. Because the rate of climate change during the Pleistocene would have made long term forecasting difficult. Because I don’t see evidence of people making medium term judgments in a reasonably optimal way. The idea that people aren’t, by nature, optimal decision makers is one of the core ideas of LW.
Unlike planning for retirement, achieving success within your cultures definition of it (i.e. status) is very important from a genetic evolution status
So how come there are so many losers around? X-) Note that culture is a fairly recent development in “genetic evolution” and for a very long time “high status” implied a front row at the feast, but also a front row at the battle. I agree that high status helped survival, but I don’t think it helped it enough so that evolution gave a major push to the fight-for-leadership genes.
I’m not actually sure that culture is recent. I would put the origins of culture at least tens of thousands of years ago, which is definitely appreciable on an evolutionary scale.
Also, status isn’t necessarily the same thing as leadership, and it seems to be the thing that people care most about after short term economic incentives (e.g. “apart from economic payoffs, social status seems to be the most important incentive and motivating force of social behavior.”-John Harsanyi). The prevalence of the human desire for social status seems pretty well-supported by the literature.
P.S. I’m enjoying this conversation.
That, actually, depends on the circumstances. But in any case, do you really suggest making friends as a good solution to who-will-feed-me problem? Don’t forget that they will get old, too.
Human tribes have been a thing for about as long as there have been humans. People with an important role in the tribe don’t starve to death. And yes, friends age, and so do children. You can make friends that aren’t the same age as you. I don’t understand why you think that human allegiances have to be founded on the nuclear family.
The reality is precisely what is being debated.
Is it? On which facts do we disagree?
I’m not sure what you mean by fact. You made the claim that in reality people have children because they think it’s a good retirement option, and that they choose the number of children that they will choose to have based on how many children they will need in order to make sure they don’t starve to death in the real world. You are claiming that humans have evolved the psychological capacity to make decades long judgments in a reasonably optimal way and that they use that capacity when deciding how many children to have. That is a claim about reality. If it were true, it would be a fact. I think that it is false. I think that people choose whether or not to have children based on culture, and that culture is largely determined by the rules “Copy what most people are doing” and “Copy what successful people are doing”. (That’s not a commentary of the depth or richness of culture. Complex systems often have simple rules.) I also think that successful people in the present and near-past have tended to have less children and I think that the falling birthrate can be attributed to that. I’m fairly sure that the falling birthrate has much more to do with cultural definitions of success than with anyone’s concern for feeding themselves 40 years in the future.
Unlike planning for retirement, achieving success within your cultures definition of it (i.e. status) is very important from a genetic evolution status and would be selected for. I think it’s much more likely that evolution equipped humans to seek cultural success, than it is that evolution equipped humans to sacrifice having children based on concerns for how best to spend their reproductively inactive retirement.
Parents often devote significant resources to caring for special needs children who are unlikely to grow into good providers.
All the more reason to have a large extended family. These children will grow into adults who continue to need extra support, and there’s no reason for parents to support them on their own. The more siblings you have to help out the better.
From a selfish perspective, the correct decision isn’t to have more children. It’s to kill or disown the ones who not only won’t repay your investment, but will actually compete with you for the return on your other investment in your other children.
It’s also not totally obvious to me that children are a particularly good investment from a long-term wealth or even a guaranteed income perspective.
This is because you are thinking of wealth as money. For much of the population of the world, and increasingly so as you go back in time, wealth means enough food on the table, enough food in the root cellar to get you through the winter, and enough grain seed to replant + keep you alive a year or two if the crops fail + plus enough to plant again once the famine is over. As long as another set of hands increases productivity, another pair of hands is a good investment.
I have a pretty broad-minded view of wealth actually. If you’re a New Guinea highlander you can invest in mokas. You can trade your neighbors for goats or land. You can accumulate social capital by being generous and well-liked. You can enter into partnerships with younger partners. Another set of hands is only a good investment if it offers nearly the best return for investment, which is a much higher hurdle than merely “increasing productivity.” It would actually be enormously surprising if the best selfish return you could possibly get for your time and effort was finding a mate and having children, especially given the high infant/child mortality rates. If children were such a good investment then why did we need a modest proposal?
Women in general were low status. Many of their concerns and desires were ignored unless they happened to match concerns and desires that benefitted men. The fact that women didn’t have alternatives to being a mother was just a special case of that..
How did men benefit? Did all men benefit? Were the men also constrained by cultural roles that served to benefit women?
women’s desires were considered irrelevant by society.
This is too strong a statement
Almost any statement interpreted while ignoring connotation is too strong. “Women’s desires were considered irrelevant by society” means “an important set of women’s desires relevant to the current conversation were considered irrelevant by society”, not “all women’s desires were considered irrelevant by society”. Don’t ignore connotation.
Context is probably a better word to use than connotation.
My argument is precisely that women’s desires were considered relevant. I think that society, which, is after all about half women, never has considered the desires of women to be irrelevant nor has it ever considered the desires of men to be irrelevant. Society has definite opinions about what sorts of desires are socially appropriate, but that’s very different from considering desires irrelevant. I think that your objection is about a perceived lack of social roles, especially formal social roles, for unmarried women in some subset of human cultures. Most traditional human societies also lack important social roles for unmarried men.
The transition to an emphasis on personal merit as a source of status rather than familial success has created high status social roles for both men and women outside of the context of family and reproduction. Because men were less tied to reproduction both biologically and culturally, that transition disproportionately affected men at its beginning and for a while Western cultures had many social roles for unmarried men and virtually none for unmarried women. But that was a fairly anomalous period in human history, and for the vast majority of history women have been just about as important to human economic production as men, and as the status of child production has continued to drop, fathers and mothers both have encouraged their daughters to pursue education and careers and other paths desires that lead to positions of high social status.
Being low status has always meant being vulnerable to social violence, and ascribing status is one of the ways that societies create and maintain social norms. The attractiveness of a position in society is dictated by the value and status society ascribes to it, and that valuation is always a set of “external reasons”. Particularly low status groups or members of society, who are perceived as different or in violation of important social norms are often ascribed the status of “criminal” or “enemy” and are left especially vulnerable to social violence.
women’s desires were considered irrelevant by society.
This is too strong a statement. Many women desire to have children. That desire was hardly considered irrelevant by society. Similarly, many women desire to get married or to worship God. These desires weren’t considered irrelevant by society. Quite the opposite. It was considered very important that women have these desires. Desires that led to high social status, like wanting to marry a young man in good standing in the community, were strongly encouraged. Desires that led to low or uncertain social positions like becoming a transient were discouraged, and if pursued in spite of discouragement, punished for undermining the social order. Society almost never respects the desire to become a social deviant.
A society that accords value based on nuclear family size has the social roles of mother, father, and provider and ascribes status to its members based on their success in those roles. In traditional societies, both men and women have jointly fulfilled the latter role. The proliferation of new social roles and highly-esteemed places in the community that have nothing to do with the nuclear family, (i.e. having achievements other than children become markers of status) is the reason that men and women both have a place in the community other than as parents and children.
A man living in a tribe of subsistence foragers can’t ever choose to become a full-time string theorist. He can sometimes choose to become a full-time shaman or priest. Both professions study mainly imaginary things, the difference is that one of these roles is ascribed status by the community while the other is not.
I don’t see humans commonly engaging in a lot of decades off long-term thinking
No particular need. First, it is what happens by default if you don’t take heroic birth control measures (remember, no pill or effective condoms), second, it’s culturally ingrained, that’s what everyone does
I’m a little uncomfortable classifying infanticide as heroic, but that aside I feel like your claim is shifting. At first you claimed that people choose to have children because they are making an optimal selfish long-term retirement decision and that they choose to have children as a good investment in service to that goal. Now you’re saying that people don’t really choose to have children for that reason, but that they have children in response to biological pressures and cultural norms. But the claim that family size is driven primarily by cultural norms, which are largely dictated by the perception of which behaviors are regarded as high status, is literally my original claim.
It’s also not totally obvious to me that children are a particularly good investment from a long-term wealth or even a guaranteed income perspective.
You’re a peasant in Mozambique. Or in XII-century France. What are your other options?
Make friends with people that I didn’t help create? Accumulate wealth? There are lots of durable human social institutions other than the nuclear family. There are certainly more of them in the modern world, but it’s not like all those childless medieval monks starved to death.
But why is that even an option?
You seem to be surprised that what evolutionary psychology says must happen does not happen in reality. I would like to suggest that this a problem for the theory, not for the reality.
The reality is precisely what is being debated. I am making the claim that the choices that populations of people make, esp. with regard to family size, can be understood in terms of evolution and selection, and that they should reflect, in some form or fashion adaptations consistent with genetic self-interest. You are making the claim that people’s choices are more driven by their own rational self-interest, and that understanding the incentives available to individual rational actors is the better predictor of behavior. It seems to me that here you’re just labeling your claim “reality” and saying that if evolution disagrees with it then that’s a problem with evolution.
“There are 729,500 single women my age in New York City. My picture and profile successfully filtered out 729,499 of them and left me with the one I was looking for.”
I know this is sort of meant as a joke, but I feel like one of the more interesting questions that could be addressed in an analysis like this is what percentage of the women in the dating pool could you actually have had a successful relationship with. How strong is your filter and how strong does it need to be? There’s a tension between trying to find/obtain the best of many possible good options, and trying to find the one of a handful of good options in a haystack of bad ones.
I’m somewhat amazed that you looked at 300 profiles, read 60 of them, and liked 20 of them enough to send them messages. Only 1 in 5 potential matches met your standards for appearance, but 1 in 3 met your standards based on what they wrote, and that’s not even taking into account the difference in difficulty between reading a profile and composing a message.
You make a big deal about the number of people available online, but in your previous article on soccer players you implied that the average had a much larger effect on the tails than the average did. If you’re really looking for mates in the tails of the distribution, and 1 in 729,500 is about 4.5 sigma event, then being involved in organizations whose members are much more like your ideal mate on average may be a better strategy than online dating.