In general, I don’t yet see a strong reason to think that our general brain architecture is the sole, or potentially even primary reason why we’ve developed civilization, discontinuous with the rest of the animal kingdom. A strong requirement for civilization is the development of cultural accumulation via language, and more specifically, the ability to accumulate knowledge and technology over generations.
In The Secrets of Our Success, Joe Henrich argues that without our stock of cultural knowledge, individual humans are not particularly more generally intelligent than apes. (Neanderthals may very well have been more generally intelligent than humans—and indeed, their brains are bigger than ours.)
And, he claims, to the extent that individual humans are now especially intelligent, this was because of culture-driven natural selection. For Henrich, the story of human uniqueness is a story of a feedback loop: increased cultural know-how, which drives genetic selection for bigger brains and better social learning, which leads to increased cultural know-how, which drives genetic selection for bigger brains….and so forth, until you have a very weird great ape that is weak, hairless, and has put a flag on the moon.
Note: this evolution + culture feedback loop is still a huge discontinuity that led to massive changes in relatively short evolutionary time!
Just having a generalist brain doesn’t seem like enough; for example, could there have been a dolphin civilization?
Heinrich speculates that a bunch of idiosyncratic features came together to launch us into the feedback loop that led to us being cultural species. Most species, including dolphins, do not get onto this feedback loop because of a “startup” problem: bigger brains will give a fitness advantage only up to a certain point, because individual learning can only be so useful. For there to be further selection for bigger brains, you need a stock of cultural know-how (cooking, hunting, special tools) that makes individual learning very important for fitness. But, to have a stock of cultural know-how, you need big brains.
Heinrich speculates that humans overcame the startup problem due to a variety of factors that came together when we descended from the trees and started living on the ground. The important consequences of a species being on the ground (as opposed to in the trees):
It frees up your hands for tool use. Captive chimps, which are more “grounded” than wild chimps, make more tools.
It’s easier for you to find tools left by other people.
It’s easier for you to see what other people are doing and hang out with them. (“Hang out” being inapt, since that’s precisely not what you’re doing).
You need to group up with people to survive, since there are terrifying predators on the ground. Larger groups offer protection; these larger groups will accelerate the process of people messing around with tools and imitating each other.
Larger groups also produce new forms of social organization. Apparently, in smaller groups of chimps, the reproductive strategy that every male tries to follow is “fight as many males as you can for mating opportunities.” But in a larger group, it becomes better for some males to try to pair bond – to get multiple reproductive opportunities with one female, by hanging around her and taking care of her.
Pair bonding in turn allows for more kinship relationships. Kinship relationships mean you grow up around more people; this accelerates learning. Kinship also allows for more genetic selection for big-brained, slow-developing learners: it becomes less prohibitively costly to give birth to big-brained, slow-growing children, because more people are around to help out and pool food resources.
This story is, by Henrich’s own account, quite speculative. You can find it in Chapter 16 of the book.
In The Secrets of Our Success, Joe Henrich argues that without our stock of cultural knowledge, individual humans are not particularly more generally intelligent than apes.
I 75% agree with this, but I do think that individual humans are smarter than individual chimpanzees. A big area of disagreement is distinguishing between “intrinsic ability to innovate” vs. “ability to process culture”, and whether it’s even possible to distinguish the two. I wrote a post about this two years ago.
For Henrich, the story of human uniqueness is a story of a feedback loop: increased cultural know-how, which drives genetic selection for bigger brains and better social learning, which leads to increased cultural know-how, which drives genetic selection for bigger brains….and so forth, until you have a very weird great ape that is weak, hairless, and has put a flag on the moon.
This is the big crux for me on the evolution of humans and its relevance to the foom debate.
Roughly, I think Henrich’s model is correct. I think his model provides a simple, coherent explanation for why humans dominate the world, and why it happened on such a short timescale, discontinuously with other animals.
Of course, intelligence plays a large role on his model: you can’t get ants who can go to the moon, no matter how powerful their culture. But the the great insight is that our power does not come from our raw intelligence: it comes from our technology/culture, which is so powerful because it was allowed to accumulate.
Cultural accumulation is a zero-to-one discontinuity. That is, you can go a long time without any of it, and then something comes along that’s able to do it just a little bit and then shortly after, it blows up. But after you’ve already reached one, going from “being able to accumulate culture at all” to “being able to accumulate it slightly faster” does not give you the same discontinuous foom as before.
We could, for example, imagine that an AI that can accumulate culture slightly faster than other humans. Since this AI is only slightly better than humans, however, it doesn’t go and create its own culture on its own. Unlike the humans—who actually did go and create their own culture completely on their own, separate from other animals—the AI will simply be one input to the human economy.
This AI would be important input to our economy for sure, but not a completely separate entity producing its own distinct civilization, like the prototypical AI that spins up nanobot factories and kills us all within 3 minutes. It will be more like the brilliant professor, or easily-copyable-worker. In other words, it might speed up our general civilizational abilities to develop technology, and greatly enhance our productive capabilities. But it won’t, on its own, discontinuously produce technology 2.0 (where 1.0 was humans and animals roughly are technology 0.0).
I think a superintelligent AI can FOOM its way to manufacturing nanobots because the biggest bottleneck to engineering and manufacturing those is research that can be done without needing input from the physical universe beyond the physics we already know, and the machines we already have, with very slight upgrades or creative usages beyond what they were designed for. Manufacturing nanobots is like a logic brain teaser for a sufficiently intelligent reasoner. I guess you have a different perspective in that you think the process requires a culture of socializing beings, and/or more input from the physical universe?
In The Secrets of Our Success, Joe Henrich argues that without our stock of cultural knowledge, individual humans are not particularly more generally intelligent than apes. (Neanderthals may very well have been more generally intelligent than humans—and indeed, their brains are bigger than ours.)
And, he claims, to the extent that individual humans are now especially intelligent, this was because of culture-driven natural selection. For Henrich, the story of human uniqueness is a story of a feedback loop: increased cultural know-how, which drives genetic selection for bigger brains and better social learning, which leads to increased cultural know-how, which drives genetic selection for bigger brains….and so forth, until you have a very weird great ape that is weak, hairless, and has put a flag on the moon.
Note: this evolution + culture feedback loop is still a huge discontinuity that led to massive changes in relatively short evolutionary time!
Heinrich speculates that a bunch of idiosyncratic features came together to launch us into the feedback loop that led to us being cultural species. Most species, including dolphins, do not get onto this feedback loop because of a “startup” problem: bigger brains will give a fitness advantage only up to a certain point, because individual learning can only be so useful. For there to be further selection for bigger brains, you need a stock of cultural know-how (cooking, hunting, special tools) that makes individual learning very important for fitness. But, to have a stock of cultural know-how, you need big brains.
Heinrich speculates that humans overcame the startup problem due to a variety of factors that came together when we descended from the trees and started living on the ground. The important consequences of a species being on the ground (as opposed to in the trees):
It frees up your hands for tool use. Captive chimps, which are more “grounded” than wild chimps, make more tools.
It’s easier for you to find tools left by other people.
It’s easier for you to see what other people are doing and hang out with them. (“Hang out” being inapt, since that’s precisely not what you’re doing).
You need to group up with people to survive, since there are terrifying predators on the ground. Larger groups offer protection; these larger groups will accelerate the process of people messing around with tools and imitating each other.
Larger groups also produce new forms of social organization. Apparently, in smaller groups of chimps, the reproductive strategy that every male tries to follow is “fight as many males as you can for mating opportunities.” But in a larger group, it becomes better for some males to try to pair bond – to get multiple reproductive opportunities with one female, by hanging around her and taking care of her.
Pair bonding in turn allows for more kinship relationships. Kinship relationships mean you grow up around more people; this accelerates learning. Kinship also allows for more genetic selection for big-brained, slow-developing learners: it becomes less prohibitively costly to give birth to big-brained, slow-growing children, because more people are around to help out and pool food resources.
This story is, by Henrich’s own account, quite speculative. You can find it in Chapter 16 of the book.
I 75% agree with this, but I do think that individual humans are smarter than individual chimpanzees. A big area of disagreement is distinguishing between “intrinsic ability to innovate” vs. “ability to process culture”, and whether it’s even possible to distinguish the two. I wrote a post about this two years ago.
This is the big crux for me on the evolution of humans and its relevance to the foom debate.
Roughly, I think Henrich’s model is correct. I think his model provides a simple, coherent explanation for why humans dominate the world, and why it happened on such a short timescale, discontinuously with other animals.
Of course, intelligence plays a large role on his model: you can’t get ants who can go to the moon, no matter how powerful their culture. But the the great insight is that our power does not come from our raw intelligence: it comes from our technology/culture, which is so powerful because it was allowed to accumulate.
Cultural accumulation is a zero-to-one discontinuity. That is, you can go a long time without any of it, and then something comes along that’s able to do it just a little bit and then shortly after, it blows up. But after you’ve already reached one, going from “being able to accumulate culture at all” to “being able to accumulate it slightly faster” does not give you the same discontinuous foom as before.
We could, for example, imagine that an AI that can accumulate culture slightly faster than other humans. Since this AI is only slightly better than humans, however, it doesn’t go and create its own culture on its own. Unlike the humans—who actually did go and create their own culture completely on their own, separate from other animals—the AI will simply be one input to the human economy.
This AI would be important input to our economy for sure, but not a completely separate entity producing its own distinct civilization, like the prototypical AI that spins up nanobot factories and kills us all within 3 minutes. It will be more like the brilliant professor, or easily-copyable-worker. In other words, it might speed up our general civilizational abilities to develop technology, and greatly enhance our productive capabilities. But it won’t, on its own, discontinuously produce technology 2.0 (where 1.0 was humans and animals roughly are technology 0.0).
I think a superintelligent AI can FOOM its way to manufacturing nanobots because the biggest bottleneck to engineering and manufacturing those is research that can be done without needing input from the physical universe beyond the physics we already know, and the machines we already have, with very slight upgrades or creative usages beyond what they were designed for. Manufacturing nanobots is like a logic brain teaser for a sufficiently intelligent reasoner. I guess you have a different perspective in that you think the process requires a culture of socializing beings, and/or more input from the physical universe?