Nature can only do so much to improve our intelligence, being stuck with living cells as computational circuits in a finite brain volume, with finite energy supply.
This is true, and it’s one of the main reasons I think AI will eventually overtake us no matter how much genetic engineering we do. But like I said in the post, there is enough additive variance in existing gene pool to create humans with predicted IQs of over a thousand. We are far from the actual physical limits on brain size, neural conduction speed, neuron size, and many others.
It isn’t clear how meaningful the intelligence differences really are in terms of utility on actual tasks.
How many company founders emphasize that “attracting talent” is the most difficult part of making their company successful? How many Nobel winners have IQs several standard deviations above the mean? It is very clear that intelligence has a huge impact on performance on a wide variety of tasks. If you want more examples of this I suggest you read one of Gwern’s many essays on the topic
Thing is, the road to get there isn’t going to involve a whole lot of someone solving math problems in their head as they pound a keyboard through the night writing reams of custom code.
This isn’t an accurate characterization of the type of task that unusually intelligent people excel at. I agree with you that raw intelligence isn’t the only thing need to solve this problem. Creating proper organizational incentive structures will matter a lot too, as will clear-headed thinking about the problem. But intelligent people are actually very good at exactly those types of things. Look at how many of the top scientists in the country played a significant role in the Manhattan Project.
A single person is not going to meaningfully solve this problem by themselves.
Which is why I didn’t suggest we create one super-genius and call it a day. I don’t want access to pre-implantation genetic screening or iterated embryo selection to be available to only the privileged elite. It needs to be broadly available to any parent that wants it. And the benefits can and will go to many fields, not just AI.
Just as a general policy, anything current flesh and blood humans with are having trouble with, that smarter humans have less trouble with, current humans can probably write a piece of software that is better than the efforts of any humans. With today’s techniques.
This is just obviously wrong. TAI cannot be created by current. humans with today’s techniques. It’s likely going to take decades to create the technology to do so, and it’s going to take some of the smartest researchers in the world to do it.
My support for the last paragraph is that many of the things we credit “exceptionally smart” people with doing like solving equations can be automated. Or exploring function spaces for a better solution. Or, well, any problem that has a checkable answer, which are the very things iq tests measure.
It’s not on an IQ test how to imagine a better aircraft that is both creative and meets design specs. It’s always problems that a clear answer exists for.
Anyways in my personal experience I have met a lot of “brittle” people. They have no outer visualization for how a machine actually works and just get stuck the moment they hit a problem that wasn’t in a training exercise at school. Basic ideas just don’t occur to them.
But yeah if you put me up against them on rigidly defined problems taught in a book I might be slightly slower.
Note that I personally test at around 80-97th percentile depending on the test. (MCATs was 97). This tells me that whatever intelligence I have lucked into having is substantially above average but not the best.
I am saying an army of people only as good as me—top quintile—can and will create TAI decades before genetic engineering will matter.
I am saying an army of people only as good as me—top quintile—can and will create TAI decades before genetic engineering will matter.
Yes, this is a concern for the utility of this approach. If TAI is created before 2050, none of this work will matter much because none of the unusually intelligent people we’ve been able to create will have had time to make meaningful contributions to the field of AI. In that sense, research in this field is a gamble that only starts paying off if AI takes until at least 2050. Genetic engineering will have a progressively larger impact the longer it takes to develop TAI.
This timing concern was actually one of my chief worries about going into genetics as a career. I won’t be able to switch careers and start having a large impact on AI if research in that field progresses faster than expected. So it’s possible there will come a point in the future where I am stuck on the sidelines in the final years before TAI is created, watching 30 years of work come to nothing.
But I think 50% odds of having a huge impact are worth taking, and I think the biological route to superintelligence is severely neglected right now. Who is actually working on genetic engineering right now? I literally know one person who has both expressed an interest in genetic engineering for intelligence and has real scientific expertise in the field: Steven Hsu. And sadly he seems to have turned away from his earlier goals after his public humiliation at the hands of misguided student activists at Michigan State University.
I am hopeful that as pre-implantation genetic screening via IVF becomes a more normalized part of the pregnancy process, attitudes will change. It’s pretty silly that so many people think enhancing our children’s potential via physical exercise and healthy food is acceptable but that genetic intervention should be off-limits.
Oh. The reason you shouldn’t go into genetics as a career is you will not be permitted to do anything on humans until after we have TAI. Your career will just be wasted. You should work on AI unless you are already in a PhD program.
There are countless legal and structural barriers in the way.
Can’t do it without enough power to overthrow a western government. Only thing that could even theoretically do that would be a TAI fighting on your side...
There are at least two companies in the US alone already doing pre-implantation screening for polygenic disease risk right now, and one of them is offering screening for unusually low IQ already. It’s not that big of a stretch to imagine that parents will want to actively screen for IQ or other important traits in the next decade.
There are no legal barriers to embryo selection for intelligence. There may be some put up at some point in the future (which is a source of worry for me), but the current barriers are technological, not legal.
There was a survey done in Singapore and 87% of parents said they would be willing to intervene genetically to make their children smarter if the option was available. Attitudes in Korea are similar. If worse comes to worse I’ll just work for a company or in a lab somewhere that hasn’t banned it.
This is true, and it’s one of the main reasons I think AI will eventually overtake us no matter how much genetic engineering we do. But like I said in the post, there is enough additive variance in existing gene pool to create humans with predicted IQs of over a thousand. We are far from the actual physical limits on brain size, neural conduction speed, neuron size, and many others.
How many company founders emphasize that “attracting talent” is the most difficult part of making their company successful? How many Nobel winners have IQs several standard deviations above the mean? It is very clear that intelligence has a huge impact on performance on a wide variety of tasks. If you want more examples of this I suggest you read one of Gwern’s many essays on the topic
This isn’t an accurate characterization of the type of task that unusually intelligent people excel at. I agree with you that raw intelligence isn’t the only thing need to solve this problem. Creating proper organizational incentive structures will matter a lot too, as will clear-headed thinking about the problem. But intelligent people are actually very good at exactly those types of things. Look at how many of the top scientists in the country played a significant role in the Manhattan Project.
Which is why I didn’t suggest we create one super-genius and call it a day. I don’t want access to pre-implantation genetic screening or iterated embryo selection to be available to only the privileged elite. It needs to be broadly available to any parent that wants it. And the benefits can and will go to many fields, not just AI.
This is just obviously wrong. TAI cannot be created by current. humans with today’s techniques. It’s likely going to take decades to create the technology to do so, and it’s going to take some of the smartest researchers in the world to do it.
My support for the last paragraph is that many of the things we credit “exceptionally smart” people with doing like solving equations can be automated. Or exploring function spaces for a better solution. Or, well, any problem that has a checkable answer, which are the very things iq tests measure.
It’s not on an IQ test how to imagine a better aircraft that is both creative and meets design specs. It’s always problems that a clear answer exists for.
Anyways in my personal experience I have met a lot of “brittle” people. They have no outer visualization for how a machine actually works and just get stuck the moment they hit a problem that wasn’t in a training exercise at school. Basic ideas just don’t occur to them.
But yeah if you put me up against them on rigidly defined problems taught in a book I might be slightly slower.
Note that I personally test at around 80-97th percentile depending on the test. (MCATs was 97). This tells me that whatever intelligence I have lucked into having is substantially above average but not the best.
I am saying an army of people only as good as me—top quintile—can and will create TAI decades before genetic engineering will matter.
Yes, this is a concern for the utility of this approach. If TAI is created before 2050, none of this work will matter much because none of the unusually intelligent people we’ve been able to create will have had time to make meaningful contributions to the field of AI. In that sense, research in this field is a gamble that only starts paying off if AI takes until at least 2050. Genetic engineering will have a progressively larger impact the longer it takes to develop TAI.
This timing concern was actually one of my chief worries about going into genetics as a career. I won’t be able to switch careers and start having a large impact on AI if research in that field progresses faster than expected. So it’s possible there will come a point in the future where I am stuck on the sidelines in the final years before TAI is created, watching 30 years of work come to nothing.
But I think 50% odds of having a huge impact are worth taking, and I think the biological route to superintelligence is severely neglected right now. Who is actually working on genetic engineering right now? I literally know one person who has both expressed an interest in genetic engineering for intelligence and has real scientific expertise in the field: Steven Hsu. And sadly he seems to have turned away from his earlier goals after his public humiliation at the hands of misguided student activists at Michigan State University.
I am hopeful that as pre-implantation genetic screening via IVF becomes a more normalized part of the pregnancy process, attitudes will change. It’s pretty silly that so many people think enhancing our children’s potential via physical exercise and healthy food is acceptable but that genetic intervention should be off-limits.
Oh. The reason you shouldn’t go into genetics as a career is you will not be permitted to do anything on humans until after we have TAI. Your career will just be wasted. You should work on AI unless you are already in a PhD program.
There are countless legal and structural barriers in the way.
The effective altruist case for regime change??
Can’t do it without enough power to overthrow a western government. Only thing that could even theoretically do that would be a TAI fighting on your side...
There are at least two companies in the US alone already doing pre-implantation screening for polygenic disease risk right now, and one of them is offering screening for unusually low IQ already. It’s not that big of a stretch to imagine that parents will want to actively screen for IQ or other important traits in the next decade.
There are no legal barriers to embryo selection for intelligence. There may be some put up at some point in the future (which is a source of worry for me), but the current barriers are technological, not legal.
There was a survey done in Singapore and 87% of parents said they would be willing to intervene genetically to make their children smarter if the option was available. Attitudes in Korea are similar. If worse comes to worse I’ll just work for a company or in a lab somewhere that hasn’t banned it.
Embryo selection is a weak form of genetic engineering though, literally just restricting certain rolls from a die.
This is not how you get someone with a 1000 iq, its how you make 130 iq more common.