But it ultimately doesn’t matter, because the brain just learns too slowly.
Why think the brain learns too slowly? If I can boost my sample efficiency I can learn new subjects quicker, remember more facts, and do better thought-result attribution. All these seem short-term beneficial. Unless you think there’s major critical period barriers here, these all seem likely results.
Though I do agree that a person with the genes of a genius for 2 years will be far less a genius than a person with the genes of a genius for 25 years. It seems a stretch to say the first change can be rounded off as not matteringthough.
It would matter in a world without AI, but that is not the world we live in. Yes if you condition on some indefinite AI pause or something then perhaps, but that seems extremely unlikely. It takes about 30 years to train a new brain—so the next generation of humans won’t reach their prime until around the singularity, long after AGI.
Though I do agree that a person with the genes of a genius for 2 years
Most genius is determined prenatally and during ‘training’ when cortex/cerebellum modules irreversibly mature, just as the capabilities of GPT4 are determined by the initial code and the training run.
I think I agree with everything that you said except that it won’t matter. It seems like it’d very much matter if in the most successful case we make people learn skills, facts, and new fields 6x faster. Maybe you think 6x is too high, and its more like 1.05x, enough to notice a difference over 30 years, but not over 2-5.
So other than the medical issues which makes this idea unviable (off target edits causing cancer, major brain firmware edits causing uncontrollable seizures), we can also bound our possible performance increases.
We haven’t increased brain volume meaningfully, the patients skull plates are fixed. And they still have one set of eyes and one set of hands. Nerve transmission velocities haven’t been improved either.
In terms of real performance is 6x achievable? Does it mean anything? Even the most difficult tasks humans have ever accomplished require taking in the latest data from the last round of testing and then choosing what to try next based on this information. This “innovation feedback cycle” is several steps, and the “think about it” step, even if we posit we can make it infinitely fast, would be limited by “view the data” and “communicate with other people” steps.
That is I am taking a toy model:
View the data, think about the next experiment, tell someone/something else to do the experiment
Are the only 3 steps.
If the view/tell steps take more than 1⁄6 of the total time 6x performance increase is impossible. This is Amdahls law.
Viewing and telling are themselves learned skills that can be sped up. Most are far from their maximal physically possible reading/listening speed, or their maximal physically possible writing speed. For example, just after high school I tried to read sutton & barto and spent a week reading each chapter. Later I read it & spent a day on each chapter. That’s a 7x improvement just from meta-learning!
You’re still i/o limited though. And optimizations you mention come with tradeoffs, skipping words by speed reading for example won’t work at all when the content has high information density
I still read every word, I just knew better what to think about, recall was faster, etc. I was reading at a leisurely pace as well. If you want to call learning what to pay attention to & how to pay attention to it not an i/o problem, just the physical limits, then I do think i/o is very very fast, taking <<1/6 of time.
Depends on what it is. Experimenting with AI? Fixing cars? Working on particle physics as CERN? Developing the atomic bomb? Discovering the mass of the electron? Performing surgery? Developing new surgery methods?
I/O is at least 90 percent of each of those tasks.
I don’t know how to think about i/o in the tasks you mention, so I don’t think the question is very useful. Definitely on an individual level, much time is spent on i/o, but that’s beside the point, as I said above people can do more efficient i/o than they currently do, and generally smarter people are able to do more efficient i/o. When I ask myself why we aren’t better at the tasks you mention, mostly I think firstly we are coordination constrained, and secondly we are intelligence constrained. Maybe we’re also interface constrained, which seems related to i/o, but generally we make advancements in interfaces, and this improves productivity in all the tasks you mention, and smarter people can use & make smarter interfaces if that is indeed the problem.
A good motivator: There exist 10x software engineers, who are generally regarded as being 10x better programmers than regular engineers. If i/o was the limiter for programming ability, then such people would be expected to simply have better keyboards, finger dexterity, and eyesight. Possibly they have these to some extent, but I expect their main edge over other 1x engineers is greater sample efficiency when generalizing from one programming task to another. We can thus conclude that i/o takes up <1/10 the time in programming. Probably <<1/10.
There also probably exist 10x surgeons, experimentalists, and mechanics. Perhaps there also exist 10x particle physicists at CERN, though there are fewer of them, and it may be less obvious.
So if 10x software engineers exist, they develop architecture and interfaces and use patterns where over time 1⁄10 the total amount of human time per feature is used. Bad code consumes enormous amounts of time to deal with, where bad architecture that blocks adding new features or makes localizing a bug difficult would be the worst.
But to be this good mostly comes from knowledge, learned either in school or over a lot of time from doing it the wrong way and learning how it fails.
It’s not an intelligence thing. A genius swe can locate a bug on a hunch, a 10x swe would write code where the bug doesn’t exist or is obvious every time.
A lot of the other examples I gave I have the impression that no, I/o is everything. Finding the mass of the electron was done with laborious effort over many hours, most of it dealing with the equipment. Nobody can cut 10 times faster in surgery, hands can’t run that quickly. Same with fixing a car. Cern scientists obviously are limited by all sorts of equipment issues. Same with AI research—the limiting factor has always been equipment from the start. “Equipment issues” mean either you get your hands dirty fixing it yourself—that’s I/O or spare parts bound—or you tell someone else to do it and their time to fix is bound the same way.
Some of the best scientists in history could fix equipment issues themselves, this likely broadened their skill base and made their later discoveries feasible.
They aren’t the same thing? I mean for the topics of interest, AI alignment, there is nothing to learn from other humans or improve on past a certain baseline level of knowledge. Past a certain point reading papers on it I suspect your learning curve would go negative because you’re just learning on errors people before you made.
Improving past that point has to be designing and executing high knowledge gain experiments, and that’s I/o and funding bound.
I would argue that the above is the rule for anything humans cannot already do.
Were you thinking of skills where it’s a confined objective task? Like StarCraft 2 or Go? The former being strongly I/o bound.
I’m very confident we’re talking past each other, and I’m not in the mood to figure out what we actually disagree on. I think we’re using “i/o” differently, and I claim your use permits improvements to the process, which contradicts your argument.
Why think the brain learns too slowly? If I can boost my sample efficiency I can learn new subjects quicker, remember more facts, and do better thought-result attribution. All these seem short-term beneficial. Unless you think there’s major critical period barriers here, these all seem likely results.
Though I do agree that a person with the genes of a genius for 2 years will be far less a genius than a person with the genes of a genius for 25 years. It seems a stretch to say the first change can be rounded off as not matteringthough.
It would matter in a world without AI, but that is not the world we live in. Yes if you condition on some indefinite AI pause or something then perhaps, but that seems extremely unlikely. It takes about 30 years to train a new brain—so the next generation of humans won’t reach their prime until around the singularity, long after AGI.
Most genius is determined prenatally and during ‘training’ when cortex/cerebellum modules irreversibly mature, just as the capabilities of GPT4 are determined by the initial code and the training run.
I think I agree with everything that you said except that it won’t matter. It seems like it’d very much matter if in the most successful case we make people learn skills, facts, and new fields 6x faster. Maybe you think 6x is too high, and its more like 1.05x, enough to notice a difference over 30 years, but not over 2-5.
So other than the medical issues which makes this idea unviable (off target edits causing cancer, major brain firmware edits causing uncontrollable seizures), we can also bound our possible performance increases.
We haven’t increased brain volume meaningfully, the patients skull plates are fixed. And they still have one set of eyes and one set of hands. Nerve transmission velocities haven’t been improved either.
In terms of real performance is 6x achievable? Does it mean anything? Even the most difficult tasks humans have ever accomplished require taking in the latest data from the last round of testing and then choosing what to try next based on this information. This “innovation feedback cycle” is several steps, and the “think about it” step, even if we posit we can make it infinitely fast, would be limited by “view the data” and “communicate with other people” steps.
That is I am taking a toy model:
View the data, think about the next experiment, tell someone/something else to do the experiment
Are the only 3 steps.
If the view/tell steps take more than 1⁄6 of the total time 6x performance increase is impossible. This is Amdahls law.
Viewing and telling are themselves learned skills that can be sped up. Most are far from their maximal physically possible reading/listening speed, or their maximal physically possible writing speed. For example, just after high school I tried to read sutton & barto and spent a week reading each chapter. Later I read it & spent a day on each chapter. That’s a 7x improvement just from meta-learning!
You’re still i/o limited though. And optimizations you mention come with tradeoffs, skipping words by speed reading for example won’t work at all when the content has high information density
I still read every word, I just knew better what to think about, recall was faster, etc. I was reading at a leisurely pace as well. If you want to call learning what to pay attention to & how to pay attention to it not an i/o problem, just the physical limits, then I do think i/o is very very fast, taking <<1/6 of time.
Depends on what it is. Experimenting with AI? Fixing cars? Working on particle physics as CERN? Developing the atomic bomb? Discovering the mass of the electron? Performing surgery? Developing new surgery methods?
I/O is at least 90 percent of each of those tasks.
I don’t know how to think about i/o in the tasks you mention, so I don’t think the question is very useful. Definitely on an individual level, much time is spent on i/o, but that’s beside the point, as I said above people can do more efficient i/o than they currently do, and generally smarter people are able to do more efficient i/o. When I ask myself why we aren’t better at the tasks you mention, mostly I think firstly we are coordination constrained, and secondly we are intelligence constrained. Maybe we’re also interface constrained, which seems related to i/o, but generally we make advancements in interfaces, and this improves productivity in all the tasks you mention, and smarter people can use & make smarter interfaces if that is indeed the problem.
A good motivator: There exist 10x software engineers, who are generally regarded as being 10x better programmers than regular engineers. If i/o was the limiter for programming ability, then such people would be expected to simply have better keyboards, finger dexterity, and eyesight. Possibly they have these to some extent, but I expect their main edge over other 1x engineers is greater sample efficiency when generalizing from one programming task to another. We can thus conclude that i/o takes up <1/10 the time in programming. Probably <<1/10.
There also probably exist 10x surgeons, experimentalists, and mechanics. Perhaps there also exist 10x particle physicists at CERN, though there are fewer of them, and it may be less obvious.
So if 10x software engineers exist, they develop architecture and interfaces and use patterns where over time 1⁄10 the total amount of human time per feature is used. Bad code consumes enormous amounts of time to deal with, where bad architecture that blocks adding new features or makes localizing a bug difficult would be the worst.
But to be this good mostly comes from knowledge, learned either in school or over a lot of time from doing it the wrong way and learning how it fails.
It’s not an intelligence thing. A genius swe can locate a bug on a hunch, a 10x swe would write code where the bug doesn’t exist or is obvious every time.
A lot of the other examples I gave I have the impression that no, I/o is everything. Finding the mass of the electron was done with laborious effort over many hours, most of it dealing with the equipment. Nobody can cut 10 times faster in surgery, hands can’t run that quickly. Same with fixing a car. Cern scientists obviously are limited by all sorts of equipment issues. Same with AI research—the limiting factor has always been equipment from the start. “Equipment issues” mean either you get your hands dirty fixing it yourself—that’s I/O or spare parts bound—or you tell someone else to do it and their time to fix is bound the same way.
Some of the best scientists in history could fix equipment issues themselves, this likely broadened their skill base and made their later discoveries feasible.
You are operating on the wrong level of analysis here. The question is about skill improvement, not execution.
They aren’t the same thing? I mean for the topics of interest, AI alignment, there is nothing to learn from other humans or improve on past a certain baseline level of knowledge. Past a certain point reading papers on it I suspect your learning curve would go negative because you’re just learning on errors people before you made.
Improving past that point has to be designing and executing high knowledge gain experiments, and that’s I/o and funding bound.
I would argue that the above is the rule for anything humans cannot already do.
Were you thinking of skills where it’s a confined objective task? Like StarCraft 2 or Go? The former being strongly I/o bound.
I’m very confident we’re talking past each other, and I’m not in the mood to figure out what we actually disagree on. I think we’re using “i/o” differently, and I claim your use permits improvements to the process, which contradicts your argument.