I’ve done a lot of this kind of data cleaning, proof of concept, then scaling to deployed reliable product in my career. I agree it’s as you describe.
The thing about using this as a way to predict that the economic/technological effects of a very general AI that’s competent enough to do R&D is.… I don’t think that change has to route through existing companies.
If the AI is sufficiently capable, and can exploit existing openly available research papers to learn about the world.…
If this AI can use existing publicly available datasets to build specialized prediction tools for itself, along the lines of AlphaFold...
If the AI can skillfully control generalized robotic actuators (e.g. robotic arms, humanoid robotic bodies), and interpret visual data intelligently, and make skillful use of tools built for humans....
Such an AI can leapfrog over a ton of existing tech. At small scale anyway. So then you must look at special cases where a small scale prototype of a new technology could be transformative for the world.
There’s not a lot of contexts I can imagine this being the case for. Most of them come down to situations where the small scale prototype is in some way self-replicating, so that it can scale. The others involve using a high-leverage small scale prototype to influence decision-making processes of society at large (e.g. super-persuasion or brain-control chips inserted into key political and corporate leaders).
Self-replicating technology is totally a viable and dangerous thing though. The most obvious case within the reach of current tech is bioweapons. Self-replicating nanotech remains a possibility on the table, theoretically allowed by physics but not yet demonstrated. What about industrial uses of self-replicating technology? Strange synthetic biology?
These things all seem quite unlikely in our current technological regime, but imagine a million AIs working together who are much smarter, faster, unsleeping, super-knowledgeable, super well-coordinated, obsessively work-focused, fast learners, good at crafting specialized tools for handling specific datasets or physical world problems. Things might change surprisingly quickly!
Here’s some categories I think could be pretty world changing, even without buy-in from existing industry:
Cheap production of novel drugs with powerful and specific effects
Engineered organisms (animals, plants, or fungi) or collectives thereof which can make use of hydrocarbons in plastic and can successfully reproduce rapidly using just the material in a landfill plus sunlight and water, while also doing some kind of economically useful activity.
Same, but you feed them on industrial agricultural wastes (easier)
Same, but you can directly feed them fossil fuels (harder)
crude oil consumption, plus ability to live deep under the ocean, plus ability to export products in such a way that they were easy to harvest from AI-controlled submarines or ships above would enable a huge rapid growth of an industrial base.
Organism which can thrive in hard-vacuum, harness solar power and heat gradients, feed on ice, carbon, oxygen, etc. could be ‘planted’ by a small mission to a large asteroid or comet. It would be hard for this to be directly economically productive back on Earth, but could make for a big headstart for the beginnings of a race to colonize the local solar system.
Organic or cyborg computing systems. Scientists have already discussed the possibility of wiring up a bunch of rodent brains together through a computer network using brain-computer-interfaces. But what if you could skip using fragile inconvenient animal brains? What if you could get fungal mycelium to work like neurons for computation? You can grow a whole lot of engineered fungal mycelium really fast, and feed it quite easily. Even if the result was much lower average utility per cubic millimeter than a rodent brain, the ability to manufacture and use it on an industrial scale would likely make up for that.
Plants capable of producing plastic. This could do a lot, actually. Imagine if there were plants which could grow on the interface between land and sea, and produce little plastic bubbles for themselves which functioned as solar stills. They could refine fresh water from salt water using sunlight. What about plants forming highly robust and lengthy lateral stems which could pump this water inland and receive sugars back? Suddenly you have plants which can colonize deserts which border oceans. Plants are limited in how far they can move water vertically because of various difficulties including the vaporization pressure of water and limitations of capillary action. Horizontally though, plants can form enormous root-stem-root-stem complexes and the resulting colonies can cover many square miles.
I could go on about possibilities for hours. I’m not a super-intelligent corporation of a million R&D AIs, so of course my ideas are far more limited in variety and plausibility than what they could come up with. The point is, once things get that far, we can’t predict them using the trends and behavior patterns of traditional human corporations.
Yes, I agree that that’s what the post was talking about. I do think my comment is still relevant since the transition time from pre-superintelligence human-level-AGI economy to superintelligent-AGI-economy may be just a few months. Indeed, that is exactly what I expect due to the high likelihood I place on the rapid effects of recursive self-improvement enabled by human-level-AGI.
I would expect that the company developing the human-level-AGI may even observe the beginnings of a successful RSI process and choose to pursue that in secret without ever releasing the human-level-AGI. Afterall, if it’s useful for rapid potent RSI then it would give their competitors a chance to catch up or overtake them if their competitors also had access to the human-level-AGI.
Thus, from the point of view of outside observers, it may seem that we jump straight from no-AGI to a world affected by technology developed by superintelligent-AGI, without ever seeing either the human-level-AGI or the superintelligent-AGI deployed.
[Edit: FWIW I think that Tom Davidson’s report errs slightly in the other direction, of forecasting that things in the physical world might move somewhat faster than I expect. Maybe 1.2x to 5x faster than I expect. So that puts me somewhere in-between world-as-normal and world-goes-crazy in terms of physical infrastructure and industry. On the other hand, I think Tom Davidson’s report somewhat underestimates the rate at which algorithm / intelligence improvement could occur. Algorithmic improvements which ‘piggyback’ on existing models, and thus start warm and improve cheaply from there, could have quite fast effects with no wait time for additional compute to become available or need to retrain from scratch. If that sort of thing snowballed, which I think it might, the result could get quite capable quite fast. And that would all be happening in software changes, and behind closed doors, so the public wouldn’t necessarily know anything about it.]
I agree that if the AI can run its own experiments (via robotic actuators) it can do R&D prototyping independently of existing private/corporate data, and that’s potentially the whole game.
My current impression is that, as of 2024, we’re starting to see enough investment into AI-controlled robots that in a few years it would be possible to get an “AI experimenter”, albeit in the restricted set of domains where experiments can be automated easily. (biological experiments that are basically restricted to pipetting aqueous solutions and imaging the results? definitely yes. most sorts of benchtop electronics prototyping and testing? i imagine so, though I don’t know for sure. the full range of reactions/syntheses a chemist can run at a lab bench? probably not for some time; creating a “mechanical chemist” is a famously hard problem since methods are so varied, though obviously it’s not in principle impossible.)
I’ve done a lot of this kind of data cleaning, proof of concept, then scaling to deployed reliable product in my career. I agree it’s as you describe.
The thing about using this as a way to predict that the economic/technological effects of a very general AI that’s competent enough to do R&D is.… I don’t think that change has to route through existing companies.
If the AI is sufficiently capable, and can exploit existing openly available research papers to learn about the world.…
If this AI can use existing publicly available datasets to build specialized prediction tools for itself, along the lines of AlphaFold...
If the AI can skillfully control generalized robotic actuators (e.g. robotic arms, humanoid robotic bodies), and interpret visual data intelligently, and make skillful use of tools built for humans....
Such an AI can leapfrog over a ton of existing tech. At small scale anyway. So then you must look at special cases where a small scale prototype of a new technology could be transformative for the world.
There’s not a lot of contexts I can imagine this being the case for. Most of them come down to situations where the small scale prototype is in some way self-replicating, so that it can scale. The others involve using a high-leverage small scale prototype to influence decision-making processes of society at large (e.g. super-persuasion or brain-control chips inserted into key political and corporate leaders).
Self-replicating technology is totally a viable and dangerous thing though. The most obvious case within the reach of current tech is bioweapons. Self-replicating nanotech remains a possibility on the table, theoretically allowed by physics but not yet demonstrated. What about industrial uses of self-replicating technology? Strange synthetic biology?
These things all seem quite unlikely in our current technological regime, but imagine a million AIs working together who are much smarter, faster, unsleeping, super-knowledgeable, super well-coordinated, obsessively work-focused, fast learners, good at crafting specialized tools for handling specific datasets or physical world problems. Things might change surprisingly quickly!
Here’s some categories I think could be pretty world changing, even without buy-in from existing industry:
Cheap production of novel drugs with powerful and specific effects
Engineered organisms (animals, plants, or fungi) or collectives thereof which can make use of hydrocarbons in plastic and can successfully reproduce rapidly using just the material in a landfill plus sunlight and water, while also doing some kind of economically useful activity.
Same, but you feed them on industrial agricultural wastes (easier)
Same, but you can directly feed them fossil fuels (harder)
crude oil consumption, plus ability to live deep under the ocean, plus ability to export products in such a way that they were easy to harvest from AI-controlled submarines or ships above would enable a huge rapid growth of an industrial base.
https://youtube.com/shorts/c1ZhujO0Fqo?si=yMYIh_cGEP20oulb
Organism which can thrive in hard-vacuum, harness solar power and heat gradients, feed on ice, carbon, oxygen, etc. could be ‘planted’ by a small mission to a large asteroid or comet. It would be hard for this to be directly economically productive back on Earth, but could make for a big headstart for the beginnings of a race to colonize the local solar system.
Organic or cyborg computing systems. Scientists have already discussed the possibility of wiring up a bunch of rodent brains together through a computer network using brain-computer-interfaces. But what if you could skip using fragile inconvenient animal brains? What if you could get fungal mycelium to work like neurons for computation? You can grow a whole lot of engineered fungal mycelium really fast, and feed it quite easily. Even if the result was much lower average utility per cubic millimeter than a rodent brain, the ability to manufacture and use it on an industrial scale would likely make up for that.
Plants capable of producing plastic. This could do a lot, actually. Imagine if there were plants which could grow on the interface between land and sea, and produce little plastic bubbles for themselves which functioned as solar stills. They could refine fresh water from salt water using sunlight. What about plants forming highly robust and lengthy lateral stems which could pump this water inland and receive sugars back? Suddenly you have plants which can colonize deserts which border oceans. Plants are limited in how far they can move water vertically because of various difficulties including the vaporization pressure of water and limitations of capillary action. Horizontally though, plants can form enormous root-stem-root-stem complexes and the resulting colonies can cover many square miles.
I could go on about possibilities for hours. I’m not a super-intelligent corporation of a million R&D AIs, so of course my ideas are far more limited in variety and plausibility than what they could come up with. The point is, once things get that far, we can’t predict them using the trends and behavior patterns of traditional human corporations.
This seems right, though I’d interpreted the context of Sarah’s post to be more about what we expect in a pre-superintelligence economy.
Yes, I agree that that’s what the post was talking about. I do think my comment is still relevant since the transition time from pre-superintelligence human-level-AGI economy to superintelligent-AGI-economy may be just a few months. Indeed, that is exactly what I expect due to the high likelihood I place on the rapid effects of recursive self-improvement enabled by human-level-AGI.
I would expect that the company developing the human-level-AGI may even observe the beginnings of a successful RSI process and choose to pursue that in secret without ever releasing the human-level-AGI. Afterall, if it’s useful for rapid potent RSI then it would give their competitors a chance to catch up or overtake them if their competitors also had access to the human-level-AGI.
Thus, from the point of view of outside observers, it may seem that we jump straight from no-AGI to a world affected by technology developed by superintelligent-AGI, without ever seeing either the human-level-AGI or the superintelligent-AGI deployed.
[Edit: FWIW I think that Tom Davidson’s report errs slightly in the other direction, of forecasting that things in the physical world might move somewhat faster than I expect. Maybe 1.2x to 5x faster than I expect. So that puts me somewhere in-between world-as-normal and world-goes-crazy in terms of physical infrastructure and industry. On the other hand, I think Tom Davidson’s report somewhat underestimates the rate at which algorithm / intelligence improvement could occur. Algorithmic improvements which ‘piggyback’ on existing models, and thus start warm and improve cheaply from there, could have quite fast effects with no wait time for additional compute to become available or need to retrain from scratch. If that sort of thing snowballed, which I think it might, the result could get quite capable quite fast. And that would all be happening in software changes, and behind closed doors, so the public wouldn’t necessarily know anything about it.]
I agree that if the AI can run its own experiments (via robotic actuators) it can do R&D prototyping independently of existing private/corporate data, and that’s potentially the whole game.
My current impression is that, as of 2024, we’re starting to see enough investment into AI-controlled robots that in a few years it would be possible to get an “AI experimenter”, albeit in the restricted set of domains where experiments can be automated easily. (biological experiments that are basically restricted to pipetting aqueous solutions and imaging the results? definitely yes. most sorts of benchtop electronics prototyping and testing? i imagine so, though I don’t know for sure. the full range of reactions/syntheses a chemist can run at a lab bench? probably not for some time; creating a “mechanical chemist” is a famously hard problem since methods are so varied, though obviously it’s not in principle impossible.)