Tangent: I don’t think I understand the distinction you’ve made between “AI designed nanotechnology” and “human technology”. Human technology already includes “actual nanotechnology”, e.g. nanolithography in semiconductor production.
I agree that if the AGI gives us a blueprint for the smallest self-replicating nanobot that we’ll need to bootstrap the rest of the nanobot swarm, all we have to do is assemble that first nanobot, and the rest follows. It’s very elegant.
We still need to build that very first self-replicating nanobot though.
We can either do so atom-by-atom with some type of molecular assembler like the ones discussed in Nanosystems, or we can synthesize DNA and use clever tricks to get some existing biology to build things we want for us, or maybe we can build it from a process that the AGI gives us that only uses chemical reactions or lab/industrial production techniques.
If we go with the molecular assembler approach, we need to build one of those first, so that we can build the first self-replicating nanobot. This is effectively the same argument I made above, so I’m going to skip it.
If we go with the DNA approach, then the AGI needs to give us that DNA sequence, and we have to hope that we can create it in a reasonable time despite our poor yield rate and time for on DNA synthesis on longer sequences. If the sequence is too long, we might be in a place where we first need to ask the AGI to design new DNA synthesis machines, otherwise we’ll be stuck. In that world, we return to my arguments above. In the world where the AGI gave us a reasonably length DNA sequence, say the size of a very small cell or something, we can continue. The COVID-19 vaccine provides an example of how this goes. We have an intelligent entity (humans) writing code in DNA, synthesizing that DNA, converting it to mRNA, and getting a biological system (human cells) to read that code and produce proteins. Humanity has these tools. I am not sure why we would assume that the company that develops AGI has them. At multiple steps in the chain of what Pfizer and Moderna did to bring mRNA vaccines to market, there are single vendor gatekeepers who hold the only tooling or processes for industrial production. If we assume that you have all of the tooling and processes, we still need to talk about cycle times. I believe Pfizer aimed to get the cycle time (raw materials → synthesized vaccines) for a batch of vaccine down from 15 weeks to 8 weeks. This is an incredibly complex, amazing achievement—we literally wrote a program in DNA, created a way to deliver it to the human body, and it executed successfully in that environment. However, it’s also an example of the current limitations we have. Synthesizing from scratch the mRNA needed to generate a single protein takes >8 weeks, even if you have the full assembly line figured out. This will get faster in time, and we’ll get better at doing it, but I don’t see any reason to think that we’ll have some type of universal / programmable assembly line for an AGI to use anytime soon.
If we go with a series of chemical reactions/lab/industrial production techniques, we need to build clean rooms and labs and vacuum chambers and whatever else is used to implement whatever process the AGI gives us for synthesizing the nanobots. Conceptually this is the simplest idea for how you could get something to work quickly. If the AGI gave you a list of chemicals, metals, biological samples and a step-by-step process of how to mix, drain, heat, sift, repeat, and at the end of this process you had self-replicating nanobots, that would be pretty cool. This is basically taking evolution’s random walk from a planetary petri dish to the life we see today and asking, “could an AGI shorten the duration from a billion years of random iterative development into mere weeks of some predetermined process to get the first self-replicating nanobots?” The problem with programming is that interpreting code is hard. Anything that can interpret the nanobot equivalent of machine code, like instructions for how and where to melt GPU factories, is going to be vastly more complex than the current state-of-the-heart R&D being done by any human lab today. I don’t see a way where this doesn’t reduce to the same Factorio problem I’ve been describing. We’ll first need to synthesize A, so that we can synthesize B, so that we can synthesize C, so that we can synthesize D, and each step will require novel setups and production lines and time, and at the end of it we’ll have a sequence of steps that looks an awful lot like a molecular assembly line for the creation of the very first self-replicating nanobots.
The hypothetical world(s) where these types of constraints aren’t problems for a “pivotal act” are world(s) where the AGI can give us a recipe for the self-replicating nanobots that we can build in our living room at home with a pair of tweezers and materials from Amazon. The progression of human technology over the past ~60 years in the fields of nano-scale engineering or synthetic biology has been increasingly elaborate, complex, time-consuming, and low-yield processes or lab equipment to replicate the simplest structures that life produces ad-hoc. I am certain this limitation will be conquered, and I’m equally certain that AI/ML systems will be instrumental in doing so, but I have no evidence to rationally conclude that there’s not a mountain of prerequisite tools still remaining for humanity to build before something like “design anything at any scale” capabilities are generally available in a way that an AGI could make use of them.
Tangent: If we’re concerned about destroying the world, deliberately building self-replicating nanobots that start simple but rapidly assemble into something arbitrarily complex from the whims of an AGI seems like a bad idea, which is why my original post was focused on a top-down hardware/software systems engineering process where the humans involved could presumably understand the plans, schematics, and programming that the AGI handed to them prior to the construction and deployment of those nanobots.
Tangent: I don’t think I understand the distinction you’ve made between “AI designed nanotechnology” and “human technology”. Human technology already includes “actual nanotechnology”, e.g. nanolithography in semiconductor production.
I agree that if the AGI gives us a blueprint for the smallest self-replicating nanobot that we’ll need to bootstrap the rest of the nanobot swarm, all we have to do is assemble that first nanobot, and the rest follows. It’s very elegant.
We still need to build that very first self-replicating nanobot though.
We can either do so atom-by-atom with some type of molecular assembler like the ones discussed in Nanosystems, or we can synthesize DNA and use clever tricks to get some existing biology to build things we want for us, or maybe we can build it from a process that the AGI gives us that only uses chemical reactions or lab/industrial production techniques.
If we go with the molecular assembler approach, we need to build one of those first, so that we can build the first self-replicating nanobot. This is effectively the same argument I made above, so I’m going to skip it.
If we go with the DNA approach, then the AGI needs to give us that DNA sequence, and we have to hope that we can create it in a reasonable time despite our poor yield rate and time for on DNA synthesis on longer sequences. If the sequence is too long, we might be in a place where we first need to ask the AGI to design new DNA synthesis machines, otherwise we’ll be stuck. In that world, we return to my arguments above. In the world where the AGI gave us a reasonably length DNA sequence, say the size of a very small cell or something, we can continue. The COVID-19 vaccine provides an example of how this goes. We have an intelligent entity (humans) writing code in DNA, synthesizing that DNA, converting it to mRNA, and getting a biological system (human cells) to read that code and produce proteins. Humanity has these tools. I am not sure why we would assume that the company that develops AGI has them. At multiple steps in the chain of what Pfizer and Moderna did to bring mRNA vaccines to market, there are single vendor gatekeepers who hold the only tooling or processes for industrial production. If we assume that you have all of the tooling and processes, we still need to talk about cycle times. I believe Pfizer aimed to get the cycle time (raw materials → synthesized vaccines) for a batch of vaccine down from 15 weeks to 8 weeks. This is an incredibly complex, amazing achievement—we literally wrote a program in DNA, created a way to deliver it to the human body, and it executed successfully in that environment. However, it’s also an example of the current limitations we have. Synthesizing from scratch the mRNA needed to generate a single protein takes >8 weeks, even if you have the full assembly line figured out. This will get faster in time, and we’ll get better at doing it, but I don’t see any reason to think that we’ll have some type of universal / programmable assembly line for an AGI to use anytime soon.
If we go with a series of chemical reactions/lab/industrial production techniques, we need to build clean rooms and labs and vacuum chambers and whatever else is used to implement whatever process the AGI gives us for synthesizing the nanobots. Conceptually this is the simplest idea for how you could get something to work quickly. If the AGI gave you a list of chemicals, metals, biological samples and a step-by-step process of how to mix, drain, heat, sift, repeat, and at the end of this process you had self-replicating nanobots, that would be pretty cool. This is basically taking evolution’s random walk from a planetary petri dish to the life we see today and asking, “could an AGI shorten the duration from a billion years of random iterative development into mere weeks of some predetermined process to get the first self-replicating nanobots?” The problem with programming is that interpreting code is hard. Anything that can interpret the nanobot equivalent of machine code, like instructions for how and where to melt GPU factories, is going to be vastly more complex than the current state-of-the-heart R&D being done by any human lab today. I don’t see a way where this doesn’t reduce to the same Factorio problem I’ve been describing. We’ll first need to synthesize A, so that we can synthesize B, so that we can synthesize C, so that we can synthesize D, and each step will require novel setups and production lines and time, and at the end of it we’ll have a sequence of steps that looks an awful lot like a molecular assembly line for the creation of the very first self-replicating nanobots.
The hypothetical world(s) where these types of constraints aren’t problems for a “pivotal act” are world(s) where the AGI can give us a recipe for the self-replicating nanobots that we can build in our living room at home with a pair of tweezers and materials from Amazon. The progression of human technology over the past ~60 years in the fields of nano-scale engineering or synthetic biology has been increasingly elaborate, complex, time-consuming, and low-yield processes or lab equipment to replicate the simplest structures that life produces ad-hoc. I am certain this limitation will be conquered, and I’m equally certain that AI/ML systems will be instrumental in doing so, but I have no evidence to rationally conclude that there’s not a mountain of prerequisite tools still remaining for humanity to build before something like “design anything at any scale” capabilities are generally available in a way that an AGI could make use of them.
Tangent: If we’re concerned about destroying the world, deliberately building self-replicating nanobots that start simple but rapidly assemble into something arbitrarily complex from the whims of an AGI seems like a bad idea, which is why my original post was focused on a top-down hardware/software systems engineering process where the humans involved could presumably understand the plans, schematics, and programming that the AGI handed to them prior to the construction and deployment of those nanobots.