Trying for minimal technicalities:
There are at least 3 different technologies with not much surface-level usage similarities referred to as “nanotech”.
Assemblers: basically 3d printers, but way more flexible and able to make things like food, robots, or more assemblers.
Materials: diamondoids, buckytubes, circuitry. We already have some of these really, it’s just that we’d get more kinds of them, and they’d be really cheap to make with a nanotech assembler. Stronger, faster, more powerful versions of what modern tech already can do.
Nanobots, particularly medical: Basically can do all the things living cells can do, but better, and also being able to do most of the things machines can do, and commandable in exact detail. There are also a number of different ways they’d grant immortality enough that they are almost sure to do so even if most of them end up not working out.
Now you can ask questions about each one of these in order, with more specifics.
The question is about all these technologies—though it’s about 2 mainly insofar as 2 is an extension of 1.
So the question is why expect any of these technologies to mature on a timescale of decades?
(Or, assuming FOOM, why assume they’d be relatively low-hanging fruit for a FOOMing AI, such that “trick humans into building me nano assemblers” is a prime strategy for a boxed AI to escape?)
As I said, 2 is already here, and it’s becoming more here gradually.
For 3, we have a proof of concept to rip of: biological cells. Those also happens to have a specialized assembler in them already; the ribosome. And we can print instructions for it already. There’s only 1 problem left and that’s the protein folding problem. The protein folding problem is somewhat rapidly made progress on software wise, and even if that were to fail it won’t be all that long before we ca simply brute force it with computing power. Now, the other kinds of nanobots are less clear.
The assembler (1) is trickier; however, Drexler already sorta made a blueprint for one I think, and 3 will help a great deal with it as well.
For the fooming, it’s the 3 one, and ways to use it. As I said we already have the hardware, and things like the protein folding problem is exactly what an AI would be great at. Once it’s solved that it has full control over biology and can essentially make The Thing and/or a literal mind control virus, and take over that way.
There’s only 1 problem left and that’s the protein folding problem. The protein folding problem is somewhat rapidly made progress on software wise, and even if that were to fail it won’t be all that long before we ca simply brute force it with computing power.
Okay, so one sub-piece of puzzlement I have is why talk of protein folding as a problem that is either solved or unsolved—as if we (or more frighteningly, an AI) could suddenly go from barely being able to do it to 100% capable.
I was also under the impression that protein folding was mathematically horrible in a way that makes it unlikely to be brute forced any time soon, though I just now realized that I may have been thinking of the general problem of predicting chemistry from physics, maybe protein folding is much easier.
Predicting chemistry from physics should be easy with a quantum computer, but appears hard with a classical computer. Often people say that even once you make a classical approximation, ie, assume that the dynamics are easy on a classical computer, that the problem of finding the minimum energy state of a protein is NP-hard. That’s true, but a red herring, since the protein isn’t magically going to know the minimum energy state. Though it’s still possible that there’s some catalyst to push it into the right state, so simulating the dynamics in a vacuum won’t get you the right answer (cf prions). Anyhow, there’s some hope that evolution has found a good toolbox for designing proteins and that if can figure out the abstractions that evolution is using, it will all become easy. In particular, there are building blocks like the alpha helix. Certainly an engineer, whether evolution or us, doesn’t need to understand every protein, just know how to make enough.
I think the possibility that a sufficiently smart AI would quickly find an adequate toolbox for designing proteins is quite plausible. I don’t know what Eliezer means, but the possibility seems to me adequate for his arguments.
I’m not sure protein folding can be brute forced without quantum computers. There’s too many ways for it to fold. In real life, I’m pretty sure quantum tunneling gets involved. Simulations have worked, but I there’s a limit to that.
Trying for minimal technicalities: There are at least 3 different technologies with not much surface-level usage similarities referred to as “nanotech”.
Assemblers: basically 3d printers, but way more flexible and able to make things like food, robots, or more assemblers.
Materials: diamondoids, buckytubes, circuitry. We already have some of these really, it’s just that we’d get more kinds of them, and they’d be really cheap to make with a nanotech assembler. Stronger, faster, more powerful versions of what modern tech already can do.
Nanobots, particularly medical: Basically can do all the things living cells can do, but better, and also being able to do most of the things machines can do, and commandable in exact detail. There are also a number of different ways they’d grant immortality enough that they are almost sure to do so even if most of them end up not working out.
Now you can ask questions about each one of these in order, with more specifics.
The question is about all these technologies—though it’s about 2 mainly insofar as 2 is an extension of 1.
So the question is why expect any of these technologies to mature on a timescale of decades?
(Or, assuming FOOM, why assume they’d be relatively low-hanging fruit for a FOOMing AI, such that “trick humans into building me nano assemblers” is a prime strategy for a boxed AI to escape?)
As I said, 2 is already here, and it’s becoming more here gradually.
For 3, we have a proof of concept to rip of: biological cells. Those also happens to have a specialized assembler in them already; the ribosome. And we can print instructions for it already. There’s only 1 problem left and that’s the protein folding problem. The protein folding problem is somewhat rapidly made progress on software wise, and even if that were to fail it won’t be all that long before we ca simply brute force it with computing power. Now, the other kinds of nanobots are less clear.
The assembler (1) is trickier; however, Drexler already sorta made a blueprint for one I think, and 3 will help a great deal with it as well.
For the fooming, it’s the 3 one, and ways to use it. As I said we already have the hardware, and things like the protein folding problem is exactly what an AI would be great at. Once it’s solved that it has full control over biology and can essentially make The Thing and/or a literal mind control virus, and take over that way.
Okay, so one sub-piece of puzzlement I have is why talk of protein folding as a problem that is either solved or unsolved—as if we (or more frighteningly, an AI) could suddenly go from barely being able to do it to 100% capable.
I was also under the impression that protein folding was mathematically horrible in a way that makes it unlikely to be brute forced any time soon, though I just now realized that I may have been thinking of the general problem of predicting chemistry from physics, maybe protein folding is much easier.
Predicting chemistry from physics should be easy with a quantum computer, but appears hard with a classical computer. Often people say that even once you make a classical approximation, ie, assume that the dynamics are easy on a classical computer, that the problem of finding the minimum energy state of a protein is NP-hard. That’s true, but a red herring, since the protein isn’t magically going to know the minimum energy state. Though it’s still possible that there’s some catalyst to push it into the right state, so simulating the dynamics in a vacuum won’t get you the right answer (cf prions). Anyhow, there’s some hope that evolution has found a good toolbox for designing proteins and that if can figure out the abstractions that evolution is using, it will all become easy. In particular, there are building blocks like the alpha helix. Certainly an engineer, whether evolution or us, doesn’t need to understand every protein, just know how to make enough.
I think the possibility that a sufficiently smart AI would quickly find an adequate toolbox for designing proteins is quite plausible. I don’t know what Eliezer means, but the possibility seems to me adequate for his arguments.
Ah, that’s helpful.
I’m not sure protein folding can be brute forced without quantum computers. There’s too many ways for it to fold. In real life, I’m pretty sure quantum tunneling gets involved. Simulations have worked, but I there’s a limit to that.