To add to the deadly virus point, a bad actor could design both the virus and the cure, vaccinate themselves, and then spread the virus to everyone else. I had the same thought and always been afraid of giving people ideas. I’m still uncomfortable that it’s being discussed even if I know others will probably think of it
Vaccine design is hard, and requires lots of work. Seems strange to assert that someone could just do it on the basis of a theoretical design. Viral design, though, is even harder, and to be clear, we’ve never seen anyone build one from first principles; the most we’ve seen is modification of extant viruses in minor ways where extant vaccines for the original virus are likely to work at least reasonably well.
Yes—I didn’t say it was hard without AI, I said it was hard. Using the best tech in the world, humanity doesn’t *even ideally* have ways to get AI to design safe useful vaccines in less than months, since we need to do actual trials.
To add to the deadly virus point, a bad actor could design both the virus and the cure, vaccinate themselves, and then spread the virus to everyone else. I had the same thought and always been afraid of giving people ideas. I’m still uncomfortable that it’s being discussed even if I know others will probably think of it
Vaccine design is hard, and requires lots of work. Seems strange to assert that someone could just do it on the basis of a theoretical design. Viral design, though, is even harder, and to be clear, we’ve never seen anyone build one from first principles; the most we’ve seen is modification of extant viruses in minor ways where extant vaccines for the original virus are likely to work at least reasonably well.
The context is AI safety and naturally you’re meant to interpret the bad actor as having access to and using a powerful AI.
Yes—I didn’t say it was hard without AI, I said it was hard. Using the best tech in the world, humanity doesn’t *even ideally* have ways to get AI to design safe useful vaccines in less than months, since we need to do actual trials.