I think we’re kind of talking past each other, and it’s a matter of emphasis.
The details for how to do gain-of-function research have been reasonably public for decades. Biology class exists, and is in fact mandatory to graduate high school. Coursera has free biology classes for some subjects and cheap ones for others. I don’t see how any of that is meaningfully different from taking out a billboard in Times Square and telling people that if they study the relevant literature real hard, they too can create deadly pandemics in their basement with kidnapped feral cats.
Lest you think I’m being flippant, I asked GPT-4 to describe the 2014 pause in gain-of-function research, and this is what it said:
“”″
The pause in gain-of-function (GOF) research in biology, particularly in the United States, refers to a moratorium that was implemented in October 2014. This moratorium was a response to public concerns about the safety and security risks associated with such research. Gain-of-function research involves manipulating viruses or other organisms to increase their capabilities, like making a virus more transmissible or deadly, often to understand more about disease pathways and to develop treatments or vaccines.
Here’s an overview of how the pause came about:
1. **Rising Concerns**: Prior to 2014, there had been increasing public and scientific debate over the risks and benefits of gain-of-function research. High-profile experiments, such as those involving H5N1 influenza (bird flu) that made it transmissible in mammals, raised alarm about the potential for accidental or deliberate misuse of these pathogens.
2. **The U.S. Government’s Decision**: In response to these concerns, the U.S. government announced a funding pause for any new gain-of-function research that could make influenza, MERS, or SARS viruses more virulent or transmissible. This decision was made to allow time for a thorough assessment of the risks and benefits of this type of research.
3. **Deliberations and Recommendations**: During the moratorium, the National Science Advisory Board for Biosecurity (NSABB) and other bodies were tasked with evaluating the risks and developing a framework for future GOF research. This process involved extensive consultation with researchers, bioethicists, and the public.
4. **Lifting the Pause**: The pause was lifted in December 2017, following the establishment of the HHS P3CO (Potential Pandemic Pathogen Care and Oversight) Framework. This framework set out new guidelines and oversight mechanisms for gain-of-function research, aimed at minimizing risks while allowing important scientific research to continue.
The pause was significant as it highlighted the complex ethical, safety, and security issues surrounding gain-of-function research. It also underscored the need for robust regulatory frameworks to balance scientific advancement with risk management.
“”″
To me, this sounds a lot more like my infopandora story than the typical rationalist infohoarder story about how one deals with infohazards. There was a clear danger, the public was alerted, the public was unhappy, changes were made, research was directed into narrower, safer channels, and society went back to doing its thing.
if they study the relevant literature real hard, they too can create deadly pandemics in their basement with kidnapped feral cats.
Here’s a question:
Question A: Suppose that a person really really wants to create a novel strain of measles that is bio-engineered to be resistant to the current measles vaccine. This person has high but not extreme intelligence and conscientiousness, and has a high school biology education, and has 6 months to spend, and has a budget of $10,000, and has access to typical community college biology laboratory equipment. What’s the probability that they succeed?
I feel extremely strongly that the answer right now is “≈0%”. That’s based for example on this podcast interview with one of the world experts on those kinds of questions.
What do you think the answer to Question A is?
If you agree with me that the answer right now is “≈0%”, then I have a follow-up question:
Question B: Suppose I give you a magic wand. If you wave the wand, it will instantaneously change the answer to Question A to be “90%”. Would you wave that wand or not?
(My answer is “obviously no”.)
There was a clear danger, the public was alerted, the public was unhappy, changes were made, research was directed into narrower, safer channels, and society went back to doing its thing.
I’m strongly in favor of telling the public what gain-of-function research is and why they should care about its occurrence.
I’m strongly opposed to empowering millions of normal members of the public to do gain-of-function research in their garages.
Do you see the difference?
If you’re confused by the biology example, here’s a physics one:
I’m strongly in favor of telling the public what uranium enrichment is and why they should care about its occurrence.
I’m strongly opposed to empowering millions of normal members of the public to enrich uranium in their garages [all the way to weapons-grade, at kilogram scale, using only car parts and household chemicals].
Hm, perhaps you have convinced me that I am omitting something important from my description of my mental models. I don’t think my internal mental models are as deeply messed up as our disagreements about these topics suggest they would be. But maybe they are?
I would indeed wave a slightly different magic wand, which is labeled “every government on Earth knows what gain-of-function research is and how to do it, if they didn’t already, but my government gets first crack at it for as long as they like and will ask for”. But that is a fundamentally different wand than the one you were asking about.
So probably the disconnect in emphasis comes down to my unusual political beliefs and the resulting wild divergence in political priorities.
I think we’re kind of talking past each other, and it’s a matter of emphasis.
The details for how to do gain-of-function research have been reasonably public for decades. Biology class exists, and is in fact mandatory to graduate high school. Coursera has free biology classes for some subjects and cheap ones for others. I don’t see how any of that is meaningfully different from taking out a billboard in Times Square and telling people that if they study the relevant literature real hard, they too can create deadly pandemics in their basement with kidnapped feral cats.
Lest you think I’m being flippant, I asked GPT-4 to describe the 2014 pause in gain-of-function research, and this is what it said:
“”″
The pause in gain-of-function (GOF) research in biology, particularly in the United States, refers to a moratorium that was implemented in October 2014. This moratorium was a response to public concerns about the safety and security risks associated with such research. Gain-of-function research involves manipulating viruses or other organisms to increase their capabilities, like making a virus more transmissible or deadly, often to understand more about disease pathways and to develop treatments or vaccines.
Here’s an overview of how the pause came about:
1. **Rising Concerns**: Prior to 2014, there had been increasing public and scientific debate over the risks and benefits of gain-of-function research. High-profile experiments, such as those involving H5N1 influenza (bird flu) that made it transmissible in mammals, raised alarm about the potential for accidental or deliberate misuse of these pathogens.
2. **The U.S. Government’s Decision**: In response to these concerns, the U.S. government announced a funding pause for any new gain-of-function research that could make influenza, MERS, or SARS viruses more virulent or transmissible. This decision was made to allow time for a thorough assessment of the risks and benefits of this type of research.
3. **Deliberations and Recommendations**: During the moratorium, the National Science Advisory Board for Biosecurity (NSABB) and other bodies were tasked with evaluating the risks and developing a framework for future GOF research. This process involved extensive consultation with researchers, bioethicists, and the public.
4. **Lifting the Pause**: The pause was lifted in December 2017, following the establishment of the HHS P3CO (Potential Pandemic Pathogen Care and Oversight) Framework. This framework set out new guidelines and oversight mechanisms for gain-of-function research, aimed at minimizing risks while allowing important scientific research to continue.
The pause was significant as it highlighted the complex ethical, safety, and security issues surrounding gain-of-function research. It also underscored the need for robust regulatory frameworks to balance scientific advancement with risk management.
“”″
To me, this sounds a lot more like my infopandora story than the typical rationalist infohoarder story about how one deals with infohazards. There was a clear danger, the public was alerted, the public was unhappy, changes were made, research was directed into narrower, safer channels, and society went back to doing its thing.
Here’s a question:
Question A: Suppose that a person really really wants to create a novel strain of measles that is bio-engineered to be resistant to the current measles vaccine. This person has high but not extreme intelligence and conscientiousness, and has a high school biology education, and has 6 months to spend, and has a budget of $10,000, and has access to typical community college biology laboratory equipment. What’s the probability that they succeed?
I feel extremely strongly that the answer right now is “≈0%”. That’s based for example on this podcast interview with one of the world experts on those kinds of questions.
What do you think the answer to Question A is?
If you agree with me that the answer right now is “≈0%”, then I have a follow-up question:
Question B: Suppose I give you a magic wand. If you wave the wand, it will instantaneously change the answer to Question A to be “90%”. Would you wave that wand or not?
(My answer is “obviously no”.)
I’m strongly in favor of telling the public what gain-of-function research is and why they should care about its occurrence.
I’m strongly opposed to empowering millions of normal members of the public to do gain-of-function research in their garages.
Do you see the difference?
If you’re confused by the biology example, here’s a physics one:
I’m strongly in favor of telling the public what uranium enrichment is and why they should care about its occurrence.
I’m strongly opposed to empowering millions of normal members of the public to enrich uranium in their garages [all the way to weapons-grade, at kilogram scale, using only car parts and household chemicals].
Hm, perhaps you have convinced me that I am omitting something important from my description of my mental models. I don’t think my internal mental models are as deeply messed up as our disagreements about these topics suggest they would be. But maybe they are?
I would indeed wave a slightly different magic wand, which is labeled “every government on Earth knows what gain-of-function research is and how to do it, if they didn’t already, but my government gets first crack at it for as long as they like and will ask for”. But that is a fundamentally different wand than the one you were asking about.
So probably the disconnect in emphasis comes down to my unusual political beliefs and the resulting wild divergence in political priorities.
This last wand being approximately the bargain the Oppenheimer struck with the American government in running the Manhattan Project.