The debate about gain of function research started as a debate about infohazards when Fouchier and Kawaoka modified H5N1 in 2011 and published the modified sequence.
It’s possible that gain of function research is therefore mentally associated as being an infohazard. The more recent FHI papers for example mention gain of function research only in relation to infohazards and not the problem of lab leaks in labs doing gain of function research.
The OpenPhil analysis that speaks of gain of function research by calling it dual use research also has a frame that suggests that possible military use or someone stealing engineered viruses and intentionally spreading them is what the problem is about.
This seems to reflect the general human bias that we have an easier time imagining other humans intentionally creating harm then accidentially creating harm. It’s quite similar to naive people thinking that the problem of AGI is humans using AGI’s for nefarious ends.
(I’m not sure to what extent you’re trying to “give background info” versus “be more specific about how people thought of GoF research as an infohazard” versus “be more specific about how GoF research actually was an infohazard” versus other things, so I might be talking past you a bit here.)
The debate about gain of function research started as a debate about infohazards when Fouchier and Kawaoka modified H5N1 in 2011 and published the modified sequence.
So this seems to me likely to be an infohazard that was found through GoF research, but not obviously GoF-research-as-infohazard. That is, even if we grant that the modified sequence was an infohazard and a mistake to publish, it doesn’t then follow that it’s a mistake to talk about GoF research in general. Because when GoF research is already happening, it’s already known within certain circles, and those circles disproportionately contain the people we’d want to keep the knowledge from. It might be the case that talking about GoF research is a mistake, but it’s not obviously so.
What I’m trying to get at is that “info hazard concerns” is pretty vague and not very helpful. What were people concerned about, specifically, and was it a reasonable thing to be concerned about? (It’s entirely possible that people made the mental leap from “this thing found through GoF is an infohazard” to “GoF is an infohazard”, but if so it seems important to realize that that’s a leap.)
a frame that suggests that possible military use or someone stealing engineered viruses and intentionally spreading them is what the problem is about.
Here, too: if this is what we’re worried about, it’s not clear that “not talking about GoF research” helps the problem at all.
The debate about gain of function research started as a debate about infohazards when Fouchier and Kawaoka modified H5N1 in 2011 and published the modified sequence.
It’s possible that gain of function research is therefore mentally associated as being an infohazard. The more recent FHI papers for example mention gain of function research only in relation to infohazards and not the problem of lab leaks in labs doing gain of function research.
The OpenPhil analysis that speaks of gain of function research by calling it dual use research also has a frame that suggests that possible military use or someone stealing engineered viruses and intentionally spreading them is what the problem is about.
This seems to reflect the general human bias that we have an easier time imagining other humans intentionally creating harm then accidentially creating harm. It’s quite similar to naive people thinking that the problem of AGI is humans using AGI’s for nefarious ends.
(I’m not sure to what extent you’re trying to “give background info” versus “be more specific about how people thought of GoF research as an infohazard” versus “be more specific about how GoF research actually was an infohazard” versus other things, so I might be talking past you a bit here.)
So this seems to me likely to be an infohazard that was found through GoF research, but not obviously GoF-research-as-infohazard. That is, even if we grant that the modified sequence was an infohazard and a mistake to publish, it doesn’t then follow that it’s a mistake to talk about GoF research in general. Because when GoF research is already happening, it’s already known within certain circles, and those circles disproportionately contain the people we’d want to keep the knowledge from. It might be the case that talking about GoF research is a mistake, but it’s not obviously so.
What I’m trying to get at is that “info hazard concerns” is pretty vague and not very helpful. What were people concerned about, specifically, and was it a reasonable thing to be concerned about? (It’s entirely possible that people made the mental leap from “this thing found through GoF is an infohazard” to “GoF is an infohazard”, but if so it seems important to realize that that’s a leap.)
Here, too: if this is what we’re worried about, it’s not clear that “not talking about GoF research” helps the problem at all.