Biorisk—well wouldn’t it be nice if we’d all been familiar with the main principles of biorisk before 2020? i certainly regretted sticking my head in the sand.
> If concerned, intelligent people cannot articulate their reasons for censorship, cannot coordinate around principles of information management, then that itself is a cause for concern. Discussions may simply move to unregulated forums, and dangerous ideas will propagate through well intentioned ignorance.
Well. It certainly sounds prescient in hindsight, doesn’t it?
Infohazards in particular cross my mind: so many people operate on extremely bad information right now. Conspiracies theories abound, and I imagine the legitimate coordination for secrecy surrounding the topic do not help in the least. What would help? Exactly this essay. A clear model of *what* we should expect well-intentioned secrecy to cover, so we can reason sanely over when it’s obviously not.
Y’all done good. This taxonomy clarifies risk profiles better than Gregory Lewis’ article, though I think his includes a few vivid-er examples.
I opened a document to experiment tweaking away a little dryness from the academic tone. I hope you don’t take offense. Your writing represents massive improvements in readability in its examples and taxonomy, and you make solid, straightforward choices in phrasing. No hopelessly convoluted sentence trees. I don’t want to discount that. Seriously! Good job.
As I read I had a few ideas spark on things that could likely get done at a layman level, in line with spiracular’s comment. That comment could use some expansion, especially in the direction of “Prefer to discuss this over that, or discuss in *this way* over *that way” for bad topics. Very relevantly, I think basic facts should get added to some the good discussion topics, since they represent information it’s better to disseminate! we seek to review basic facts under the good discussion topics, since they represent information it’s better to disseminate (EDIT, see comments).
Summarize or link to standard lab safety materials.
Summarize the various levels of PPE and sanitation practices. It doesn’t have to get into the higher end to prove useful for people:
how do you keep dishes sanitary?
the fridge?
a wound?
How can you neutralize sewage,
purify water
responsibly use antiobiotics?
The state of talent… I imagine there’s low-hanging fruit here but idk what it is. Could list typical open positions and what the general degree track looks like.
Give a quick overview of the major biosecurity funds
Do a give-well-esque summary of which organizations have room for more funding, and which promising subcause-areas have relatively few/poor organizations pursuing them. (open phil’s)
“Basic facts” as “safe discussion topics”: Ooh, I disagree! I think this heuristic doesn’t always hold, especially for people writing on a large platform.
For basic information, it is sometimes a good idea to think twice if a fact might be very-skewed towards benefiting harmful actions over protective ones. If you have a big platform, it is especially important to do so.
(It might actually be more important for someone to do this for basic facts, than sophisticated ones? They’re the ones a larger audience of amateurs can grasp.)
If something is already widely known, that does somewhat reduce the extent of your “fault” for disseminating it. That rule is more likely to hold for basic facts.
But if there is a net risk to a piece of information, and you are spreading it to people who wouldn’t otherwise know? Then larger audiences are a risk-multiplier. So, sometimes spreading a low-risk basic thing widely could be more dangerous, overall, than spreading an high-risk but obscure and specialist thing.
It was easy for me to think of at least 2 cases where spreading an obvious, easy-to-grasp fact could disproportionately increase the hazard of bad actors relative to good ones, in at least some petty ways. Here’s one.
Ex: A member of the Rajneeshee cult once deliberately gave a bunch of people food poisoning, then got arrested. This is a pretty basic fact. But I wouldn’t want to press a button that would disseminate this fact to 10 million random people? People knowing about this isn’t actually particularly protective against food poisoning, and I’d bet that there is least 1 nasty human in 10 million people. If I don’t have an anticipated benefit to sharing, I would prefer not to risk inspiring that person.
On the other hand, passing around the fact that a particular virus needs mucus membranes to enter cells seems… net-helpful? It’s easier for people to use that to advise their protective measures, and it’s unlikely to help a rare bad actor who is sitting the razor’s-edge case where they would have infected someone IF ONLY they had known to aim for the mucus membranes, AND where they only knew about that because you told them.
(And then you have complicated intermediate cases. Off the top of my head, WHO’s somewhat-dishonest attempt to convince people that masks don’t work, in a bid to save them for the medical professionals? I don’t think I like what they did (they basically set institutional trust on fire), but the situation they were in does tug at some edge-cases around trying to influence actions vs beliefs. The fact that masks did work, but had a limited supply, meant that throwing information in any direction was going to benefit some and harm others. It also highlights that, paradoxically, it can be common for “basic” knowledge to be flat-out wrong, if your source is being untrustworthy and you aren’t being careful...)
Lab Safety Procedures/PPE/Sanitation: I think I have some ideas for where I could start on that? BSL is probably a good place to start.
I’d feel pretty weird posting about that on LessWrong, tbh? (I still might, though.)
I don’t currently feel like writing this. But, I’ll keep it in mind as a possibility.
Summary of orgs, positions, room-for-funding: I do not have the means, access, clearance, or credentials to do this. (I don’t care about me lacking some of those credentials, but other people have made it clear that they do.)
I really would like this to exist! I get the sense that better people than me have tried, and were usually were only able to get part-way, but I haven’t tracked it recently. This has led me to assume that this task is more difficult than you’d expect. I have seen a nice copy of a biosecurity-relevant-orgs spreadsheet circulating at one point, though (which I think could get partial-credit).
The closest thing I probably could output are some thoughts on what broad-projects or areas of research seem likely to be valuable and/or underfunded. But I would expect it to be lower-resolution, and less valuable to people.
Thanks for the proposed edits! I’ll look them over.
“Careful, clear, and dry” was basically the tone that I intended. I will try to incorporate the places where your wording was clearer than mine, and I have found several places where it was.
In hindsight, I see I wrote very unclearly.. It sounds like I recommended “basic facts” as a separate category of Open Discussion topics. You correctly point out the serious issues with assuming “basic” means “safe”. It does not. It really, really does not!
Certainly not what I meant to say. I meant we (the lesswrong community) should actively discuss basic facts *within* the good discussion topics.
I’ve actually had several people say they liked the Concrete Examples section, but that they wish I’d said more that would help them recreate the thought-process.
Unfortunately, these were old thoughts for me. The logic behind a lot of them feels… “self-evident” or “obvious” to me, or something? Which makes me a worse teacher, because I’m a little blind to what it is about them that isn’t landing.
I’d need to understand what people were seeing or missing, to be able to offer helpful guidance around it. And… nobody commented.
(My rant on basic knowledge was a partial-attempt on my part, to crack open my logic for one of these.)
Edit: I added my core heuristic to the Concrete Examples thread
Biorisk—well wouldn’t it be nice if we’d all been familiar with the main principles of biorisk before 2020? i certainly regretted sticking my head in the sand.
> If concerned, intelligent people cannot articulate their reasons for censorship, cannot coordinate around principles of information management, then that itself is a cause for concern. Discussions may simply move to unregulated forums, and dangerous ideas will propagate through well intentioned ignorance.
Well. It certainly sounds prescient in hindsight, doesn’t it?
Infohazards in particular cross my mind: so many people operate on extremely bad information right now. Conspiracies theories abound, and I imagine the legitimate coordination for secrecy surrounding the topic do not help in the least. What would help? Exactly this essay. A clear model of *what* we should expect well-intentioned secrecy to cover, so we can reason sanely over when it’s obviously not.
Y’all done good. This taxonomy clarifies risk profiles better than Gregory Lewis’ article, though I think his includes a few vivid-er examples.
I opened a document to experiment tweaking away a little dryness from the academic tone. I hope you don’t take offense. Your writing represents massive improvements in readability in its examples and taxonomy, and you make solid, straightforward choices in phrasing. No hopelessly convoluted sentence trees. I don’t want to discount that. Seriously! Good job.
As I read I had a few ideas spark on things that could likely get done at a layman level, in line with spiracular’s comment. That comment could use some expansion, especially in the direction of “Prefer to discuss this over that, or discuss in *this way* over *that way” for bad topics. Very relevantly,
I think basic facts should get added to some the good discussion topics, since they represent information it’s better to disseminate!we seek to review basic facts under the good discussion topics, since they represent information it’s better to disseminate (EDIT, see comments).Summarize or link to standard lab safety materials.
Summarize the various levels of PPE and sanitation practices. It doesn’t have to get into the higher end to prove useful for people:
how do you keep dishes sanitary?
the fridge?
a wound?
How can you neutralize sewage,
purify water
responsibly use antiobiotics?
The state of talent… I imagine there’s low-hanging fruit here but idk what it is. Could list typical open positions and what the general degree track looks like.
Give a quick overview of the major biosecurity funds
Do a give-well-esque summary of which organizations have room for more funding, and which promising subcause-areas have relatively few/poor organizations pursuing them. (open phil’s)
“Basic facts” as “safe discussion topics”: Ooh, I disagree! I think this heuristic doesn’t always hold, especially for people writing on a large platform.
For basic information, it is sometimes a good idea to think twice if a fact might be very-skewed towards benefiting harmful actions over protective ones. If you have a big platform, it is especially important to do so.
(It might actually be more important for someone to do this for basic facts, than sophisticated ones? They’re the ones a larger audience of amateurs can grasp.)
If something is already widely known, that does somewhat reduce the extent of your “fault” for disseminating it. That rule is more likely to hold for basic facts.
But if there is a net risk to a piece of information, and you are spreading it to people who wouldn’t otherwise know? Then larger audiences are a risk-multiplier. So, sometimes spreading a low-risk basic thing widely could be more dangerous, overall, than spreading an high-risk but obscure and specialist thing.
It was easy for me to think of at least 2 cases where spreading an obvious, easy-to-grasp fact could disproportionately increase the hazard of bad actors relative to good ones, in at least some petty ways. Here’s one.
Ex: A member of the Rajneeshee cult once deliberately gave a bunch of people food poisoning, then got arrested. This is a pretty basic fact. But I wouldn’t want to press a button that would disseminate this fact to 10 million random people? People knowing about this isn’t actually particularly protective against food poisoning, and I’d bet that there is least 1 nasty human in 10 million people. If I don’t have an anticipated benefit to sharing, I would prefer not to risk inspiring that person.
On the other hand, passing around the fact that a particular virus needs mucus membranes to enter cells seems… net-helpful? It’s easier for people to use that to advise their protective measures, and it’s unlikely to help a rare bad actor who is sitting the razor’s-edge case where they would have infected someone IF ONLY they had known to aim for the mucus membranes, AND where they only knew about that because you told them.
(And then you have complicated intermediate cases. Off the top of my head, WHO’s somewhat-dishonest attempt to convince people that masks don’t work, in a bid to save them for the medical professionals? I don’t think I like what they did (they basically set institutional trust on fire), but the situation they were in does tug at some edge-cases around trying to influence actions vs beliefs. The fact that masks did work, but had a limited supply, meant that throwing information in any direction was going to benefit some and harm others. It also highlights that, paradoxically, it can be common for “basic” knowledge to be flat-out wrong, if your source is being untrustworthy and you aren’t being careful...)
Edit: Just separating this for coherence’s sake
Lab Safety Procedures/PPE/Sanitation: I think I have some ideas for where I could start on that? BSL is probably a good place to start.
I’d feel pretty weird posting about that on LessWrong, tbh? (I still might, though.)
I don’t currently feel like writing this. But, I’ll keep it in mind as a possibility.
Summary of orgs, positions, room-for-funding: I do not have the means, access, clearance, or credentials to do this. (I don’t care about me lacking some of those credentials, but other people have made it clear that they do.)
I really would like this to exist! I get the sense that better people than me have tried, and were usually were only able to get part-way, but I haven’t tracked it recently. This has led me to assume that this task is more difficult than you’d expect. I have seen a nice copy of a biosecurity-relevant-orgs spreadsheet circulating at one point, though (which I think could get partial-credit).
The closest thing I probably could output are some thoughts on what broad-projects or areas of research seem likely to be valuable and/or underfunded. But I would expect it to be lower-resolution, and less valuable to people.
Thanks for the proposed edits! I’ll look them over.
“Careful, clear, and dry” was basically the tone that I intended. I will try to incorporate the places where your wording was clearer than mine, and I have found several places where it was.
Thank you for your reply! I’m very pleased.
In hindsight, I see I wrote very unclearly.. It sounds like I recommended “basic facts” as a separate category of Open Discussion topics. You correctly point out the serious issues with assuming “basic” means “safe”. It does not. It really, really does not!
Certainly not what I meant to say. I meant we (the lesswrong community) should actively discuss basic facts *within* the good discussion topics.
Ah! Thanks for the clarification.
I’ve actually had several people say they liked the Concrete Examples section, but that they wish I’d said more that would help them recreate the thought-process.
Unfortunately, these were old thoughts for me. The logic behind a lot of them feels… “self-evident” or “obvious” to me, or something? Which makes me a worse teacher, because I’m a little blind to what it is about them that isn’t landing.
I’d need to understand what people were seeing or missing, to be able to offer helpful guidance around it. And… nobody commented.
(My rant on basic knowledge was a partial-attempt on my part, to crack open my logic for one of these.)
Edit: I added my core heuristic to the Concrete Examples thread