A dissenting voice on info-hazards. I appreciate the bulleted list starting of premises and building towards conclusions. Unfortunately I don’t think all the reasoning holds up to close scrutiny. For example, the conclusion that “infohoarders are like black holes for infohazards” conflicts with the premise that “two people can keep a secret if one of them is dead”. The post would have been stronger if it had stopped before getting into community dynamics.
Still, this post moved and clarified my thinking. My sketch at a better argument for a similar conclusion is below:
Definitions:
hard-info-hazard: information that reliably causes catastrophe, no mitigation possible.
soft-info-hazard: information that risks catastrophe, but can be mitigated.
Premises:
Two people can keep a secret if one of them is dead.
If there are hard-info-hazards then we are already extinct, we just don’t know it.
You, by yourself, are not smart enough to tell if an info-hazard is hard or soft.
Authorities with the power to mitigate info-hazards are not aligned with your values.
Possible strategies on discovering an infohazard:
Tell nobody.
Tell everybody.
Follow a responsible disclosure process.
Expected Value calculations left as an exercise for the reader, but responsible disclosure seems favored. The main exception is if we are in Civilizational Hospice where we know we are going extinct in the next decade anyway and are just trying to live our last few years in peace.
A dissenting voice on info-hazards. I appreciate the bulleted list starting of premises and building towards conclusions. Unfortunately I don’t think all the reasoning holds up to close scrutiny. For example, the conclusion that “infohoarders are like black holes for infohazards” conflicts with the premise that “two people can keep a secret if one of them is dead”. The post would have been stronger if it had stopped before getting into community dynamics.
Still, this post moved and clarified my thinking. My sketch at a better argument for a similar conclusion is below:
Definitions:
hard-info-hazard: information that reliably causes catastrophe, no mitigation possible.
soft-info-hazard: information that risks catastrophe, but can be mitigated.
Premises:
Two people can keep a secret if one of them is dead.
If there are hard-info-hazards then we are already extinct, we just don’t know it.
You, by yourself, are not smart enough to tell if an info-hazard is hard or soft.
Authorities with the power to mitigate info-hazards are not aligned with your values.
Possible strategies on discovering an infohazard:
Tell nobody.
Tell everybody.
Follow a responsible disclosure process.
Expected Value calculations left as an exercise for the reader, but responsible disclosure seems favored. The main exception is if we are in Civilizational Hospice where we know we are going extinct in the next decade anyway and are just trying to live our last few years in peace.