(rambly thoughts about my interior thought process incoming)
So, I don’t endorse the actual algorithm I was running here. (i.e “notice dangerous information --> speak out about dangerous information” rather than “make even a crude attempt to reflect on the overall stakes”, which I do think I should do more often)
I think the algorithm Davis followed was basically correct (as I understand it, “start writing a post on dangerous info --> reflect on overall risk of using a particular example / check for less dangerous examples --> publish article with less dangerous examples and/or decide the risk is acceptable”)
It’s particularly salient to me that Ziz is correct to call me out here because I had recently noticed an inconsistency in myself: If I saw someone make a dangerous-seeming-decision, and they already double checked their reasoning, and the triple-checked their reasoning seeking out someone with different priors… I would probably demand that they quadruple-check their reasoning.
Which is maybe fine, except that if they had only doublechecked their math… I’m aware that I’d be satisfied if I demanded that they triplecheck in. And if they had quadruplechecked it, I’d probably demand that they check it a 5th time.
I lean towards “it’s better to have this algorithm to not have it to make sure people are doublechecking their dangerous decisions at all, but it’s definitely better to actually have a principled take on how much danger is reasonable.”
And this post was the first instance of me running into this behavior pattern since reflecting on it.
That all said...
In this particular post, which is literally about being careful with information hazards, which includes a potential information hazard… it seems sort of amiss to not at least address where to draw the line?
If this doubled such attacks, it would not be a feather on the scales, and you know that.
(rambly thoughts about my interior thought process incoming)
So, I don’t endorse the actual algorithm I was running here. (i.e “notice dangerous information --> speak out about dangerous information” rather than “make even a crude attempt to reflect on the overall stakes”, which I do think I should do more often)
I think the algorithm Davis followed was basically correct (as I understand it, “start writing a post on dangerous info --> reflect on overall risk of using a particular example / check for less dangerous examples --> publish article with less dangerous examples and/or decide the risk is acceptable”)
It’s particularly salient to me that Ziz is correct to call me out here because I had recently noticed an inconsistency in myself: If I saw someone make a dangerous-seeming-decision, and they already double checked their reasoning, and the triple-checked their reasoning seeking out someone with different priors… I would probably demand that they quadruple-check their reasoning.
Which is maybe fine, except that if they had only doublechecked their math… I’m aware that I’d be satisfied if I demanded that they triplecheck in. And if they had quadruplechecked it, I’d probably demand that they check it a 5th time.
I lean towards “it’s better to have this algorithm to not have it to make sure people are doublechecking their dangerous decisions at all, but it’s definitely better to actually have a principled take on how much danger is reasonable.”
And this post was the first instance of me running into this behavior pattern since reflecting on it.
That all said...
In this particular post, which is literally about being careful with information hazards, which includes a potential information hazard… it seems sort of amiss to not at least address where to draw the line?