All of these sound sensible. But for 1-6, when I reverse the advice, it sounds roughly equally sensible, and it feels very hard for me to know whether people are more often erring in one direction other. So I’m wary of believing the overall claim that these are major traps for new alignment researchers.
I have the intuition (maybe from applause lights) that if negating a point sounds obviously implausible, then the point is obviously true and it is therefore somewhat meaningless to claim it.
My idea in writing this was to identify some traps that I thought were non obvious (some of which I think I fell into as new alignment researcher).
What would the sensible reverse of number 5? I can generate those them for 1-4 and 6, but I am unsure what the benefit could be of confusing intuitions with testable hypotheses?
Reversal: when you have different intuitions about high-level questions, it’s often not worth spending a lot of time debating them extensively—instead, move onto doing whatever research your intuitions imply will be valuable.
ah, like that. Thank you for explaining. I wouldn’t consider that a reversal cause you’re then still converting intuitions into testable hypotheses. But the emphasis on discussion versus experimentation is then reversed indeed.
All of these sound sensible. But for 1-6, when I reverse the advice, it sounds roughly equally sensible, and it feels very hard for me to know whether people are more often erring in one direction other. So I’m wary of believing the overall claim that these are major traps for new alignment researchers.
I have the intuition (maybe from applause lights) that if negating a point sounds obviously implausible, then the point is obviously true and it is therefore somewhat meaningless to claim it.
My idea in writing this was to identify some traps that I thought were non obvious (some of which I think I fell into as new alignment researcher).
What would the sensible reverse of number 5? I can generate those them for 1-4 and 6, but I am unsure what the benefit could be of confusing intuitions with testable hypotheses?
Reversal: when you have different intuitions about high-level questions, it’s often not worth spending a lot of time debating them extensively—instead, move onto doing whatever research your intuitions imply will be valuable.
ah, like that. Thank you for explaining. I wouldn’t consider that a reversal cause you’re then still converting intuitions into testable hypotheses. But the emphasis on discussion versus experimentation is then reversed indeed.