I think it’s a positive if alignment researchers feel like it’s an allowed option to trust their own technical intuitions over the technical intuitions of this or that more-senior researcher.
Overly dismissing old-guard researchers is obviously a way the field can fail as well. But the field won’t advance much at all if most people don’t at least try to build their own models.
Koen also leans more on deference in his comment than I’d like, so I upvoted your ‘deferential but in the opposite direction’ comment as a corrective, handoflixue. :P But I think it would be a much better comment if it didn’t conflate epistemic authority with “fame” (I don’t think fame is at all a reliable guide to epistemic ability here), and if it didn’t equate “appealing to your own guesses” with “anti-vaxxers”.
Alignment is a young field; “anti-vaxxer” is a term you throw at people after vaccines have existed for 200 years, not a term you throw at the very first skeptical researchers arguing about vaccines in 1800. Even if the skeptics are obviously and decisively wrong at an early date (which indeed not-infrequently happens in science!), it’s not the right way to establish the culture for those first scientific debates.
I think it’s a positive if alignment researchers feel like it’s an allowed option to trust their own technical intuitions over the technical intuitions of this or that more-senior researcher.
Overly dismissing old-guard researchers is obviously a way the field can fail as well. But the field won’t advance much at all if most people don’t at least try to build their own models.
Koen also leans more on deference in his comment than I’d like, so I upvoted your ‘deferential but in the opposite direction’ comment as a corrective, handoflixue. :P But I think it would be a much better comment if it didn’t conflate epistemic authority with “fame” (I don’t think fame is at all a reliable guide to epistemic ability here), and if it didn’t equate “appealing to your own guesses” with “anti-vaxxers”.
Alignment is a young field; “anti-vaxxer” is a term you throw at people after vaccines have existed for 200 years, not a term you throw at the very first skeptical researchers arguing about vaccines in 1800. Even if the skeptics are obviously and decisively wrong at an early date (which indeed not-infrequently happens in science!), it’s not the right way to establish the culture for those first scientific debates.