FYI I think this is worth fleshing out into a top level post (esp. given that it’s ‘Pause Debate’ week).
I’m not actually sure it needs much fleshing out. I think the main bit here that feels unjustified, or insufficiently-justified for the strength of the claim, is:
That’s changed a bit lately, in part because a bunch of people seem to think that making technical progress on alignment is hopeless. I think this is just not an epistemically reasonable position to take: history is full of cases where people dramatically underestimated the growth of scientific knowledge, and its ability to solve big problems.
FYI I think this is worth fleshing out into a top level post (esp. given that it’s ‘Pause Debate’ week).
I’m not actually sure it needs much fleshing out. I think the main bit here that feels unjustified, or insufficiently-justified for the strength of the claim, is:
Have edited slightly to clarify that it was “leading experts” who dramatically underestimated it. I’m not really sure what else to say, though...