B. They lose sight of the terminal goal. The real goal is not to skill-up in ML. The real goal is not to replicate the results of a paper. The real goal is not even to “solve inner alignment.” The real goal is to not die & not lose the value of the far-future.
I’d argue that if they solved inner alignment totally, then the rest of the alignment problems becomes far easier if not trivial to solve.
But solving inner alignment may not be the easiest way to drive down P(doom), and not the best way for a given person specifically to drive down P(doom), so keeping your eyes on the prize and being ready to pivot to a better project is valuable even if your current project’s success would save the world.
I’d argue that if they solved inner alignment totally, then the rest of the alignment problems becomes far easier if not trivial to solve.
But solving inner alignment may not be the easiest way to drive down P(doom), and not the best way for a given person specifically to drive down P(doom), so keeping your eyes on the prize and being ready to pivot to a better project is valuable even if your current project’s success would save the world.