I think the optimistic case might be that in order to get the AGI to do anything useful at all you have to get at least part-way to a solution to the alignment problem, because otherwise its outputs will include many that will be so obviously “wrong” that you’d never actually let it do anything in which being wrong mattered.
I think the optimistic case might be that in order to get the AGI to do anything useful at all you have to get at least part-way to a solution to the alignment problem, because otherwise its outputs will include many that will be so obviously “wrong” that you’d never actually let it do anything in which being wrong mattered.
Unfortunately humanity will not get partial credit for almost avoiding extinction.
There will be no partial credit on Humanity’s AI Alignment Exam. I like that!