Garrett Baker comments on Current AIs Provide Nearly No Data Relevant to AGI Alignment

[Garrett Baker]

So I restate my final paragraph:

Maybe you don’t think this is autonomous enough for you. After all its people writing the paper, people who will come up with the ideas of what to use the materials for, and people who built this very particular ML setup in the first place. But then your prediction becomes these tasks will not be automateable by LLMs without making them dangerous. To me these tasks seem pretty basic, likely beyond current LLM abilities, but GPT-5 or 6? Not out of the question given no major architecture or training changes.