LLMs/GPTs get their capabilities not through directly pursuing instrumental convergence, but through mimicking humans who hopefully have pursued instrumental convergence (the whole “stochastic parrot” insight), so it’s unclear what “bad instrumental convergence” even looks like in LLMs/GPTs or what it means to erase it.
The closest thing I can see to answer the question is that LLMs sort of function as search engines and you want to prevent bad actors from gaining an advantage with those search engines so you want to censor stuff that is mostly helpful for bad activities.
They seem to have done quite well at that, so it seems basically feasible. Of course LLMs will still ordinarily empower bad actors just as they ordinarily empower everyone, so it’s not a full solution.
I don’t consider this very significant though as I have a hard time imagining that stochastic parrots will be the full extent of AI forever.
LLMs/GPTs get their capabilities not through directly pursuing instrumental convergence, but through mimicking humans who hopefully have pursued instrumental convergence (the whole “stochastic parrot” insight), so it’s unclear what “bad instrumental convergence” even looks like in LLMs/GPTs or what it means to erase it.
The closest thing I can see to answer the question is that LLMs sort of function as search engines and you want to prevent bad actors from gaining an advantage with those search engines so you want to censor stuff that is mostly helpful for bad activities.
They seem to have done quite well at that, so it seems basically feasible. Of course LLMs will still ordinarily empower bad actors just as they ordinarily empower everyone, so it’s not a full solution.
I don’t consider this very significant though as I have a hard time imagining that stochastic parrots will be the full extent of AI forever.