The situation where AI is a good tool for manipulating public opinion, and the leading AI company has a bad reputation, seems unstable. Maybe AI just needs to get a little better, and then AI-written arguments in favor of AI will win public opinion decisively? This could “lock in” our trajectory even worse than now, and could happen long before AGI.
This fail-state is particularly worrying to me, although it is not obvious whether there is enough time for such an effect to actually intervene on the future outcome.
I worry about that, and I worry about the first AI-based consumer products that are really fun (like video games are) because pleasure is the motivation in most motivated cognition (at least in Western liberal societies).
The situation where AI is a good tool for manipulating public opinion, and the leading AI company has a bad reputation, seems unstable. Maybe AI just needs to get a little better, and then AI-written arguments in favor of AI will win public opinion decisively? This could “lock in” our trajectory even worse than now, and could happen long before AGI.
This fail-state is particularly worrying to me, although it is not obvious whether there is enough time for such an effect to actually intervene on the future outcome.
I worry about that, and I worry about the first AI-based consumer products that are really fun (like video games are) because pleasure is the motivation in most motivated cognition (at least in Western liberal societies).