The problem is that normal people very often give up collective resources to look good. They just don’t give up their personal resources. For the AI, the former is sufficient.
The argument is that the AI will be able to create fanatics, a claim you should update in favor of given the known “success” of the YouTube algorithm at passing relaxed versions of this benchmark, and the probable “success” of many other arguments on the internet which radicalize portions of the population.
As epistemic learned helplessness is a thing, this will not actually work on most people.
Furthermore, your idea that fanatics can be convinced to give up resources pretty much requires fanatics. Normal people won’t behave this way.
The problem is that normal people very often give up collective resources to look good. They just don’t give up their personal resources. For the AI, the former is sufficient.
The scenario requires not only that they give them up, but that they give them up on a very immediate basis, which is less likely.
The argument is that the AI will be able to create fanatics, a claim you should update in favor of given the known “success” of the YouTube algorithm at passing relaxed versions of this benchmark, and the probable “success” of many other arguments on the internet which radicalize portions of the population.