Machine learning methods can often have good accuracy at population level, but fail spectacularly on specific instances, and if the instance-level outputs are visible to the public, these failures may be quite embarrassing: imagine if somebody posted a quote by Churchill or a passage from the Qur’an and it was mistakenly tagged as trolling.
Even if the machine learning system was not misclassifying, it can never be better than the data it is trained on. If you train it on user ratings, it will turn into a popularity contest, with unpopular opinions being tagged as trolling, especially once the politically-motivated users figure out how to manipulate the system by voting strategically. If it is based on a dataset curated by the company it would be perceived as the company exerting ideological and political bias in a subtle, opaque manner.
In general it would be seen as a powerful institution putting an official seal of approval/mark of shame on speech and people. It reeks of totalitarianism.
imagine if somebody posted a quote by Churchill or a passage from the Qur’an and it was mistakenly tagged as trolling.
If the trolling criteria include “racism” or “threats of violence” (as in the OP), I think both of these sources would be correctly matched by the software. (Which is not to say I think we should censor them.)
If the criteria include the generous “language [which] is offensive” (also in the OP), I think most language ever written would turn out to be offensive to someone.
If the trolling criteria include “racism” or “threats of violence” (as in the OP), I think both of these sources would be correctly matched by the software. (Which is not to say I think we should censor them.)
My suggestion was not to train the system on user ratings:
The first is to let a number of sensible people give their troll scores of different Facebook posts and tweets (using the general and vague definition of what is to count as trolling). You would feed this into your algorithms, which would learn which combinations of words are characteristic of trolls (as judged by these people), and which arent’t. The second is to simply list a number of words or phrases which would count as characteristic of trolls, in the sense of the general and vague definition.
So, essentially it would depend on the company opinion.
Anyway, lists of words or short phrases won’t work. Keep in mind that trolls are human intelligences, any AI short of Turing-test level won’t beat human intelligences at their own game.
Machine learning methods can often have good accuracy at population level, but fail spectacularly on specific instances, and if the instance-level outputs are visible to the public, these failures may be quite embarrassing: imagine if somebody posted a quote by Churchill or a passage from the Qur’an and it was mistakenly tagged as trolling.
Even if the machine learning system was not misclassifying, it can never be better than the data it is trained on.
If you train it on user ratings, it will turn into a popularity contest, with unpopular opinions being tagged as trolling, especially once the politically-motivated users figure out how to manipulate the system by voting strategically.
If it is based on a dataset curated by the company it would be perceived as the company exerting ideological and political bias in a subtle, opaque manner.
In general it would be seen as a powerful institution putting an official seal of approval/mark of shame on speech and people. It reeks of totalitarianism.
If the trolling criteria include “racism” or “threats of violence” (as in the OP), I think both of these sources would be correctly matched by the software. (Which is not to say I think we should censor them.)
If the criteria include the generous “language [which] is offensive” (also in the OP), I think most language ever written would turn out to be offensive to someone.
Indeed.
My suggestion was not to train the system on user ratings:
So, essentially it would depend on the company opinion.
Anyway, lists of words or short phrases won’t work. Keep in mind that trolls are human intelligences, any AI short of Turing-test level won’t beat human intelligences at their own game.