Sometimes there is Hassrede (language of hatred) online recognizable at first glance, such as when there are insults and threats on social media, such as “shut up,” “# delete,” or “you should shoot them all”. But how to use artificial intelligence (AI) to identify more hidden forms of linguistic violence? engaged in it A study by researchers in Bochum and Berlin. It was published in the Journal of Open Humanities Data (JOHD).
Computer programs are already being used to identify and, where possible, eliminate hate speech on social media. “This makes it easy to recognize direct insults and profanity,” said Tatjana Scheffler, a linguist and professor at Bochum. in his university report. A comparison with a list of words is often sufficient. AI algorithms would be taught with data sets and could translate what they have learned into new data.
Harmful language is not just about insults and threats
However, there are other less recognizable forms. “You can also do harm by talking about others in a certain way or by creating a certain mood,” says Tatiana Scheffler. It can even turn into real action. Among other things, social media was held accountable for the fact that supporters of then-US President Donald Trump stormed the Washington Capitol on January 6, 2021.
Tatjana Scheffler’s team examined 26,431 messages from 521 Telegram Messenger users. They have been published since 2016. December to 2021 month of January. a channel where people with extreme right-wing moods exchange ideas. Specific plans to storm the Capitol stemmed from the theoretical idea of overthrowing the government.
One-fifth of the reports were analyzed by hand. In addition to Tatjana Scheffler, the team also included Veronika Solopova from the Dahlem Center for Machine Learning and Robotics at the Freie Universität Berlin and Mihaela Popa-Wyatt from the Leibniz Center for General Linguistics in Berlin. The researchers compared their results with the results of automated processes that companies use in hate speech or offensive language.
Hate speech is often perceived differently by people and machines
The categories examined included direct insults, such as ‘syringe’ or ‘retarded’, and incitement to violence. There were also expressions that were downplayed in the context, such as “they are a disease” (“they are a disease”). Another category was related to the so-called different – comments used to distinguish one group of people from another, for example (translated): “Are women not included in this conversation? If not, why not?
According to the report, 4505 notifications were included in the comparison. 3395 of them were classified as harmless by both scientists and automated processes. 275 they agreed that they contained harmful languages. 835 posts rated man and machine differently. “About half misclassified algorithms as hate speech or slander; Unlike scientists, they did not recognize others as harmful language, ”the conclusion said.
When it comes to incendiary comments, inside information, and so-called other, automated procedures were often flawed. “When we see cases where established methods make mistakes, it helps to improve the algorithms of the future,” sums up Tatiana Scheffler. With her team, she is also developing automated processes that should recognize malicious language even better. Linguistic insights also arise from this: “Certain grammatical structures may indicate that a term means derogatory,” Scheffler explains.
The automatic tool alone cannot work
It gives an example of the term ‘couple’. This can be quite harmless when using recipes. It becomes a swear word when you say “You’re a couple!” Contextual information can also help identify: “Which person commented? Has she made contemptuous remarks about others in the past? Who is being approached – a politician or a journalist? ”All of this can increase the AI’s hit rate.
But Tatiana Scheffler is also certain: “In addition to human experience …