advertisement
advertisement

Facebook’s AI for identifying hate speech seems to be working

Facebook says its automated detection systems contributed to a 134% increase in hate speech removals during the second quarter. But the numbers imply that the company has been oblivious to a significant amount of hate on its platform in the past.

Facebook’s AI for identifying hate speech seems to be working
[Source images: Fidan/iStock; Nao Triponez/Pexels]

Facebook reported a huge, 134% increase in the number of hate posts it removed from its primary social media platform from the first quarter to the second quarter. The company says in a new Community Standards Enforcement Report that it took action on 9.6 million hateful posts in the first quarter and 22.5 million in the second.

advertisement
advertisement

Facebook VP of Integrity Guy Rosen said during a media call that the main reason for the uptick is an improvement to the AI detection systems Facebook is relying on to remove hateful content. Facebook also said it’s expanded some of its automation technology to cover Spanish, Arabic, and Indonesian posts.

Something similar happened on Instagram. Facebook says its AI systems detected 84% of (known) hate speech on that service, after detecting just 45% of the hate speech in the first quarter of the year. The company says it removed 3.3 million hateful Instagram posts in the second quarter, versus just 808,900 in the first quarter. “These increases were driven by expanding our proactive detection technologies in English and Spanish,” the company said in a statement.

The numbers suggest real progress in the improvements to Facebook’s AI systems, on which it is pinning much of its hope for controlling a large and evolving hate and misinformation problem on its platforms. But the substantial increases in removals also suggest that Facebook may have been—and indeed may still be—simply missing significant amounts of hate content on its platforms. An alternate reason for the large uptick in hate speech removals is that there may have been a general increase in hateful content as the coronavirus spread. There is some data supporting a rise in xenophobia toward Chinese people in the U.S.

Mainly, the company finds toxic posts via its AI and user reports. It has no way of knowing exactly how much hate survives undetected.

It can, however, use data science to make estimates, as Rosen explains on Twitter:

advertisement

“Prevalence” refers to Facebook’s estimate of how many times its members are likely to see a specific kind of harmful content before its detected, or if it isn’t detected at all. The estimate is derived from taking samples of the content views of sample groups of Facebook users. But Facebook has yet to produce prevalence numbers for hate posts and several other categories of harmful content.

Facebook said that because of coronavirus it had to send many of its content moderation contract workers home during the second quarter. Some kinds of harmful content such as hateful or misleading videos could not be reviewed from home because of their graphic nature, Rosen said. He said Facebook has been relying on its AI systems to prioritize the most dangerous kinds of toxic content—such as violence in live video—for review by the human content moderators who are available.

Facebook also updated its policies to call out a couple of specific kinds of hate speech. It’s now banned content that depicts people in blackface (but not news reports about blackface), and content promoting the stereotype of “Jewish people controlling the world.”

Misinformation is not covered in the Community Standards Enforcement Report, because Facebook has no community standard addressing misinformation, but the company provided some stats on its actions against such content between April and June. It applied warning labels to 98 million pieces of COVID-19-related misinformation, which includes any posts about fake preventative measures and cures, and removed more than 7 million pieces of such misinformation.

advertisement
advertisement

About the author

Fast Company Senior Writer Mark Sullivan covers emerging technology, politics, artificial intelligence, large tech companies, and misinformation. An award-winning San Francisco-based journalist, Sullivan's work has appeared in Wired, Al Jazeera, CNN, ABC News, CNET, and many others.

More