As one of the leading social media platforms, Meta witnesses a deluge of content being posted on its platforms every second. However, it’s important to acknowledge that not all of this content is guaranteed to be safe and needs to be moderated.
In November last year, Meta announced that it took action against a total of 23 million pieces of harmful content created by Indians across its social media apps Facebook and Instagram.
Filtering out harmful content on Meta’s apps has been a challenge for content moderators as they are continuously bombarded with disturbing and inappropriate content. Extreme visual content can include depictions or actual acts of gore or lethal violence, such as murder, suicide, violent extremism, animal abuse, hate speech, sexual abuse, child or revenge pornography and more.
BBC’s recent report highlighted that Sama, a company entrusted with the task of moderating Facebook posts in East Africa, has come to regret its decision in hindsight. Former employees based in Kenya have expressed distress due to exposure to distressing and graphic content.
To address content moderation problems faced by companies, OpenAI seems to have discovered a potential solution. In a recent blog post, OpenAI stated that businesses can utilize GPT-4 for content moderation purposes.
This is the first time when an LLM is being put to use for this purpose. OpenAI in their blog said that content moderation using GPT-4 results in much faster iteration on policy changes, reducing the cycle from months to hours. It further elaborates that GPT-4 is able to interpret rules and nuances in long content policy documentation and adapt instantly to policy updates.
As per OpenAI’s research findings, GPT-4 that underwent training for content moderation outperforms human moderators with basic training. However, both GPT-4 and humans fall short when compared to highly skilled and experienced human moderators.
“We believe this offers a more positive vision of the future of digital platforms, where AI can help moderate online traffic according to platform-specific policy and relieve the mental burden of a large number of human moderators,” write OpenAI authors Lilian Weng View, Vik Goel and Andrea Vallone.
Can Meta take inspiration from OpenAI?
Meta has adopted AI for content moderation but still the social media giant hasn’t explored yet using LLM for content moderation. It uses AI tools but still largely depends on outsource partners who hire employees for content moderation.
Hiring humans for content moderation has led to a lot of trouble for the social media platform as they are not able to cope up with the gory content. Meta has been trying to automate the content moderation process gradually and it would be nice if they can take inspiration from OpenAI.
OpenAI has taken a lead in the positive direction by exploring the use of GPT-4 for content moderation. OpenAI is not keeping this tool to themselves and anyone with OpenAI’s API access can implement this approach to create their own AI-assisted moderation system.
However, the prospect of Meta adopting the GPT-4 API to address their challenges seems improbable, given the competitive rivalry between the two companies in the realm of generative AI.
It would be nice if Meta can train Llama 2 for content moderation purposes similar to what OpenAI is doing and make it open source. It goes without saying Meta has more experience in content moderation as compared to OpenAI.
In December 2021, Meta came up with a new AI system using a method called “few-shot learning,” in which models start with a general understanding of many different topics and then use much fewer — or sometimes zero — labeled examples to learn new tasks.
Last year, Meta launched a tool called Hasher-Matcher-Actioner (HMA) to be adopted by a range of companies to help them stop the spread of terrorist content on their platforms. It is especially useful for smaller companies who don’t have the same resources as bigger ones. HMA is built on Meta’s previous open source image and video matching software that can be used to flag any type of violating content.
Meta believes in open sourcing tools. If Llama 2, equipped for content moderation, becomes accessible, it could prove invaluable for both companies and Meta itself. This tool would facilitate the removal of content that holds potential harm for consumers along with less dependence on human workforce for content moderation relieving them from the mental burden.
Notably, Meta has also expressed its ongoing efforts to label AI-generated content across its platforms lately.
