Navigating the complexities of socio-linguistic communication on social media is challenging, especially when platforms rely on automated systems to detect and address abusive language. Recent research highlights the limitations of machine learning algorithms in identifying hate speech, particularly when it involves implicit or reclaimed language. These systems often struggle with content targeting individuals based on intersecting identity characteristics such as gender, race, ethnicity, or disability.
To address these challenges, developers are encouraged to incorporate diverse perspectives into the training data and methods used for developing hate-speech detection algorithms. This is particularly important in the context of US-centered social media analysis due to a rise in potentially abusive content associated with national elections. A recent report by CDT found that women of color political candidates faced disproportionate amounts of online hate speech leading up to the 2024 elections.
The study used detection algorithms to count posts containing hate speech directed at candidates but likely understated the problem due to algorithmic flaws. An earlier CDT report noted that women of color experience more violent online harassment when targeted as political candidates. Such hateful posts contribute to a chilling effect that prompts self-censorship among those targeted.
A lack of transparency around platform policies and moderation practices adds to these challenges. Major platforms like Instagram, Facebook, and X use machine-learning algorithms for content moderation. Natural Language Processing (NLP) plays a crucial role in analyzing text, with Large Language Models (LLMs) being effective for processing large datasets.
However, biases in model training can lead systems to incorrectly flag posts or misidentify targets of hate speech. Research shows that detection often misidentifies dialects or group-specific language such as African American English or LGBTQ+ vernacular. Automated moderation tends to disadvantage marginalized groups; for example, Meta's algorithms reportedly detect more hate speech targeting white people than Black people on Instagram.
Studies indicate only 17% of posts containing "misogynoir" were correctly classified by popular detection algorithms on X. Transgender Facebook users also face higher levels of content removal when posting about their identities.
Expanding transparent and community-driven research is essential for improving AI tools like automated moderation and reducing bias in detection models. Group-specific approaches focus on understanding how models respond to community-specific language use.
Improving access to datasets is another crucial step. A review found that only 51% of datasets used in published articles are publicly available, with less than 35% mentioning methodology. Tools cataloging hate speech datasets help crowdsource research opportunities across disciplines.
Expanding diverse research efforts and increasing dataset access are vital for developing fairer AI tools and mitigating adverse effects of current moderation practices. Interdisciplinary collaboration enhances data collection quality and leads to more representative training models with stronger ethical considerations.
