Time and Again, Social Media Giants Get Content Moderation Wrong: Silencing Speech about Al-Aqsa Mosque is Just the Latest Example
Since May 7, Al-Aqsa, one of the holiest sites for Muslims, and the neighborhood of Sheik Jarrah in Jerusalem have been the site of violent attacks against Palestinians, many of whom had come to the mosque to worship during Ramadan. During this violence, people took to Facebook and Instagram to post about what was happening to Palestinians using hashtags that referenced the mosque (#AlAqsa). But Instagram, owned by Facebook, many of the posts through its content-filtering system because it inaccurately identified the hashtag as referencing a terrorist organization. While this may have been an error — though a demeanning and culturally ignorant one —its impact on users’ expression and the flow of information cannot be ignored. Palestinians and their supporters were silenced by one of the most powerful communications platforms in human history at a critical moment.
This wasn’t an isolated incident. In today’s world, a small handful of private corporations make some of the most important decisions regarding our online speech. Companies like Facebook, Twitter, and YouTube set rules for what kinds of content they’ll allow people to publish, defining what constitutes “hate speech,” “praise of terrorism,” and “fake news.” And they rely on algorithms to automatically flag certain words or images that appear to cross the lines. Once posts are removed or accounts are suspended, users — particularly those who don’t make headlines and cannot access backchannels — have too little recourse to appeal or get reinstated.
The major social media companies often get content moderation wrong — both because of their vague and sweeping rules, and because they make mistakes when applying those rules, often through blunt automated detection systems. Perfect content moderation may be impossible, but the major platforms can do better. They should give users more control, and respond to user experiences and reports, rather than rely so heavily on automated detection. They should also provide transparency, clarity, and appeals processes to regular users, and not just those that make headlines or have a personal connection at the company.
Here’s a rundown of some recent examples of content moderation gone wrong.
What Qualifies as Praise of “Terrorist” Groups?
Instagram’s ban on #AlAqsa is not the first time that the social media giants have misapplied their rules regarding praise or support of “terrorist” content. Over the summer, dozens of Tunisian, Syrian, and Palestinian activists and journalists covering human rights abuses and civilian airstrikes that Facebook had pursuant to its policy against “praise, support, or representation of” terrorist groups. Facebook deleted at least 52 accounts belonging to Palestinian journalists and activists in a single day in May, and more than 60 belonging to Tunisian journalists and activists that same month.
Relatedly, in October 2020, , all refused to host San Francisco State University’s roundtable on Palestinian rights, “Whose Narratives? Gender, Justice and Resistance”—featuring Leila Khaled, a member of the Popular Front for the Liberation of Palestine. The companies decided to censor the roundtable after it became a target of a coordinated campaign by pro-Israel groups that disagree with Khaled’s political views. In this instance, the companies pointed to anti-terrorism laws, rather than their own policies, as justification.
But these decisions, too, highlight platforms’ role in curtailing speech in an increasingly online world — not to mention problems with the underlying material support laws. One fundamental problem is that governments—not to mention the social media giants—do not have an agreed upon and transparent definition for terms like “terrorism,” “violent extremism,” or “extremism,” let alone “support” for them. As a result, rules regulating such content can be highly subjective and open the door to biased enforcement.
“Hate” Against Whom?
For years, Facebook moderators have treated the speech of women and people of color — including specifically when describing their experiences with sexual harassment and racism — differently than that of men and white people, pursuant to their community standards regarding hateful speech.
In 2017, when Black women and white people posted the exact same content, Facebook .
Notwithstanding tweaks to their community standards and algorithms, that problem persists. This year, for example, Facebook removed posts in groups created by users as spaces to vent about sexism and racism for .” Meanwhile, “posts disparaging or even threatening women” stayed up. The company banned phrases like “men are trash,” “men are scum,” and even “I dislike men” — but posts like “women are trash” and “women are scum” were not removed, even if users reported the posts.
Similarly, in June 2020, at the height of protests against the police murder of George Floyd, Facebook’s automated system removed derogatory terms like more often than slurs against Jewish, Black, and transgender individuals. Even after Facebook attempted to tweak its algorithmic enforcement to address this, Instagram still removed a post calling on others to “thank a Black woman for saving our country,” pursuant to the company’s “hate speech” guidelines.
As and the reported, these policies and applications have forced users to avoid using certain words, swapping out “m3n” for “men” and “wipipo” for “white.”
What is “Sexually Suggestive” and What is Socially Acceptable?
Similar problems have arisen with enforcement of the companies’ policies regarding “sexually suggestive” content and posts regarding “sexual solicitation.”
In 2019, after Instagram repeatedly took down a topless photo of Nyome Nicholas-Williams, a fat Black woman, in which her arms covered her breasts. The users noted that nude images of skinny white women were not subjected to the same treatment and less likely to be considered inherently sexual.
More recently, Facebook , an independent band whose members are a same-sex couple, as “sexually explicit” because it pictured the two women with their foreheads touching. As an experiment, Unsung Lilly uploaded the same ad with two different photos, one with a “nonromantic” photo of themselves and the other of a heterosexual couple touching foreheads. Both were approved by Facebook.
Along with queer users, sex workers, certified sex educators, and sexual wellness brands are also regarding “sexual solicitation.” These users report that posts including “flagged words, like ‘sex’ and ‘clitoris,’ have been removed from Instagram’s search function.”
Censorship Decisions are Arbitrary
Social media giants silence speech in arbitrary ways. For example, Facebook recently banned a recent college graduate for three days after he posted and deeming America the “land of ignorance and greed.” The company also drew seemingly random regarding absentee voting, poll-watching, and COVID-19 during the 2020 presidential election, removing such posts as “If you vote by mail, you will get Covid!” and “My friends and I will be doing our own monitoring of the polls to make sure only the right people vote,” but permitting “If you vote by mail, be careful, you might catch Covid-19!” and “I heard people are disrupting going to the polls today. I’m not going to the polling station.”
A Path Forward
These examples highlight ongoing problems with the social media giants’ content moderation policies and practices — specifically, a lack of clarity in community standards, policies that cover too much speech, a disconnect between automated systems and actual user reports, and insufficient access to appeals. These problems have significant impacts in the offline world, altering discourse and limiting access to potentially critical information.
For these reasons, if they are to err, the major social media companies must err on the side of enabling expression. They should establish content policies that are viewpoint neutral, and that favor user control over censorship. And they should ensure that any expansion of content policies comes with a commensurate expansion of due process protections — clear rules, transparent processes, and the option of appeals — for all users. Because of the scale at which these platforms function, errors are inevitable, but there can be fewer and even those that still occur need not be permanent.
Learn More About the Issues on This Page
-
Press ReleaseDec 2024
Free Speech
National Security
ACLU and Partners Urge Supreme Court to Block TikTok BanACLU and Partners Urge Supreme Court to Block TikTok Ban
WASHINGTON — Today, the American Civil Liberties Union, the Electronic Frontier Foundation (EFF), and the Knight First Amendment Institute at Columbia University filed an amicus brief urging the Supreme Court to block the enforcement of a law that would effectively ban people in America from using TikTok as soon as January 19, 2025. Earlier this month, the U.S. Court of Appeals for the D.C. Circuit rejected TikTok’s challenge to the law. TikTok has asked the Supreme Court for a temporary injunction to prevent the app from being banned while the court considers whether to take the case, saying that unless the justices intervene, the law will “shutter one of America’s most popular speech platforms the day before a presidential inauguration.” “The Constitution imposes an extraordinarily high bar on this kind of mass censorship,”said Patrick Toomey, deputy director of ACLU’s National Security Project. “The Supreme Court should take up this important case and protect the rights of millions of Americans to freely express themselves and engage with others around the world.” The brief argues that the D.C. Circuit failed to fully address the law’s profound implications for the First Amendment rights of the 170 million Americans who use TikTok. While the lower court’s decision correctly recognized that the statute triggers First Amendment scrutiny, it barely addressed users’ First Amendment interests in speaking, sharing, and receiving information on the platform. The court also perplexingly attempted to cast the government’s ban on TikTok as a vindication of users’ First Amendment rights, which it is not. The rights groups also explain that the law was intended to suppress certain content and viewpoints that many legislators believe could be amplified on TikTok, including the risk of foreign “propaganda.” But under the First Amendment, the government must meet a very high bar to restrict speech based on concerns about its “motivating ideology” or “perspective,” and the government has not come close to meeting that bar here. “The government should not be able to restrict speech, especially to the extent here, based on guessing about the mere possibility of uncertain future harm,” said David Greene, civil liberties director at EFF. “The Supreme Court should put the TikTok ban on hold while it considers the DC Circuit’s erroneous ruling.” Finally, the brief underscores that the government can’t impose this type of sweeping ban unless it’s necessary to prevent extremely serious and imminent harm to national security. But the government has not provided evidence of impending harm, or evidence that banning TikTok is the only available way to address its concerns. As the brief explains, the D.C. Circuit improperly treated the government’s invocation of “national security” as a trump card and failed to hold the government to its burden. “Restricting citizens’ access to foreign media is a practice that has long been associated with repressive regimes, and we should be very wary of letting the practice take root here,” said Jameel Jaffer, executive director at the Knight First Amendment Institute at Columbia University. “It would do lasting damage to the First Amendment and our democracy if the Supreme Court let this ban go into effect even temporarily.” You can find the brief online here: https://www.supremecourt.gov/DocketPDF/24/24A587/335380/20241217144322392_24A587%20TikTok%20v%20Garland%20Amicus%20Brief%20pdfa.pdf -
MississippiJul 2024
Free Speech
Privacy & Technology
NetChoice v. FitchNetChoice v. Fitch
Whether a law that mandates age verification for all users of social media services violates the First Amendment. -
News & CommentaryDec 2024
Free Speech
Can you define pornography? Neither can the government.Can you define pornography? Neither can the government.
We break down the absurd-but-true history of trying to censor sexual speech under the guise of "keeping kids safe." -
Minnesota Supreme CourtNov 2024
Free Speech
Energy Transfer LP v. Greenpeace International, Unicorn RiotEnergy Transfer LP v. Greenpeace International, Unicorn Riot
This case in the Minnesota Supreme Court asks whether the MFFIA's protections apply to newsgatherers even if they are alleged to have engaged in trespassing while newsgathering. The ACLU’s State Supreme Court Initiative, alongside the ACLU of Minnesota and law firm Biersdorf & Associations, filed a brief representing Unicorn Riot, a media organization that covered protests of the Dakota Access Pipeline. The brief argues that MFFIA and constitutional reporter’s privileges, under both the U.S. and Minnesota constitutions, protect Unicorn Riot from having to comply with Energy Transfer’s subpoenas.