Invisible Rulers

36 Some of Twitter’s policies regularly intersected with active fronts in the culture war, turning the policies themselves into a matter of controversy as well as an opportunity to capture attention. For example, it had a long-standing rule that prohibited harassing individual transgender users (such as by referring to them with prior names or pronouns) while still allowing criticism of trans-related policies and laws or identity politics writ large. This attempt to minimize the harassment of individual users, however, was often framed as inherently anticonservative; some argued that it forced conservatives to use pronouns that they disagreed with, which impinged upon their freedom of speech. The targets of the speech, however, were also users who wanted to express themselves on Twitter as well. As with many moderation decisions, calls sometimes hinged on whether individual moderators felt that a word or phrase was harassment, merely offensive, or an overreaction by the reportee. More importantly, however, the policy became a source of secondary attention-capture for clout and profit, as those who alleged that they were moderated unfairly—or felt they went unheard—subsequently commanded attention cycles highlighting the incidents.

37 Jeremy Boreing (@JeremyDBoreing), “I appreciate the reply. We posted the two clips flagged by Twitter and they were indeed labeled ‘hateful conduct’ and the share functions were disabled on the…,” Twitter, Juny 1, 2023, https://twitter.com/JeremyDBoreing/status/1664332765226057730?s=20.

38 Daysia Tolentino and David Ingram, “Musk’s response to an anti-trans video sparks 24 hours of chaos at Twitter,” NBC News, June 3, 2023, https://www.nbcnews.com/tech/social-media/musk-elon-twitter-ella-irwin-trans-video-what-is-a-woman-stream-rcna87429.

39 Robert B. Cialdini, Influence: Science and Practice, 5th ed. (1984; reis., Boston: Pearson, 2009), 210–211.

40 “Community Principles,” TikTok, last updated March 2023, https://www.tiktok.com/community-guidelines/en/community-principles.

41 “Ensuring Respect for Free Expression, Through Independent Judgement,” Oversight Board, https://www.oversightboard.com.

42 For example, Twitter and Facebook have programs to ensure that moderation actions against high-profile users are checked by multiple reviewers, including at times the executive team. Preliminary indications from some of our data in the 2020 election suggested that lower-follower accounts were actioned for election misinformation even as influencers with far more reach did not appear to be. This is perhaps a natural defensive stance for Big Tech platforms, but it preferentially favors those with large audiences or offline power. Renée DiResta and Matt Debutts, “‘Newsworthiness,’ Trump, and the Facebook Oversight Board,” Columbia Journalism Review, April 26, 2021, https://www.cjr.org/the_new_gatekeepers/facebook-oversight-board-2.php.

43 For a nuanced explanation of how content moderation work is done and the mental health impacts it has, see Casey Newton, “The Trauma Floor: The Secret Lives of Facebook Moderators in America,” The Verge, February 25, 2019, https://www.theverge.com/2019/2/25/18229714/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona.

44 “Singapore Tightens the Reins on Extreme Social Media Content,” Japan Times, November 12, 2022, https://www.japantimes.co.jp/life/2022/11/12/general/social-media-content-law-singapore.

45 Yashraj Sharma, “Twitter Accused of Censorship in India as It Blocks Modi Critics,” The Guardian, April 5, 2023, https://www.theguardian.com/world/2023/apr/05/twitter-accused-of-censorship-in-india-as-it-blocks-modi-critics-elon-musk.

46 Charlie Savage, “Trump’s Order Targeting Social Media Sites, Explained,” New York Times, May 28, 2020, https://www.nytimes.com/2020/05/28/us/politics/trump-twitter-explained.html.

47 David Shepardson, “Biden Revokes Trump Order That Sought to Limit Social Media Firms’ Protections,” Reuters, May 17, 2021, https://www.reuters.com/technology/biden-revokes-trump-order-that-sought-limit-social-media-firms-protections-2021-05-15.

48 Fletcher Schoen and Christopher J. Lamb, “Deception, Disinformation, and Strategic Communications: How One Interagency Group Made a Major Difference,” Institute for National Strategic Studies, June 1, 2012, https://inss.ndu.edu/Media/News/Article/693590/deception-disinformation-and-strategic-communications-how-one-interagency-group.

49 Fred Barbash, “U.S. Ties ‘Klan’ Olympic Hate Mail to KGB,” Washington Post, August 7, 1984, https://www.washingtonpost.com/archive/politics/1984/08/07/us-ties-klan-olympic-hate-mail-to-kgb/80918fe8-fcf0-46cf-bb58-726ee46d8ce9.

50 Michael Dhunjishah, “Countering Propaganda and Disinformation: Bring Back the Active Measures Working Group?,” War Room, July 7, 2017, https://warroom.armywarcollege.edu/articles/countering-propaganda-disinformation-bring-back-active-measures-working-group.