Combating Violent Extremism: The Need to Develop a Threat Information Sharing Strategy for Social Media and Gaming Companies

Key Chapter Title List

1. Background
2. Domestic Violent Extremists' Use of Online Platforms for Radical Mobilization, Recruitment, and Incitement
3. Various Tools Reported by Selected Companies to Counter Content Related to Domestic Violent Extremism
4. FBI and DHS Have Not Developed Information Sharing Strategies with Social Media and Gaming Companies
5. Agency Comments
6. Conclusion
7. Recommendations for Executive Action
8. Experts and Their Affiliations Participating in GAO Interviews
9. FBI and DHS Perspectives on Addressing Online Domestic Violent Extremist Content
10. Online Platforms' Efforts to Prevent and Counter Violent Extremism
11. Department of Homeland Security Comments
12. GAO Contact and Staff Acknowledgments

Document Introduction

In recent years, content promoting domestic violent extremism on social media and gaming platforms has influenced several high-profile attacks, becoming one of the significant terrorism threats facing the United States. In this context, the U.S. Government Accountability Office (GAO), at the request of Congress, conducted a specialized study aimed at comprehensively examining how domestic violent extremists utilize such platforms, the countermeasures taken by companies, and the information-sharing mechanisms among relevant federal agencies.

This report focuses on three core issues: first, to clarify the primary purposes for which domestic violent extremists use social media and gaming platforms, including expanding their audience, mainstreaming extremist ideologies, radical mobilization, recruitment, and incitement; second, to elaborate on the governance tools and practices adopted by selected social media and gaming companies to combat content promoting domestic violent extremism, covering terms of service, machine learning tools, user reporting, human moderation, and platform design optimization; third, to assess the development of strategies and goals by the Federal Bureau of Investigation (FBI) and the Department of Homeland Security (DHS) regarding sharing relevant threat information with companies.

To complete the study, GAO reviewed relevant documents from the FBI and DHS, interviewed agency officials, and, through institutions like the National Academies, selected 16 experts in related fields for in-depth interviews. Additionally, five representative social media and gaming companies (including Discord, Reddit, Roblox, etc.) were selected to obtain firsthand information. The research process strictly adhered to government auditing standards to ensure the sufficiency and appropriateness of the evidence.

The study found that domestic violent extremists flexibly choose different platforms based on their functional characteristics, user demographics, and content moderation policies to achieve their extremist goals, often evading content moderation through coded language and cross-platform migration. Although companies have implemented various governance measures, their effectiveness is limited by factors such as leadership preferences, financial considerations, and limitations of moderation technologies. More critically, while the FBI and DHS have established some information-sharing mechanisms, neither has developed systematic strategies or clear goals, preventing a full assessment of the effectiveness of information sharing and its support for the agencies' core missions.

Based on these findings, GAO proposes two core recommendations: the FBI and DHS should respectively develop strategies and goals for sharing information related to domestic violent extremism with social media and gaming companies. DHS and the Department of Justice have agreed with these recommendations, with DHS planning to complete the development of its relevant strategy by June 2024. This report provides authoritative empirical evidence and a feasible path for congressional oversight, federal agency policy optimization, and corporate risk prevention and control.