Content Moderation with Shadowbanning

The task of balancing online safety with preserving freedom of expression is increasingly challenging for social media platforms. We examine shadowbanning (making a user’s content hidden without their knowledge) and content removal (removing content with a user’s knowledge) to determine their relative impacts on market coverage, moderated content, platform profit, consumer surplus, and welfare. With shadowbanning, users assess a probability, or a belief, that a platform implements shadowbanning. We employ a stylized formulation from the literature where users receive utility from posting content and reading content and may face disutility from reading extreme content. We find that if common user beliefs that the platform implements shadowbanning are not too high, then the platform prefers to shadowban over no moderation. Content removal helps the platform expand the user base across users with a low to moderate degree of extremeness, whereas shadowbanning expands the user base across users with any degree of extremeness. Consequently, the platform implementing shadowbanning has a larger user base and higher profit than one using content removal. In addition, the platform can cover the market with shadowbanning; however, it cannot do so with content removal. Even with a larger user base, shadowbanning results in a larger volume of moderated content compared with content removal. Finally, if users hold moderate common beliefs about shadowbanning, then shadowbanning increases consumer surplus and social welfare over content removal or no moderation. Our main findings remain mostly consistent when user beliefs about shadowbanning implementation are heterogeneous. However, unlike with common beliefs, highly extreme users may drop out. Additionally, results from our extended models—shadowbanning with imperfect technology, shadowbanning with mandated content removal, and shadowbanning with updated beliefs—suggest that shadowbanning can be beneficial, although the policy implications depend on which metric is prioritized: market coverage, volume of moderated content, platform profit, consumer surplus, or social welfare.

History: Martin Bichler, Senior Editor; Abhijeet Ghoshal, Associate Editor.

Funding: This work was supported by the Social Sciences and Humanities Research Council of Canada [Grant 435-2024-0855].

Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2024.1140.