My Fair AI: Accuracy Parity and Strategic Identity Disclosure in AI Content Recommendation

Digital platforms widely use artificial intelligence (AI) systems to recommend content such as videos, music, and news to consumers. However, certain consumer groups, such as niche segments, systematically receive undesirable content and inferior service because of AI bias. To rectify such performance disparities, firms consider committing to an algorithmic performance policy that ensures consistent consumer experiences across demographic groups. However, firms often rely on consumers’ voluntary identity disclosure to enforce such a policy. We develop a game-theoretic model that captures consumers’ strategic identity disclosure decisions alongside a firm’s recommendation and AI learning investment decisions. We provide a formal economic conceptualization of an accuracy parity policy and examine how it interacts with strategic consumers. We then analyze how this interaction affects the firm, its AI learning incentives, and consumer outcomes across demographic groups. Our model reveals when and why commitment to accuracy parity can unexpectedly increase a firm’s investment in AI development and benefit consumers in the dominant demographic group. At the same time, we show that such a policy can unintentionally harm the very groups it aims to protect. Our findings provide insights into the interaction between AI algorithmic constraints, strategic consumer behavior, and firm incentives in AI recommendation systems.

History: Anthony Dukes served as the senior editor.

Funding: This research was supported in part by the University of Kentucky CURATE Research Support Program.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/mksc.2024.0900.