Longitudinal Impact of Preference Biases on Recommender Systems’ Performance

Research studies have shown that recommender systems’ predictions that are observed by users can cause biases in users’ postconsumption preference ratings. This can happen as part of the standard, normal system use, where biases are typically caused by the system’s inherent prediction errors (i.e., because of the less-than-perfect accuracy of recommendation methods). Because users’ preference ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the performance of the system in the long run. We use a simulation approach to study the longitudinal impact of preference biases (and their magnitude) on the dynamics of recommender systems’ performance. Our simulation results show that preference biases significantly impair the system’s prediction performance (i.e., prediction accuracy) as well as users’ consumption outcomes (i.e., consumption relevance and diversity) over time. The impact is nonlinear to the size of the bias, that is, large bias causes disproportionately large negative effects. Also, items that are less popular and less distinctive (in terms of their content) are affected more by preference biases. Furthermore, given the impact of preference bias on the recommender systems’ performance, we explore the problem of debiasing user-submitted ratings. We empirically demonstrate that relying solely on historical rating data is unlikely to be effective in debiasing. We also propose and evaluate two debiasing approaches that take into account additional relevant information that can be collected by recommendation platforms. Our findings provide important implications for the design of recommender systems.

History: Olivia Liu Sheng, Senior Editor; Huimin Zhao, Associate Editor.

Supplemental Material: The e-companion is available at https://doi.org/10.1287/isre.2021.0133.