The Square Root Agreement Rule for Incentivizing Truthful Feedback on Online Platforms

References

• Aldous DJ (1985) Exchangeability and related topics. École d’Été de Probabilités de Saint-Flour XIII–1983 (Springer, Berlin), 1–198.Crossref, Google Scholar
• Brier GW (1950) Verification of forecasts expressed in terms of probability. Monthly Weather Rev. 78(1):1–3.Crossref, Google Scholar
• Cover T, Thomas J (2012) Elements of Information Theory (Wiley, New York).Google Scholar
• Dasgupta A, Ghosh A (2013) Crowdsourced judgement elicitation with endogenous proficiency. Baeza-Yates R, Moon S, eds. Proc. 22nd Internat. Conf. on World Wide Web (Association for Computing Machinery, New York), 319–330.Google Scholar
• Faltings B, Jurca R, Radanovic G (2017) Peer truth serum: Incentives for crowdsourcing measurements and opinions. Preprint, submitted April 18, https://arxiv.org/abs/1704.05269.Google Scholar
• Filippas A, Horton JJ, Golden J (2018) Reputation inflation. Elkind E, Vohra R, eds. Proc. ACM Conf. on Econom. and Comput. (Association for Computing Machinery, New York), 483–484.Google Scholar
• Gao XA, Wright JR, Leyton-Brown K (2019) Incentivizing evaluation with peer prediction and limited access to ground truth. Artificial Intelligence 275:618–638.Crossref, Google Scholar
• Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102(477):359–378.Crossref, Google Scholar
• Harsanyi JC, Selten R (1988) A General Theory of Equilibrium Selection in Games (MIT Press, Cambridge, MA).Google Scholar
• Hu N, Pavlou PA, Zhang JJ (2017) On self-selection biases in online product reviews. Management Inform. Systems Quart. 41(2):449–471.Crossref, Google Scholar
• Jøsang A, Ismail R, Boyd C (2007) A survey of trust and reputation systems for online service provision. Decision Support Systems 43(2):618–644.Crossref, Google Scholar
• Jurca R, Faltings B (2005) Enforcing truthful strategies in incentive compatible reputation mechanisms. Proc. Internat. Workshop on Internet and Network Econom. (Springer, Berlin), 268–277.Google Scholar
• Jurca R, Faltings B (2008) Incentives for expressing opinions in online polls. Riedl J, Sandholm T, eds. Proc. 9th ACM Conf. on Electronic Commerce (Association for Computing Machinery, New York), 119–128.Google Scholar
• Jurca R, Faltings B (2011) Incentives for answering hypothetical questions. Proc. Workshop on Social Comput. and User Generated Content.Google Scholar
• Kong Y (2020) Dominantly truthful multi-task peer prediction with a constant number of tasks. Proc. 14th Annual ACM-SIAM Sympos. on Discrete Algorithms (SIAM, Philadelphia), 2398–2411.Google Scholar
• Kong Y, Schoenebeck G (2019) An information theoretic framework for designing information elicitation mechanisms that reward truth-telling. ACM Trans. Econom. Comput. 7(1):1–33.Crossref, Google Scholar
• Li L, Tadelis S, Zhou X (2020) Buying reputation as a signal of quality: Evidence from an online marketplace. RAND J. Econom. 51(4):965–988.Crossref, Google Scholar
• Liu Y, Chen Y (2016) Learning to incentivize: Eliciting effort via output agreement. Kambhampati S, Mausam, Gabrilovich E, eds. Proc. 25th Internat. Joint Conf. on Artificial Intelligence (IJCAI, CA), 3782–3788.Google Scholar
• Luca M (2017) Designing online marketplaces: Trust and reputation mechanisms. Innovative Policy Econom. 17(1):77–93.Crossref, Google Scholar
• Miller N, Resnick P, Zeckhauser R (2005) Eliciting informative feedback: The peer-prediction method. Management Sci. 51(9):1359–1373.Link, Google Scholar
• Myerson RB (2013) Game Theory (Harvard University Press, Cambridge, MA).Crossref, Google Scholar
• Ni J, Li J, McAuley J (2019) Justifying recommendations using distantly-labeled reviews and fine-grained aspects. Jiang J, Ng V, Wan X, eds. Proc. Conf. on Empirical Methods in Natural Language Processing and 9th Internat. Joint Conf. on Natural Language Processing (AFNLP), 188–197.Google Scholar
• Nosko C, Tadelis S (2015) The limits of reputation in platform markets: An empirical analysis and field experiment. Technical report, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Prelec D (2004) A Bayesian truth serum for subjective data. Science 306(5695):462–466.Crossref, Google Scholar
• Radanovic G, Faltings B (2013) A robust Bayesian truth serum for non-binary signals. des Jardins M, Littman M, eds. Proc. 27th AAAI Conf. on Artificial Intelligence (AAAI, Palo Alto, CA), 833–839.Google Scholar
• Radanovic G, Faltings B (2015) Incentives for subjective evaluations with private beliefs. Bonet B, Koenig S, eds. Proc. 29th AAAI Conf. on Artificial Intelligence (AAAI, Palo Alto, CA), 1014–1020.Google Scholar
• Radanovic G, Faltings B, Jurca R (2016) Incentives for effort in crowdsourcing using the peer truth serum. ACM Trans. Intelligent Systems Tech. 7(4):48.Crossref, Google Scholar
• Resnick P, Kuwabara K, Zeckhauser R, Friedman E (2000) Reputation systems. Comm. ACM 43(12):45–48.Crossref, Google Scholar
• Savage LJ (1971) Elicitation of personal probabilities and expectations. J. Amer. Statist. Assoc. 66(336):783–801.Crossref, Google Scholar
• Schoenebeck G, Yu F-Y (2020) Two strongly truthful mechanisms for three heterogeneous agents answering one question. Proc. Internat. Conf. on Web and Internet Econom. (Springer, Berlin), 119–132.Google Scholar
• Schoenebeck G, Yu F-Y (2021) Learning and strongly truthful multi-task peer prediction: A variational approach. Lee JR, ed. Proc. 12th Innovations in Theoretical Comput. Sci. Conf. (Association for Computing Machinery, New York).Google Scholar
• Shnayder V, Agarwal A, Frongillo R, Parkes DC (2016) Informed truthfulness in multi-task peer prediction. Proc. ACM Conf. on Econom. and Comput. (ACM, New York), 179–196.Google Scholar
• Tadelis S (2016) Reputation and feedback systems in online platform markets. Annu. Rev. Econom. 8:321–340.Crossref, Google Scholar
• van Damme E (2002) Strategic equilibrium. Aumann RJ, Hart S eds. Handbook of Game Theory with Economic Applications (Elsevier, North Holland, Amsterdam), 1521–1596.Google Scholar
• von Ahn L, Dabbish L (2004) Labeling images with a computer game. Proc. SIGCHI Conf. on Human Factors in Comput. Systems (ACM, New York), 319–326.Google Scholar
• von Ahn L, Dabbish L (2008) Designing games with a purpose. Comm. ACM 51(8):58–67.Crossref, Google Scholar
• Wan M, McAuley J (2018) Item recommendation on monotonic behavior chains. Amatriain X, O'Donovan J, eds. Proc. 12th ACM Conf. on Recommender Systems (Association for Computing Machinery, New York), 86–94.Google Scholar
• Wan M, Misra R, Nakashole N, McAuley J (2019) Fine-grained spoiler detection from large-scale review corpora. Korhonen A, Traum D, eds. Proc. 57th Annual Meeting of the Assoc. for Comput. Linguistics (ACL, Stroudsburg, PA), 2605–2610.Google Scholar
• Witkowski J, Parkes DC (2012) A robust Bayesian truth serum for small populations. Hoffmann J, Selman B, eds. Proc. 26th AAAI Conf. on Artificial Intelligence (AAAI, Palo Alto), 1492–1498.Google Scholar
• Witkowski J, Parkes DC (2013) Learning the prior in minimal peer prediction. Proc. 3rd Workshop on Social Comput. and User Generated Content at the ACM Conf. on Electronic Commerce.Google Scholar