Incentive-Compatible Forecasting Competitions

References

• Atanasov P, Rescober P, Stone E, Servan-Schreiber E, Tetlock PE, Ungar L, Mellers B (2017) Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management Sci. 63(3):691–706.Link, Google Scholar
• Brier GW (1950) Verification of forecasts expressed in terms of probability. Monthly Weather Rev. 78(1):1–3.Crossref, Google Scholar
• Chakraborty A (2016) How companies are using Kaggle to find the best machine learning talent. Accessed December 24, 2020, https://blog.udacity.com/2016/07/companies-kaggle-machine-learning-talent.html.Google Scholar
• Chakraborty M, Das S, Lavoie A, Magdon-Ismail M, Naamad Y (2013) Instructor rating markets. DesJardins M, Littman M, eds. Proc. 27th AAAI Conf. on Artificial Intelligence (AAAI Press, Palo Alto), 159–165.Google Scholar
• Cowgill B, Zitzewitz E (2015) Corporate prediction markets: Evidence from Google, Ford, and Firm X. Rev. Econom. Stud. 82(4):1309–1341.Crossref, Google Scholar
• Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102:359–378.Crossref, Google Scholar
• Good IJ (1952) Rational decisions. J. Roy. Statist. Soc. B 14(1):107–114.Crossref, Google Scholar
• Grushka-Cockayne Y, Lichtendahl KC, Jose VR, Winkler RL (2017) Quantile evaluation, sensitivity to bracketing, and sharing business payoffs. Oper. Res. 65(3):557–836.Link, Google Scholar
• Harris D (2013) Facebook is hiring a data scientist, but you’ll have to fight for the job. Accessed December 24, 2020, https://gigaom.com/2013/08/30/facebook-is-hiring-a-data-scientist-but-youll-have-to-fight-for-the-job/.Google Scholar
• Hoeffding W (1963) Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58(301):13–30.Crossref, Google Scholar
• Hossain T, Okui R (2013) The binarized scoring rule. Rev. Econom. Stud. 80(3):984–1001.Crossref, Google Scholar
• Jia Y, Liu Y, Yu X, Voida S (2017) Designing leaderboards for gamification: Perceived differences based on user ranking, application domain, and personality traits. Mark G, Fussell S, Lampe C, Schraefel MC, eds. Proc. CHI Conf. on Human Factors in Comput. Systems (ACM, New York), 1949–1960.Google Scholar
• Jose VR (2009) A characterization for the spherical scoring rule. Theory Decision 66(3):263–281.Crossref, Google Scholar
• Jose VR (2017) Percentage and relative error measures in forecast evaluation. Oper. Res. 65(1):200–211.Link, Google Scholar
• Kaggle (2017) March machine learning mania, 1st place winner’s interview: Andrew Landgraf. Accessed December 24, 2020, https://medium.com/kaggle-blog/march-machine-learning-mania-1st-place-winners-interview-andrew-landgraf-f18214efc659.Google Scholar
• Karni E (2009) A mechanism for eliciting probabilities. Econometrica 77(2):603–606.Crossref, Google Scholar
• Kearns MJ, Vazirani UV (1994) An Introduction to Computational Learning Theory (MIT Press, Cambridge, MA).Crossref, Google Scholar
• Kilgour DM, Gerchak Y (2004) Elicitation of probabilities using competitive scoring rules. Decision Anal. 1(2):108–113.Link, Google Scholar
• Konrad KA (2009) Strategy and Dynamics in Contests (Oxford University Press, Oxford, UK).Crossref, Google Scholar
• Lambert NS (2018) Probability elicitation for agents with arbitrary risk preferences. Working paper, Stanford University, Stanford, CA.Google Scholar
• Lambert N, Langford J, Wortman J, Chen Y, Reeves D, Shoham Y, Pennock DM (2008) Self-financed wagering mechanisms for forecasting. Sandholm T, Riedl J, Fortnow L, eds. Proc. 9th ACM Conf. on Electronic Commerce (ACM, New York), 170–179.Google Scholar
• Lichtendahl KCJ, Winkler RL (2007) Probability elicitation, scoring rules, and competition among forecasters. Management Sci. 53(11):1745–1755.Link, Google Scholar
• Lichtendahl KC, Grushka-Cockayne Y, Pfeifer PE (2013) The wisdom of competitive crowds. Oper. Res. 61(6):1383–1398.Link, Google Scholar
• Machina MJ, Schmeidler D (1992) A more robust definition of subjective probability. Econometrica 60(4):745–780.Crossref, Google Scholar
• McCarthy J (1956) Measures of the value of information. Proc. National Acad. Sci. USA 42(9):654–655.Crossref, Google Scholar
• Mellers B, Ungar L, Baron J, Ramos J, Gurcay B, Fincher K, Scott SE, et al.. (2014) Psychological strategies for winning a geopolitical forecasting tournament. Psych. Sci. 25(5):1106–1115.Crossref, Google Scholar
• Palley AB, Soll JB (2019) Extracting the wisdom of crowds when information is shared. Management Sci. 65(5):1949–2443.Google Scholar
• Satopää VA, Baron J, Foster DP, Mellers BA, Tetlock PE, Ungar LH (2014) Combining multiple probability predictions using a simple logit model. Internat. J. Forecasting 30(2):344–356.Crossref, Google Scholar
• Savage LJ (1971) Elicitation of personal probabilities and expectations. J. Amer. Statist. Assoc. 66:783–801.Crossref, Google Scholar
• Schervish MJ (1989) A general method for comparing probability assessors. Ann. Statist. 17(4):1856–1879.Crossref, Google Scholar
• Selten R (1998) Axiomatic characterization of the quadratic scoring rule. Experiment. Econom. 1:43–61.Crossref, Google Scholar
• Servan-Schreiber E, Wolfers J, Pennock DM, Galebach B (2004) Prediction markets: Does money matter? Electronic Marketing 14(3):243–251.Crossref, Google Scholar
• Tetlock PE, Gardner D (2015) Superforecasting: The Art and Science of Prediction (Crown Publishing Group, New York).Google Scholar
• Witkowski J, Freeman R, Wortman Vaughan J, Pennock DM, Krause A (2018) Incentive-compatible forecasting competitions. McIlraith S, Weinberger K, eds. Proc. 32nd AAAI Conf. on Artificial Intelligence (AAAI Press, Palo Alto), 1282–1289.Google Scholar