On the Road to Making Science of “Art”: Risk Bias in Market Scoring Rules

References

• Abernethy J, Chen Y, Vaughan JW (2011) An optimization-based framework for automated market-making. Proc. 12th ACM Conf. Electronic Commerce (Association for Computing Machinery, New York), 297–306.Google Scholar
• Abernethy J, Chen Y, Vaughan JW (2013) Efficient market making via convex optimization, and a connection to online learning. ACM Trans. Econom. Comput. 1(2):12.Google Scholar
• Abernethy J, Kutty S, Lahaie S, Sami R (2014) Information aggregation in exponential family markets. Proc. Fifteenth ACM Conf. Econom. Comput. (Association for Computing Machinery, New York), 395–412.Google Scholar
• Allen F (1987) Notes-discovering personal probabilities when utility functions are unknown. Management Sci. 33(4):542–544.Link, Google Scholar
• Arrow KJ, Forsythe R, Gorham M, Hahn R, Hanson R, Ledyard JO, Levmore Set al. (2008) The promise of prediction markets. Science 320(5878):877–878.Crossref, Google Scholar
• Atanasov P, Rescober P, Stone E, Swift SA, Servan-Schreiber E, Tetlock P, Ungar L, Mellers B (2016) Distilling the wisdom of crowds: Prediction markets vs. prediction polls. Management Sci. 63(3):691–706.Link, Google Scholar
• Babcock BA, Choi EK, Feinerman E (1993) Risk and probability premiums for CARA utility functions. J. Agricultural Resource Econom. 18(1):17–24.Google Scholar
• Berg JE, Neumann GR, Rietz TA (2009) Searching for Google’s value: Using prediction markets to forecast market capitalization prior to an initial public offering. Management Sci. 55(3):348–361.Link, Google Scholar
• Bossaerts P, Fine L, Ledyard J (2002) Inducing liquidity in thin financial markets through combined-value trading mechanisms. Eur. Econom. Rev. 46(9):1671–1695.Crossref, Google Scholar
• Brahma A, Chakraborty M, Das S, Lavoie A, Magdon-Ismail M (2012) A bayesian market maker. Proc. 13th ACM Conf. Electronic Commerce (Association for Computing Machinery, New York),215–232.Google Scholar
• Brier GW (1950) Verification of forecasts expressed in terms of probability. Monthly Weather Rev. 78(1):1–3.Crossref, Google Scholar
• Chen KY, Fine LR, Huberman BA (2004) Eliminating public knowledge biases in information-aggregation mechanisms. Management Sci. 50(7):983–994.Link, Google Scholar
• Chen MK, Ingersoll JE Jr, Kaplan EH (2008) Modeling a presidential prediction market. Management Sci. 54(8):1381–1394.Link, Google Scholar
• Chen Y, Pennock DM (2007) A utility framework for bounded-loss market makers. Proc. 23rd Conf. Uncertainty Artificial Intelligence (Association for Uncertainty in Artificial Intelligence, Portland, Oregon), 49–56.Google Scholar
• Chen Y, Vaughan JW (2010) A new understanding of prediction markets via no-regret learning. Proc. 11th ACM Conf. Electronic Commerce (Association for Computing Machinery, New York), 189–198.Google Scholar
• Cowgill B, Zitzewitz E (2015) Corporate prediction markets: Evidence from Google, Ford, and Firm x. Rev. Econom. Stud. 82(4):1309–1341.Crossref, Google Scholar
• Cox JC, Harrison GW (2008) Risk Aversion in Experiments, Vol. 12 (Emerald Group Publishing, Bingley, UK).Crossref, Google Scholar
• Csaszar FA, Eggers JP (2013) Organizational decision making: An information aggregation view. Management Sci. 59(10):2257–2277.Link, Google Scholar
• Dimitrov S, Sami R, Epelman MA (2015) Subsidized prediction mechanisms for risk-averse agents. ACM Trans. Econom. Comput. 3(4):24.Google Scholar
• Dudík M, Lahaie S, Rogers R, Vaughan JW (2017) A decomposition of forecast error in prediction markets. Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, eds. Adv. Neural Inform. Processing Systems (Curran Associates, Inc., New York), 4374–4383.Google Scholar
• Economist, The (2016) Polls versus prediction markets: Who said Brexit was a surprise? (June 24), https://www.economist.com/blogs/graphicdetail/2016/06/polls-versus-prediction-markets.Google Scholar
• Filios M (2016) Identifying black swans. (May 9), https://www.linkedin.com/pulse/identifying-black-swans-your-organizationbefore-occur-matt-filiosGoogle Scholar
• Frongillo R, Reid MD (2015) Convergence analysis of prediction markets via randomized subspace descent. Cortes C, Lawrence ND, Lee DD, Sugiyama M, Garnett R, eds. Proc. 28th Internat. Conf. Neural Inform. Processing Systems, Vol. 2 (MIT Press, Cambridge, MA), 3034–3042.Google Scholar
• Frongillo RM, Penna ND, Reid MD (2012) Interpreting prediction markets: A stochastic approach. Pereira F, Burges CJC, Bottou L, Weinberger KQ, eds. Proc. 25th Internat. Conf. Neural Inform. Processing Systems, Vol. 2 (Curran Associates Inc., New York), 3266–3274.Google Scholar
• Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc. 102(477):359–378.Crossref, Google Scholar
• Good IJ (1952) Rational decisions. J. Roy. Statist. Soc. Ser. B (Methodological) 14(1):107–114.Google Scholar
• Hanson R (2003) Combinatorial information market design. Inform. Systems Frontiers 5(1):107–119.Crossref, Google Scholar
• Healy PJ, Linardi S, Lowery JR, Ledyard JO (2010) Prediction markets: Alternative mechanisms for complex environments with few traders. Management Sci. 56(11):1977–1996.Link, Google Scholar
• Heidari H, Lahaie S, Pennock DM, Vaughan JW (2015) Integrating market makers, limit orders, and continuous trade in prediction markets. Proc. Sixteenth ACM Conf. Econom. Comput. (Association for Computing Machinery, New York), 583–600.Google Scholar
• Kadane JB, Winkler RL (1988) Separating probability elicitation from utilities. J. Amer. Statist. Assoc. 83(402):357–363.Crossref, Google Scholar
• Karni E (2009) A mechanism for eliciting probabilities. Econometrica 77(2):603–606.Crossref, Google Scholar
• Lambert N (2011) Probability elicitation for agents with arbitrary risk preferences. Working paper, Graduate School of Business, Stanford University, Stanford, CA.Google Scholar
• Ledbetter J (2008) The fizz-dom of crowds: If prediction markets are so great, why have they been so wrong lately? Slate (April 16), http://www.slate.com/articles/news_and_politics/hey_wait_a_minute/2008/04/the_fizzdom_of_crowds.html.Google Scholar
• Li X, Vaughan JW (2013) An axiomatic characterization of adaptive-liquidity market makers. Proc. Fourteenth ACM Conf. Electronic Commerce (Association for Computing Machinery, New York), 657–674.Google Scholar
• Manski CF (2006) Interpreting the predictions of prediction markets. Econom. Lett. 91(3):425–429.Crossref, Google Scholar
• McCarthy J (1956) Measures of the value of information. Proc. Natl. Acad. Sci. USA 42(9):654–655.Google Scholar
• Mellers B, Stone E, Atanasov P, Rohrbaugh N, Metz SE, Ungar L, Bishop MM, Horowitz M, Merkle E, Tetlock P (2015) The psychology of intelligence analysis: Drivers of prediction accuracy in world politics. J. Experiment. Psych.: Appl. 21(1):1–14.Crossref, Google Scholar
• Murphy AH, Winkler RL (1970) Scoring rules in probability assessment and evaluation. Acta Psychologica 34:273–286.Crossref, Google Scholar
• Offerman T, Sonnemans J, Van de Kuilen G, Wakker PP (2009) A truth serum for non-Bayesians: Correcting proper scoring rules for risk attitudes. Rev. Econom. Stud. 76(4):1461–1489.Crossref, Google Scholar
• Ortner G (1998) Forecasting markets—An industrial application. Working paper, University of Vienna, Vienna.Google Scholar
• Ostrovsky M (2012) Information aggregation in dynamic markets with strategic traders. Econometrica 80(6):2595–2647.Crossref, Google Scholar
• Othman A, Pennock DM, Reeves DM, Sandholm T (2013) A practical liquidity-sensitive automated market maker. ACM Trans. Econom. Comput. 1(3):14–24.Google Scholar
• Palfrey TR, Wang SW (2012) Speculative overpricing in asset markets with information flows. Econometrica 80(5):1937–1976.Crossref, Google Scholar
• Pennock DM (2010) Why automated market makers? Oddhead (blog) (July 8), blog.oddhead.com/2010/07/08/why-automated-market-makers/.Google Scholar
• Plott CR, Chen KY (2002) Information aggregation mechanisms: Concept, design and implementation for a sales forecasting problem. Working paper, Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA.Google Scholar
• Polgreen PM, Nelson FD, Neumann GR, Weinstein RA (2007) Use of prediction markets to forecast infectious disease activity. Clinical Infectious Diseases 44(2):272–279.Crossref, Google Scholar
• Pratt JW (1964) Risk aversion in the small and in the large. Econometrica 32(1–2):122–136.Crossref, Google Scholar
• Premachandra M, Reid M (2013) Aggregating predictions via sequential mini-trading. Ong CS, Ho TB, eds. Proc. 5th Asian Conf. Machine Learn., Vol. 29 (Microtome Publishing, Brookline, MA), 373–387.Google Scholar
• Savage LJ (1971) Elicitation of personal probabilities and expectations. J. Amer. Statist. Assoc. 66(336):783–801.Crossref, Google Scholar
• Schlack JW (2015) Ask your customers for predictions, not preferences. Harvard Bus. Rev. (January 5), https://hbr.org/2015/01/ask-your-customers-for-predictions-not-preferences.Google Scholar
• Schlag KH, van der Weele JJ (2013) Eliciting probabilities, means, medians, variances and covariances without assuming risk neutrality. Theoret. Econom. Lett. 3(1):38–42.Crossref, Google Scholar
• Sethi R, Vaughan JW (2016) Belief aggregation with automated market makers. Comput. Econom. 48(1):155–178.Crossref, Google Scholar
• Slamka C, Skiera B, Spann M (2013) Prediction market performance and market liquidity: A comparison of automated market makers. IEEE Trans. Engrg. Management 60(1):169–185.Crossref, Google Scholar
• Tziralis G, Tatsiopoulos I (2012) Prediction markets: An extended literature review. J. Prediction Markets 1(1):75–91.Google Scholar
• Wah E, Lahaie S, Pennock DM (2016) An empirical game-theoretic analysis of price discovery in prediction markets. Proc. 25th Internat. Joint Conf. Artificial Intelligence (AAAI Press, New York), 510–516.Google Scholar
• Wakker P, Thaler R, Tversky A (1997) Probabilistic insurance. J. Risk Uncertainty 15(1):7–28.Crossref, Google Scholar
• Winkler RL (1969) Scoring rules and the evaluation of probability assessors. J. Amer. Statist. Assoc. 64(327):1073–1078.Crossref, Google Scholar
• Wolfers J, Zitzewitz E (2006) Interpreting prediction market prices as probabilities. Technical report, National Bureau of Economic Research, Cambridge, MA.Google Scholar