From the Editor—Median Aggregation, Scoring Rules, Expert Forecasts, Choices with Binary Attributes, Portfolio with Dependent Projects, and Information Security
Abstract
The December issue of Decision Analysis contains six articles. Our first three papers aim to improve the quality of probability judgments. Hora, Fransen, Hawkins, and Susel argue that medium aggregation of distribution functions works better than mean aggregation of probabilities. Merkle and Steyvers examine the relative attractiveness of different scorings rules. Karvetski, Olson, Mandel, and Twardy propose a coherence weighted probability aggregation rule for combining expert forecasts. Our fourth paper by Katsikopoulos presents a theory that explains why simple heuristics often perform well in multiattribute choices with binary attributes. Bhattacharjya, Eidsvik, and Mukerji provide closed-form results for value of information in the context of portfolio selection with dependent projects. Finally, Gao, Zhong, and Mei use game theory to analyze information security. All of the papers have a common purpose of improving decisions in private and public domains.
You see things; and you say ``Why?'' But I dream things that never were; and I say ``Why not?''
George Bernard Shaw
In this issue we have six papers that contribute to probability aggregation, multiattribute decisions, portfolio analysis, and economics of information security. Our first paper by Hora, Fransen, Hawkins, and Susel investigates aggregation of probability judgments provided by experts or managers. The authors propose that medium aggregations of distribution functions work better than mean aggregation of probabilities. The usefulness of the method is demonstrated in the context of national security. The simplicity of the method is likely to be appealing to risk analysis in private and public sectors. Relevant papers in INFORMS journals are: Glaser et al. (2007) and Lichtendahl et al. (2013).
Our second paper by Merkle and Steyvers examines the relative attractiveness of alternative scoring rules for a specific forecasting domain. It has been widely believed that scoring roles are robust and provide essentially similar ranking of forecasters. This paper argues that depending on the forecasting domain, different scoring rules may yield a different ranking of forecasters. The authors compare 10 forecasters with respect to their forecasts on a number of world events. The results of this paper are specifically useful to a decision maker who must choose a scoring rule from a large number of possibilities, that best suits his needs. Articles related to this paper published in INFORMS are: Nau (1985), Winkler (1994), Kilgour and Gerchak (2004), Lichtendahl and Winkler (2007), Bickel (2007), Schervish et al. (2009), Bickel (2010), and Johnstone et al. (2011).
Our third paper by Karvetski, Olson, Mandel, and Twardy proposes an approach for combining expert forecasts. A simple average of forecasts (equal weighting of forecasters) often yields a good result. It is, however, possible that some forecasters are more ``able'' than others, and giving a higher weight to their opinion may yield more accurate forecasts. The question is how one can identify the able forecasters in advance. The authors use ``coherence'' of probability judgments as a criterion to distinguish among forecasters. Coherence-weighted estimates improve accuracy over equal weighting. Other papers related to the topic of this paper are: Collopy and Armstrong (1992), Winkler and Clemen (2004), Predd et al. (2008), and Wang et al. (2011).
Our fourth paper by Katsikopoulos presents a theory that explains why simple heuristics often perform well in multiattribute choices with binary attributes. Examples of simple heuristics are equal weighting and elimination by aspects. It is clear that for decisions of strategic importance one is better served by conducting a thorough multiattribute utility analysis. In a number of decisions one faces in daily life, a quick analysis that respects the decision maker's preferences but imposes small cognitive burden, may be sufficient to yield good results. Using the theory of combinatorics, this paper provides useful insights into conditions when heuristics perform well, and therefore a preferred choice can be identified with minimal input from the decision maker. The papers related to this research are: Hogarth and Karelia (2005), Baucells et al. (2008), and Baucells and Sarin (2013).
Our fifth paper by Bhattacharjya, Eidsvik, and Mukerji examines value of information in the context of portfolio selection. In many economic and science problems, projects contained in the portfolio are probabilistically dependent. A smart approach to information gathering can improve the overall value of portfolio. Under the assumptions of risk neutrality and project returns as multivariate Gaussian distribution, the authors provide closed-form analytical results. In public sector applications, risk neutrality may be a reasonable assumption. Similarly, Gaussian models, though not perfect, may be approximately valid in a variety of applications. Keisler (2004) is relate to this work.
Our final paper by Gao, Zhong, and Mei addresses the important question of information security. Firms invest heavily to protect their information systems but hackers seek to breach the security. The decision environment is dynamic and requires strategic calculations from both the firm and hackers. The contribution of this paper is to utilize a differential game to analyze dynamic interactions. Both simultaneous and sequential moves are considered. The results of this paper are also applicable to strategic choices of a government, and terrorists who disseminate knowledge within a network of terrorists. Some papers related to this research include Cavusoglu and Raghunathan (2004), Pate-Cornell (2012), and Cavusoglu et al. (2013).
References
- (2008) Cumulative dominance and heuristic performance in binary multiattribute choice. Oper. Res. 56:1289–1304.Link, Google Scholar
- (2013) Guided decision processes. EURO J. Decision Processes 1:24–44.Crossref, Google Scholar
- (2007) Some comparisons among quadratic, spherical, and logarithmic scoring rules. Decision Anal. 4:49–65.Link, Google Scholar
- (2010) Scoring rules and decision analysis education. Decision Anal. 7:346–357.Link, Google Scholar
- (2004) Configuration of detection software: A comparison of decision and game theory approaches. Decision Anal. 1:131–148.Link, Google Scholar
- (2013) Passenger profiling and screening for aviation security in the presence of strategic attackers. Decision Anal. 10:63–81.Link, Google Scholar
- (1992) Rule-based forecasting: Development and validation of an expert systems approach to combining time series extrapolations. Management Sci. 38:1394–1414.Link, Google Scholar
- (2007) On the trend recognition and forecasting ability of professional traders. Decision Anal. 4:176–193.Link, Google Scholar
- (2005) Simple models for multiattribute choice with many alternatives: When it does and does not pay to face trade-offs with binary attributes. Management Sci. 51:1860–1872.Link, Google Scholar
- (2011) Tailored scoring rules for probabilities. Decision Anal. 8:256–268.Link, Google Scholar
- (2004) Value of information in portfolio decision analysis. Decision Anal. 1:177–189.Link, Google Scholar
- (2004) Elicitation of probabilities using competitive scoring rules. Decision Anal. 1:108–113.Link, Google Scholar
- (2007) Probability elicitation, scoring rules, and competition among forecasters. Management Sci. 53:1745–1755.Link, Google Scholar
- (2013) Is it better to average probabilities or quantiles? Management Sci. 59:1594–1611.Link, Google Scholar
- (1985) Should scoring rules be “effective”? Management Sci. 31:527–535.Link, Google Scholar
- (2012) Games, risks, and analytics: Several illustrative cases involving national security and management situations. Decision Anal. 9:186–203.Link, Google Scholar
- (2008) Aggregating probabilistic forecasts from incoherent and abstaining experts. Decision Anal. 5:177–189.Link, Google Scholar
- (2009) Proper scoring rules, dominated forecasts, and coherence. Decision Anal. 6:202–221.Link, Google Scholar
- (2011) Aggregating large sets of probabilistic forecasts by weighted coherent adjustment. Decision Anal. 8:128–144.Link, Google Scholar
- (1994) Evaluating probabilities: Asymmetric scoring rules. Management Sci. 40:1395–1405.Link, Google Scholar
- (2004) Multiple experts vs. multiple methods: Combining correlation assessments. Decision Anal. 1:167–176.Link, Google Scholar
- (2013) Components of portfolio value of information. Decision Anal. 10:171–185.Link, Google Scholar

