Elicitation from Large, Heterogeneous Expert Panels: Using Multiple Uncertainty Measures to Characterize Information Quality for Decision Analysis

References

• Adams F. Expert elicitation and Bayesian analysis of construction contract risks: An investigation. Construction Management Econom. (2006) 24:81–96Crossref, Google Scholar
• Barke R. P., Jenkins-Smith H. C. Politics and scientific expertise: Scientists, risk perception, and nuclear waste policy. Risk Anal. (1993) 13:425–429Crossref, Google Scholar
• Batz M. B., Doyle M. P., Morris J. G., Painter J., Singh R., Tauxe R. V., Taylor M. R., Danilo M. A., Wong L. F. Attributing illness to food. Emerging Infectious Disease (2005) 11:993–999Crossref, Google Scholar
• Bruine de Bruin W., Fischhoff B., Brillant L., Caruso D. Expert judgments of pandemic influenza risks. Global Public Health (2006) 1(2):179–194Crossref, Google Scholar
• Carmines E. G., Zeller R. A.Reliability and Validity Assessment (1991) (Sage Publications, Newbury Park, CA) Google Scholar
• Centers for Disease Control and Prevention (CDC) Foodborne diseases active surveillance network (FoodNet). (2003) . Emerging Infections Program Report on Foodborne Pathogens, http://www.cdc.gov/foodnet/pub/publications/2005/FNsurv2003.pdf, accessed February 1, 2006Google Scholar
• Centers for Disease Control and Prevention (CDC) Foodborne illness. (2006a) . Frequently Asked Questions, http://www.cdc.gov/ncidod/dbmd/diseaseinfo/foodborneinfections_g.htm, accessed August 9, 2006Google Scholar
• Centers for Disease Control and Prevention (CDC) Outbreak investigations. (2006b) . Food Safety Home, http://www.cdc.gov/foodsafety/outbreaks.htm, accessed February 10, 2006Google Scholar
• Center for Food Safety and Applied Nutrition (CFSAN), U.S. Food and Drug Administration Quantitative assessment of the relative risk to public health from foodborne Listeria monocytogenes among selected categories of ready-to-eat foods. (2004) . Food and Drug Administration, Washington, DC. http://www.foodsafety.gov/∼dms/lmr2-toc.htmlGoogle Scholar
• Clemen R. T.Making Hard Decisions: An Introduction to Decision Analysis (1996) (Brooks/Cole Publishing Co., Pacific Grove, CA) Google Scholar
• Clemen R. T., Winkler R. L. Combining probability distributions in risk analysis. Risk Anal. (1999) 19:187–203Crossref, Google Scholar
• Cooke R. M.Experts in Uncertainty: Opinion and Subjective Probability in Science (1991) (Oxford University Press, Oxford, UK) Crossref, Google Scholar
• Cooke R., Goossens L. H. J. Procedure guide for structured expert judgment. (2000) . Report to European Commission, Directorate-General for Research, EUR 18820 EN, LuxembourgGoogle Scholar
• Cooke R., Goossens L. H. J. Expert judgement elicitation for risk assessment of critical infrastructures. J. Risk Res. (2004) 7:643–656Crossref, Google Scholar
• Cooke R., Goossens L. H. J. Expert judgement elicitations for risk assessments of critical infrastructures. ([date unknown]) . Unpublished manuscript, Delft University of Technology, Delft, The NetherlandsGoogle Scholar
• Cooke R., Goossens L. H. J. TU delft expert judgment data base. Reliability Engrg. Safety Systems (2007) . ForthcomingGoogle Scholar
• Craig K.The Nature of Explanation (1943) (Cambridge University Press, Cambridge, UK) Google Scholar
• Epley N., Gilovich T. Putting adjustment back into the anchoring-and-adjustment heuristic: Self-generated versus experimenter provided anchors. Psych. Sci. (2001) 12:391–396Crossref, Google Scholar
• Erickson B., Nosanchuck T. Applied network sampling. Soc. Networks (1983) 15:367–382Crossref, Google Scholar
• Food and Agriculture Organization (FAO) Risk assessments of Salmonella in eggs and broiler chickens. (2002) . (Microbial Risk Assessment Series, No. 1), http://www.fao.org/ag/agn/jemra/salmonella_en.stmGoogle Scholar
• Food and Drug Administration (FDA)/Food Safety Inspection Service (USDA)/Centers for Disease Control and Prevention (CDC) (2003) . Quantitative assessment of relative risks to public health from foodborne Listeria moncytogenes among selected categories of ready-to-eat foods, http://www.foodsafety.gov/∼dms/lmr2-toc.htmlGoogle Scholar
• Food Safety Inspection Service (FSIS) Risk assessments of Salmonella enteritidis in shell eggs and Salmonella spp. in egg products. (2005) (U.S. Department of Agriculture, Washington, DC) . http://www.fsis.usda.gov/Science/Risk_Assessments/index.asp, accessed March 4, 2007Google Scholar
• Friedman D. R., Hoekstra R. M., Samuel M., Marcus R., Bender J., Shiferaw B., et al. Emerging infections program FoodNet Working Group. Risk factors for sporadic Camplobacter infection in the United States: A case-control study. Clinical Infectious Diseases (2004) 38:S285–S296Crossref, Google Scholar
• Gentner D., Stevens A.Mental Models (1983) (Lawrence Erlbaum Associates, Hillsdale, NJ) Google Scholar
• George J. F., Duffy K., Ahuja M. Countering the anchoring and adjustment bias with decision support systems. Decision Support Systems (2000) 29:195–206Crossref, Google Scholar
• Granovetter M. Network sampling: Some first steps. Amer. J. Sociol. (1973) 81:1287–1302Crossref, Google Scholar
• Hoffman F. O., Kaplan S. Beyond the domain of direct observation: How to specify a probability distribution that represents the “state of knowledge” about uncertain inputs. Risk Anal. (1999) 19:131–134Crossref, Google Scholar
• Hoffmann S. A., Fischbeck P., McWilliams M., Krupnick A. Using expert elicitation to attribute U.S. foodborne illness to food consumption. (2006) . Presentations at the FoodNet Attribution Working Group, August 3; Centers for Disease Control and Prevention, Division of Bacterial and Mycotic Diseases, August 8; and USDA Food Safety Inspection Service, Office of Public Health Science, September 27Google Scholar
• Hoffmann S. A., Fischbeck P., McWilliams M., Krupnick A. Using expert elicitation to link foodborne illnesses in the U.S. to food. J. Food Protection (2007a) 70:1220–1229Crossref, Google Scholar
• Hoffmann S. A., Fischbeck P., McWilliams M., Krupnick A. Informing risk-mitigation priorities using uncertainty measures derived from heterogeneous expert panels: A demonstration using foodborne pathogens. Reliability Engrg. Safety Systems (2007b) . ForthcomingGoogle Scholar
• Interagency Risk Assessment Consortium Food and waterborne pathogen risk ranking model: From theory to practice. (2005) . Workshop, College Park, MD, August 18, http://www.foodrisk.org/RAC_risk_ranking_workshop.cfmGoogle Scholar
• Kalton G., Anderson D. W. Sampling rare populations. J. Roy. Statist. Soc. A (1986) 149:65–82Crossref, Google Scholar
• Keith D. W. When is it appropriate to combine expert judgments? Climatic Change (1996) 33:139–143Crossref, Google Scholar
• Lempert R. J., Nakicenovic N., Sareqitz D., Schlesinger M. Characterizing climate change uncertainties for decision-makers. Climate Change (2004) 65:1–9Crossref, Google Scholar
• Manitoba Health (2006) . Communicable disease management protocol manual, http://www.gov.mb.ca/health/publichealth/cdc/protocol/investigation.pdf, accessed August 9, 2006Google Scholar
• Mastrandrea M., Schneider S. Probabilistic integrated assessment of “dangerous” climate change. Science (2004) 304:571–575Crossref, Google Scholar
• McKinley J., Marshall C. Officials narrow investigation after finding bad spinach. New York Times (2006) Sept. 21.Google Scholar
• Mead P. S., Slutsker L., Dietz V., McCaig L. F., Bresee J. S., Shapiro C., Griffin P. M., Tauxe R. V. Food-related illness and death in the United States. Emerging Infectious Diseases (1999) 5:607–625Crossref, Google Scholar
• Merrick J. R. W., van Dorp J. R., Singh A. Analysis of correlated expert judgments from extended pairwise comparisons. Decision Anal. (2005) 2(1):17–29Link, Google Scholar
• Morgan M. G., Keith D. Subjective judgments by climate experts. Environ. Sci. Tech. (1995) 29:468–476Crossref, Google Scholar
• Morgan M. G., Henrion M.Uncertainty: A Guide to Dealing with Uncertainty in Qualitative Risk and Policy Analysis (1990) (Cambridge University Press, Cambridge, MA) Crossref, Google Scholar
• Morgan M. G., Adams P., Keith D. W. Elicitation of expert judgments of aerosol forcing. Climate Change (2006) 75:195–214Crossref, Google Scholar
• Morgan M. G., Fischhoff B., Bostrom A., Atman C. J.Risk Communication: A Mental Models Approach (2002) (Cambridge University Press, Cambridge, MA) Google Scholar
• Morgan M. G., Florig H. K., DeKay M. L., Fischbeck P. Categorizing risks for risk ranking. Risk Anal. (2000) 20:49–58Crossref, Google Scholar
• National Academies of Science (NAS) Escherichia coli O157:H7 in Ground Beef: Review of a Draft Risk Assessment. (2002) (National Academies Press, Washington, DC) Google Scholar
• Newman M., Watts D., Strogatz S. Random graph models of social networks. Proc. Natl. Acad. Sci. USA (2002) 99:2566–2572Crossref, Google Scholar
• Nordhaus W. D. Expert opinion on climate change. Amer. Scientist (1994) 82(1):45–51Google Scholar
• Rea L. M., Parker R. A.Designing and Conducting Survey Research: A Comprehensive Guide (1997) (Jossey-Bass Publishers, San Francisco, CA) Google Scholar
• Thompson S. K.Sampling (2002) 2nd ed.(J. Wiley and Sons, New York) Google Scholar
• Tversky A., Kahneman D. Judgment under uncertainty: Heuristics and biases. Science (1974) 185:1124–1131Crossref, Google Scholar
• Van der Fels-Klerz I. J. H., Goossens L. H. J., Saatkamp H. W., Horst S. H. S. Elicitation of quantitative data from a heterogeneous expert panel: Formal process and application in animal health. Risk Anal. (2002) 22(1):67–81Crossref, Google Scholar