This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Hoffmann, S. Articles by McWilliams, M. Search for Related Content

Elicitation from Large, Heterogeneous Expert Panels: Using Multiple Uncertainty Measures to Characterize Information Quality for Decision Analysis

Resources for the Future, 1616 P St. NW, Washington, D.C. 20036
Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
Resources for the Future, 1616 P St. NW, Washington, D.C. 20036
Department of Economics, University of Michigan, Ann Arbor, Michigan 48109
shoffmann{at}rff.org
fischbeck{at}cmu.edu
akrupnick{at}rff.org
mrmcw{at}umich.edu

Decision analysts are frequently called on to help inform decision makers in cases involving considerable uncertainty. In such situations, expert elicitation of parameter values is frequently used to supplement more conventional research. Expert elicitations typically rely on small panels of experts. However, in cases where the information needed for risk management must draw on a broad range of disciplines or types of professional backgrounds and experience, a larger, more heterogeneous expert panel is needed. In this paper we develop a formal protocol and a suite of uncertainty measures for this work. The protocol uses formal survey methods to take advantage of variation in individual expert uncertainty and heterogeneity among experts as a means of quantifying and comparing sources of uncertainty about parameters of interest. We illustrate the use of this protocol with an expert elicitation on the distribution of foodborne illness in the United States across foods. In the survey, experts are asked to attribute illnesses associated with one of eleven major foodborne pathogens to the consumption of one of eleven categories of food. Results show how the distributions of multiple measures of uncertainty (e.g., agreement of experts and uncertainty in knowledge), made feasible by use of a large panel of experts, can help identify which of several types of risk management actions may be most appropriate.

Key Words: expert elicitation; analytical methods; risk characterization
History: Received on March 28, 2006. Accepted on May 17, 2007.

This article has been cited by other articles:

				L. R. Keller From the Editor... Decision Analysis, March 1, 2009; 6(1): 1 - 3. [Abstract] [PDF]

				L. R. Keller and K. M. Kophazi From the Editors... Decision Analysis, June 1, 2008; 5(2): 57 - 59. [Abstract] [PDF]