Expert Elicitation of Adversary Preferences Using Ordinal Judgments

We introduce a simple elicitation process where subject-matter experts provide only ordinal judgments of the attractiveness of potential targets, and the adversary utility of each target is assumed to involve multiple attributes. Probability distributions over the various attribute weights are then mathematically derived (using either probabilistic inversion or Bayesian density estimation). This elicitation process reduces the burden of time-consuming orientation and training in traditional methods of attribute weight elicitation, and explicitly captures the existing uncertainty and disagreement among experts, rather than attempts to achieve consensus by eliminating them. We identify the relationship between the two methods and conduct sensitivity analysis to elucidate how these methods handle expert consensus or disagreement. We also present a real-world application on elicitation of adversarial preferences over various attack scenarios to show the applicability of our proposed methods.