Decision-making in the public sector

Decision-making by public officials drives programs and policies and has a significant impact on the lives of citizens. I tell my public affairs students that businesses make things and governments make decisions. From local decisions to federal decisions, the impact of public sector decision-making on the lives of everyday people is significant. In contrast, private sector decision-making is likely to only impact a relatively narrow group of people – those who work at the company, those who have invested in the company, and those who buy the company’s products. It is striking that many in the private sector have been embracing the evidence-based practices and tools from research of the decision sciences for 50+ years, whereas public sector uses are few and far between.

Public sector decision-making can easily be influenced by political factors, which make good decision practices even more important. This difference between decision science use in private versus public sector can be attributed to several factors, including where these skills are taught and what the desired outcomes of the decisions are. I hope that the use of decision science practices that has been growing in the environmental, natural resources, energy and public health decision-making areas will continue to spread and therefore lead to improved decision-making throughout the public sector.

Decision Science Journey

Decision sciences are often taught in engineering and business schools (think operations research and linear optimization) but have an increasing presence in public affairs programs. My own journey into the decision sciences is illustrative: I took my first class in decision science as a master’s student at Indiana University’s public affairs program, now known as the O’Neill School of Public Affairs. I had a deep dive on decision science in my Ph.D. program at Carnegie Mellon – a program that included both engineering and public policy, but where most of the students came from a technical background. And now, I teach decision-making in public affairs at the John Glenn College of Public Affairs at Ohio State University.

My Ph.D. research focused on risk-based decision-making, which included developing tools to help nonexperts understand complex information about risks and to use that knowledge to inform their sense of priorities among those risks [1, 2] – key decision-making work. Risk ranking initiatives like these have proliferated over the past 20 years, including applications in environmental risks, food safety risks and other domains [3, 4, 5, 6]. This decision-focused work is based on the premise that when there are limited resources to address risks, understanding the relative magnitude of risks is an important first step in deciding where to focus risk-reduction efforts.

As the teaching of decision science grows in public affairs schools, a significant challenge persists in bringing these concepts into government decision-making in part because of the large diversity of backgrounds and expertise throughout government agencies. Staff can have backgrounds in law, engineering, the natural sciences, the social sciences, communications or myriad of other types of training, as well as those formally trained in public affairs. Most government workers have limited awareness of decision sciences as an area of research; in fact, it is an area in which people generally do not seek out support because they feel qualified (or feel that they should be qualified) based on their experience in another domain. For example, a highly experienced engineer would be very comfortable making decisions about engineering issues. And this makes sense in the domain of decision science as well: when you clearly understand the factors that need to go into the decision, and it is not a complex decision, you do not necessarily need to draw on the lessons of decision science. However, for complex decisions, domain expertise is not sufficient.

Complex Decisions

What is a complex decision? Decision researchers characterize complexity as follows:

• decisions that have multiple criteria (things you care about are relative to this decision) and many possible alternatives;
• decisions that have significant uncertainty in their outcomes;
• decisions with competing viewpoints among decision-makers and/or stakeholders;
• decisions with conflicting criteria (e.g., to get more of A, you will have less of B);
• decisions that will have significant (size or time frame) impacts; and
• decisions that will impact many people.

For these types of decisions, using tools and principles from the field of decision science can lead to better decisions. Certainly, there are many public affairs decisions that have these characteristics. As an aside, many personal decisions have these features also. I will explore the first two of these in more detail below.

Decisions that have multiple criteria and many alternatives are difficult. For an example, consider a situation in which a new renewable generation facility is needed for a region [7]. Many people may want the facility near their area with the expectation that it will bring jobs and economic growth to their area. Others may not want it near them because of concerns about environmental impacts or increased traffic. Local control decision processes may conflict with statewide goals. There may be some existing infrastructure that would lead to big differences in cost if it was placed in one location rather than another. How would a decision-maker decide which location is best? Typically, there may be a panel or a workgroup to make a recommendation. But how should this group review available evidence, seek additional evidence, account for conflicting objectives and compare the alternatives? This is where decision science can provide some guidance.

Decisions with significant uncertainty also present unique challenges. In most public sector decisions, the impact of the decision can be estimated but is not certain. Politicians have famously asked for a one-handed scientist to combat the testimony of uncertainty (when a scientist testified, “…but on the other hand, …”). Of course, there is always uncertainty when making estimates of potential outcomes; not formally representing that uncertainty and not accounting for it when making a decision can lead to poor decisions.

Good Decisions

But what is a “good” decision versus a “poor” decision? A basic premise of decision science is that the quality of the decision is not determined by the ultimate outcome. Rather, the quality of a decision has to do with how well it aligns with the decision-maker’s values. This is quite a different perception than the typical view of a “good decision.” Because of uncertainty, the outcome of most decisions, except the very easy ones, are unpredictable. For example, a bet on a lottery that has a 90% chance of paying out may be a good decision, but there is still a 10% chance that the outcome will not be positive. This distinction between a good decision and a good outcome is an ongoing struggle between how decision scientists see the world and how lay people do. But, the good news with the decision science view is that you can control the process, which means you can have a high-quality decision even when you are uncertain about the outcome [8].

How do you create a high-quality, decision-making process? Ensuring that the decision lines up with decision-makers’ values is possible by following basic steps laid out in many popular and technical textbooks and articles [9, 10, 11, 12]. Note that in public sector decisions, since the stakeholders will be impacted by the decision, it would be good decision practice for the decision-maker’s values to include consideration of stakeholder values. Studies have also shown advantages to including stakeholders in broader parts of the decision process; for example, by identifying alternatives that had not previously been considered [13].

General steps for good decision-making based on this literature, adapted from [9], are as follows:

1. Clearly state the problem that needs to be addressed.
2. Work on the right decision problem.
3. Specify the decision criteria.
4. Assess the values (or preferences) on the criteria.
5. Create standard and imaginative alternatives.
6. Assess the consequences.
7. Grapple with needed tradeoffs by comparing the alternatives.
8. Clarify your uncertainties and think about your risk tolerance.
9. Make your decision.

Two notes on this process: First, you are not meant to only move through the process in one direction; you may have to iterate. For example, once you start working on identifying alternatives, you may realize that you forgot to include some important criteria, so you should go back and update the criteria and continue through the process again. And second, as mentioned earlier, for public sector decisions, each of these steps should be done with the decision-maker(s) and the stakeholders, ideally in a shared collaborative and open process.

Common Errors

In my years of federal and state government experiences, I encountered many examples of a less-than-complete decision-making processes. Common errors include:

• not actively re-assessing the problem definition and framing as new information and new perspectives become available;
• focusing on the data that you have in front of you to inform the decision rather than considering what information you think is important;
• leaving stakeholders out of critical steps in the decision process, including defining the problem, defining the decision criteria, and identifying possible alternatives;
• deciding which alternative is preferred independently and then working to convince others that it is the right decision, which is related to …
• choosing an alternative based on your perception of political winds rather than reviewing the data and evidence for each possible alternative.

Although not common practice, following the nine steps of good decision-making outlined above is something that any government decision-maker could do. The biggest barrier right now to improving decision-making with these steps is lack of awareness of decision science as a field of study and the results that have come from that research. I and several others in the Decision Analysis Society of INFORMS are working to help increase the awareness so that hopefully soon the problem will shift to a high demand for these skills and practices from the public sector.

Conclusion

To conclude, “data-driven decision-making” and “evidence-based decision-making” are hot topics these days, but many who use these terms are hard pressed to say what they mean. Now that I have introduced some of the basic precepts of the decision science perspective, I can tell you what that would mean from my perspective. It would mean that research and data are used to inform estimates of the impacts of selecting each decision alternative considered. Good decision scientists are quick to reinforce a fundamental principle in decision science: that no analysis, decision analysis included, can tell you what the best alternative is. Analysis is meant to inform decision-making by providing insights about potential outcomes and uncertainties and by clarifying what the implications of any particular decision could be. Having these tools could very well increase agreement among stakeholders, or at least elucidate exactly where the disagreements are based. As mentioned earlier, they may also help identify new, preferred alternatives. These tools can also be a great help to inform efforts to communicate with people outside of the decision process about why this alternative was selected. In sum, decision analysis approaches can provide structure, consistency, transparency and understanding about public sector decisions, which would benefit the public as well as the decision-maker.

References

1. Morgan, Kara M., Michael L. DeKay, Paul S. Fischbeck, M. Granger Morgan, Baruch Fischhoff and H. Keith Florig, 2001, “A Deliberative Method for Ranking Risks (II): Evaluation of Validity and Agreement among Risk Managers,” Risk Analysis, Vol. 21, No. 5, pp. 923-923.
2. Florig, H. Keith, M. Granger Morgan, Kara M. Morgan, Karen E. Jenni, Baruch Fischhoff, Paul S. Fischbeck and Michael L. DeKay, 2001, “A Deliberative Method for Ranking Risks (I): Overview and Test Bed Development,” Risk Analysis, Vol. 21, No. 5, pp. 913-913.
3. Anderson, Michael R., 2003, “Risk-Based Decision Making for the Remediation of Petroleum Contaminated Sites,” Oregon Department of Environmental Quality.
4. Batz, Michael B., Sandra Hoffmann and J. Glenn Morris, 2012, “Ranking the Disease Burden of 14 Pathogens in Food Sources in the United States Using Attribution Data from Outbreak Investigations and Expert Elicitation,” Journal of Food Protection, Vol. 75, No. 7, pp. 1278-1291.
5. Bu, Qingwei, Donghong Wang and Zijian Wang, 2013, “Review of Screening Systems for Prioritizing Chemical Substances,” Critical Reviews in Environmental Science and Technology, Vol. 43, No. 10, pp. 1011-1041.
6. Schwarzinger, Michaël, Mostafa K. Mohamed, Rita R. Gad, Sahar Dewedar, Arnaud Fontanet, Fabrice Carrat and Stéphane Luchini, 2010, “Risk Perception and Priority Setting for Intervention among Hepatitis C Virus and Environmental Risks: A Cross-Sectional Survey in the Cairo Community,” BMC Public Health, Vol. 10, No. 1, pp. 773.
7. Stein, Eleanor, and Mike O’Boyle, 2017, “Siting Renewable Generation: The Northeast Perspective,” Energy Innovation.
8. Spetzler, Carl, Hannah Winter and Jennifer Meyer, 2016, “Decision Quality: Value Creation from Better Business Decisions,” 1st edition, Hoboken, N.J.: Wiley.
9. Hammond, John S., Ralph L. Keeney and Howard Raiffa, 2015, “Smart Choices: A Practical Guide to Making Better Decisions.”
10. Keeney, Ralph L., 1992, “Value-Focused Thinking: A Path to Creative Decisionmaking,” Cambridge, Mass.: Harvard Univ. Press.
11. Gregory, Robin, Lee Failing, Michael Harstone, Graham Long, Tim McDaniels and Dan Ohlson, 2012, “Structured Decision Making: A Practical Guide to Environmental Management Choices,” “Methods & Statistics in Ecology, Ecology & Organismal Biology, Life Sciences,” Wiley.
12. Winterfeldt, Detlof von, 2013, “Bridging the Gap between Science and Decision Making,” Proceedings of the National Academy of Sciences 110 (Supplement 3): 14055-61.
13. Gregory, Robin, and Ralph L. Keeney, 1994, “Creating Policy Alternatives Using Stakeholder Values,” Management Science, Vol. 40, No. 8, pp. 1035-1048.