Setting Strategic Objectives for the Coalition for Epidemic Preparedness Innovations: An Exploratory Decision Analysis Process

Epidemics of emerging infectious diseases (EIDs) are a growing threat to global health and prosperity. Recent outbreaks of Ebola, Zika, MERS (Middle East respiratory syndrome), and SARS (severe acute respiratory syndrome) have caused significant mortality, morbidity, and socioeconomic disruption across multiple continents (Kieny et al. 2016, Sands et al. 2016). Timely vaccine development can avert humanitarian crises and limit the socioeconomic damage associated with these outbreaks (Coalition for Epidemic Preparedness Innovations 2016). However, safe and effective vaccines for most epidemic infectious disease threats are lacking (Plotkin et al. 2015, Gouglas et al. 2018).

The World Health Organization (WHO) has argued that without coordinated investments, the world will not be able to effectively respond to future epidemics (Kieny et al. 2016). Along similar lines of reasoning, various post-Ebola outbreak reports have called for either vaccine-specific or broader product-focused research and development (R&D) funds (Plotkin et al. 2015, United Nations Secretary General 2016), financing facilities (Moon et al. 2015), partnerships (Norheim et al. 2014), or strategies (Center for Infectious Disease Research and Policy 2015) to improve global preparedness against EIDs in the future.

To address these challenges, leaders from governments, intergovernmental organizations, foundations, industry, and civil society came together in 2016 to explore new ways to drive vaccine R&D for priority EIDs (Brende et al. 2017, Røttingen et al. 2017). Between February and July of 2016, three expert task teams convened to assess problems and potential solutions for (1) pathogen prioritization, product development, and regulatory pathways; (2) partnership models; and (3) financing strategies (Røttingen et al. 2017). Several authors of this article were initially involved in the task team responsible for identifying appropriate partnership models and bringing together all task team policy recommendations into a strategy document to establish the Coalition for Epidemic Preparedness Innovations (CEPI; Coalition for Epidemic Preparedness Innovations 2016).

Task team proceedings revealed the need to identify and agree on a number of well-defined strategic objectives and operating principles for CEPI. Given the large number of stakeholders with diverse perspectives (over 100 stakeholders from industry, government, foundations, and civil society), efforts to generate consensus were at risk for devolving into a social bargaining process that could generate results that were not representative and/or were ambiguous. In an effort to lend transparency, accountability, and clarity to this exercise, we implemented a combination of value-focused thinking (VFT) and an exploratory discrete choice experiment (DCE) to identify, structure, and explore the relative importance of CEPI’s strategic objectives.

An Exploratory Decision-Analytic Approach

VFT is a long-established decision-analytic approach appropriate for identifying and structuring objectives in strategic decision problems (Keeney 1992). Such problems—framed in the management literature as wicked problems of organized complexity (Rittel and Webber 1973, Ackoff 1974, Mason and Mitroff 1981)—reflect states of extreme complexity, whereby the problems and solutions are neither obvious nor easy to agree on (Belton and Stewart 2010); multiple stakeholders are involved with multiple and often conflicting objectives (Montibeller and Franco 2010, Punt 2017); and stakeholder perspectives are negotiated through social bargaining, that is, intense dialogue processes (Thomas 1984, McMillan and Overall 2016).

The premise of VFT is that early and systematic attention to stakeholder values can lead to meaningful descriptions of objectives and justifications on why these are important, including, where possible, associations of their relevance to other objectives. Analytically, VFT defines values within a given decision context as explicit statements of what one wants to achieve (Keeney 1994), distinguishing between two types of objectives: (1) fundamental objectives, which characterize the essential reasons or endpoints for a given decision, and (2) means objectives, which enable the achievement of fundamental objectives (Keeney 1992).

A number of tools and techniques can be applied to distill the relationship between means and end objectives in VFT frameworks, several of which are reviewed in Parnell et al. (2013) and Kunz et al. (2016). Evidence of the method’s application in setting strategic objectives is rich across several domains (Keeney and McDaniels 1992, Keeney 1996, Parnell et al. 1998, McDaniels and Trousdale 1999, Tan et al. 1999, Yoo et al. 2001, Bullock et al. 2008, Morais et al. 2013, Simon et al. 2014, Kunz et al. 2016Abuabara et al. 2017). However, VFT has rarely been explicitly applied in the health space. Its limited application, for example, in evidence-based medicine may be because values have been traditionally seen as sources of bias that can and should be controlled for (Kelly et al. 2015, Neumann and Cohen 2015). This may also be because a great deal of thought has already gone into the concept of value in health, with some consensus as to what values should be achieved (Porter 2010). Consequently, the benefit of using tools to support problem structuring in a priori relatively well-defined problems is expected to be only marginal (Marsh et al. 2016). However, VFT was recently applied to help construct a multicriteria evaluation model for new medicine reimbursement decisions (Angelis and Kanavos 2017), and it has been proposed as an analytical approach to support strategy formulation for healthcare management through means–ends objective structuring (Ginter et al. 2013).

Whereas VFT can help identify and structure objectives for strategic planning, specifying the relative importance of such objectives requires appropriate preference elicitation techniques. There are numerous preference elicitation techniques in the health literature (Marsh et al. 2016). One such methodology is the DCE (Bridges et al. 2011b), which is particularly helpful in the absence of revealed preference data (Mangham et al. 2009). DCEs have become increasingly popular in health valuation (Thokala et al. 2016) and priority setting (Marsh et al. 2012, Franken and Koolman 2013, Grepin et al. 2018). They can be relatively quick preference elicitation instruments (Lagarde and Blaauw 2009), which is an advantage in time-constrained strategic decision-making contexts.

To our knowledge, no explicit VFT approaches have been applied to date to identify and structure strategic objectives of organizations investing in global health R&D. And we are aware of only one other study that has applied a DCE for health systems goal valuation (Franken and Koolman 2013). In combining VFT with DCE, we demonstrate an important role of decision analysis in strategy formulation, where the consideration of multiple objectives and their relative importance can facilitate structured dialogue processes between stakeholders making strategic decisions in global health R&D.

Our article is structured as follows. The methods section provides an overview of the analytical steps—from VFT to DCE methods—undertaken to help CEPI decision makers define and structure strategic objectives as well as determine their relative importance. The results section presents the VFT and DCE findings. Theoretical and practical lessons learnt from the application of the methodology in CEPI context are discussed in the discussion and conclusion sections.

Methods

We undertook four analytical steps to help CEPI decision makers—CEPI task teams, founding partners, and leadership group (Coalition for Epidemic Preparedness Innovations 2016)—define and structure CEPI’s strategic objectives and to determine their relative importance. First, we conducted stakeholder consultations to identify needs, challenges, potential objectives, and benefits of establishing new mechanisms for EID vaccine R&D. Second, we constructed means–ends argument chains from problem statements to fundamental objective concepts relevant to CEPI’s strategy formulation. Third, we refined the results of this objective structuring exercise with CEPI stakeholders through teleconferences and face-to-face group discussions. Fourth, we elicited preferences over alternative strategic objective formulations through a DCE. This section provides an overview of the approach adopted. More details on the methodology can be found in Appendices A and B in the online supplement to this paper.

Stakeholder Consultations

We conducted 31 in-depth, semistructured one-on-one consultations with official representatives of organizations and individual experts comprising members of CEPI task teams. Although there is no correct number as to how many such interviews one should conduct, approximately 30 is the average number of interviews conducted in exploratory, qualitative research before saturation is reached (Mason 2010). The chance of obtaining most possible answers to kick-start the VFT process was maximized by means of a saturation criterion, that is, no new ideas generated after three consecutive interviews per subject-matter expertise or sectoral affiliation (Francis 2010, Saunders et al. 2018). Saturation was reached after 28 interviews. This procedure was intended to increase the baseline content validity of the VFT exercise.

Stakeholders selected for consultation were key partners in the establishment of CEPI and who met at least one of the following criteria:

• had subject matter expertise on epidemic infectious pathogens; vaccine R&D, including nonclinical and clinical development aspects, manufacturing capacity, and regulatory pathways; partnership models; and funding strategies;

• had sectoral representation (industry, government, philanthropic sectors);

• had geographical representation (north–south balance); or

• were in a group likely to be affected by decisions on CEPI operations (i.e., industry, WHO, civil society, representatives of regions likely to be affected by EID outbreaks).

The number of interviewees and the criteria considered for stakeholder inclusion in the consultation process allowed us to ensure a sufficiently broad set of perspectives and informed the effort to identify objectives. Following good practices identified elsewhere in the literature (Keeney 1994, Kunz et al. 2016), all interviews followed the same approach (see Appendix A in the online supplement to this paper) and included questions about

• lessons for R&D partnership building from experiences with recent EID outbreaks in terms of needs and priorities, opportunities, and roadblocks;

• operating principles that should define the space within which CEPI was to operate;

• strategic objectives CEPI should aim for and prioritize to address the needs, opportunities, and roadblocks in this field;

• partnership model alternatives that CEPI should consider; and

• benefits that CEPI should anticipate from the operation of such partnership models.

The questions included in the questionnaire were crafted based on Keeney’s (1992) recommended techniques to identify objectives. Although somewhat redundant in their guidance, these questions were purposefully repetitive to allow us to make implicit objectives more explicit (Keeney 1996) and, in an implicit way, to also test for stakeholder response consistency in a qualitative manner.

In line with good practices (Kunz et al. 2016), we drew, where possible, potentially relevant concepts from the literature to steer discussions with stakeholders toward critical issues previously raised in the literature but that were not addressed adequately during the consultations.

Means–Ends Mapping

The initial consultations generated some results that were not exclusively objectives (e.g., problem statements, preferred partnership models, relevant actors and functions for CEPI, operating principles for CEPI). We separated these concepts and established relationships between them by examining the reasons for each, and, where possible, their implications. This allowed us to determine potentially fundamental objectives and policy values for CEPI, as well as to link these through means–ends argument chains. For a review of means–ends mapping methods, see Belton and Stewart (2002), Montibeller and Belton (2006), Montibeller et al. (2008), and Franco and Montibeller (2011); for further examples of means–ends mapping theory and applications in problem structuring and decision making, see Howard (1988), Belton et al. (1997), Eden and Ackermann (1998, 2013), Bana e Costa et al. (1999), Ensslin et al. (2000), Rosenhead and Mingers (2004), Bryson et al. (2004), Eden (2004), Ackermann et al. (2007), and Rodriguez et al. (2017). We depicted these objectives as a network of concepts connected by links denoting chains of arguments within and between seven reasoning clusters:

• Problems: What are the perceived problems for R&D partnership building from experiences with Ebola and other recent EID outbreaks? Why are these problems important, and what are the potential implications if these problems remain unaddressed?

• Actors: What actors can address these problems and why?

• Functions: What types of functions could and should these actors offer, including resource assets or other types of competencies and capabilities?

• Alternative models: What modes of action or partnership approaches could and should these actors establish to provide these functions?

• Priorities: Which of these modes of action or partnership approaches are most important and why?

• Expected benefits: What are the expected benefits associated with each of these partnership approaches, and why?

• Objectives: Why are these anticipated benefits important?

Although the final question listed here may not quite sound like an objectives-focused question, it is important to highlight that one often begins to think hard about fundamental objectives after some benefits become apparent as well as the reasons why these are likely to be important (Keeney 1996). Articulating the features that distinguish revealed benefits provides, therefore, a sound basis for identifying fundamental objectives within a VFT framework, ideally with such a question being logically structured toward the end of the discussion process.

Based on interviews with CEPI stakeholders, we initially identified 464 concepts and 1,274 relationships between these. After clustering the concepts and their relationships into means–ends chains of arguments according to the above procedure, we generated a reasoning map with 62 concepts and 251 means–ends argument chain connections. Redundancies of previously reported concepts were eliminated from this map (see details in Appendix B in the online supplement to this paper). In addition to serving as a practical consistency check between stakeholder responses, this last step also helped us bridge the theoretical gap between strict assumptions on attribute properties commonly required in multiattribute valuation methods versus the desired flexibility in structure and fewer modelling assumptions commonly observed in causal mapping (Montibeller and Belton 2006).

Group Discussions

A series of teleconferences, email exchanges, and face-to-face meetings (Kristensen 2016) took place with a broader set of stakeholders to validate the results of the initial consultation exercise and to clarify CEPI’s potential objectives and policy values, which would determine the context and goal orientation for CEPI’s strategy formulation. These discussions led to the specification of a provisional hierarchy of preferentially independent means–ends objectives as well as a set of policy values—such as operating and governance principles—that set the overall frame within which appropriate definitions of strategic objectives would be obtained.

Discrete-Choice Experiment

A DCE was employed to elicit stakeholder preferences among objectives and to combine these into an overall probability of attractiveness associated with alternative strategic objective formulations. DCE participants were given a series of choice sets in which they were asked to choose between strategies defined by the level of importance by strategic objective. Strategy attractiveness against each strategic objective (attribute in the DCE) was defined as one of three levels, reflecting the level of importance for a strategic objective within the strategy formulation (see Table 2). Given the time constraints on the analysis, the three levels of performance were based on initial stakeholder consultations and definitions that were derived from these. Table 2 summarizes the strategic objective definitions (attributes) and importance levels.

Following good practices in DCE implementation (Ryan et al. 2008, Mangham et al. 2009, Johnson et al. 2013, Hauber et al. 2016), an experimental design of two blocks of 9 choice sets (i.e., 18 choice sets in total) was generated using SAS JMP^® Pro 12 software (SAS Institute Inc. 2016). The software generated 1,000 alternative designs so that we could select the most optimal design based on the D-efficiency statistic. The orthogonality of the selected design was assessed based on the correlations in the covariance matrix. The highest correlation in the covariance matrix was 0.5, and the average correlation was 0.003. Manual edits to this design were made to remove any dominant choice sets and, in doing so, to improve the balance of the design.

Two other choice sets were added to each of the two blocks of choice sets: a dominance test and a consistency test. The survey was administered online using Questback Essentials^®. The order of the 18 experimental choice sets within these sets was randomized between participants.

The survey was sent to 72 recipients: members of the three CEPI task teams and the leadership group; see “Annex 3: List of CEPI members” in CEPI business plan (Coalition for Epidemic Preparedness Innovations 2016, pp. 57–59). Where multiple persons represented a single organization, a survey invitation was sent once and consolidated responses were requested for these organizations. The survey was completed by 55 respondents, representing over 100 individuals engaged in CEPI’s establishment. Figure 1 shows an example of a question included in the survey.

To assess the contribution of attribute performance to strategic objective preference, a dummy-coded, linear, conditional logistic regression was applied using JMP, version 12 (SAS Institute Inc. 2016). This type of analysis is a well-established and suitable approach for modelling discrete choices through the estimation of the probability of individuals making a particular choice from presented alternatives (McFadden 1974). Here, the utility for each choice option depends on the criterion levels defining that option. Therefore, it is not the characteristics of the DCE participants that are modelled, but the choice options.

The results of the model were used first to estimate the overall statistical significance of the attributes considered in the DCE (i.e., logworth values and likelihood ratios). Conditional on these overall attribute significance findings, the results of the model were then used to estimate the main effects of the different attribute levels; see the parameter coefficients in Table 4. Given the nature of the model and total number of survey responses received, the statistical significance of each attribute level was calculated using the Wald statistic, which is asymptotically distributed as a standard normal distribution (Wasserman 2006). The most desirable strategy formulation was identified as the one with the highest utility, defined as the sum of all statistically significant parameter coefficients associated with attribute levels in the model. Finally, the probability of different strategy formulations being preferred was estimated, for each alternative, as the ratio of the expected utility to the sum of this expected utility and the expected utility of the most preferred strategy (i.e., the baseline comparator).

Results

Our findings demonstrate that the prioritization of preparedness and market predictability objectives is likely to generate the most-supported vaccine R&D strategies against EIDs only if some importance is also placed on equity and response speed objectives.

Table 1 summarizes needs and potential objectives for EID vaccine R&D partnerships as prompted by the literature up to February 2016, aimed to facilitate ideation fluency in stakeholder consultations. For a comprehensive reporting of stakeholder input, see Table B.1 in Appendix B in the online supplement to this paper. Needs range from fully dedicated and centralized approaches to highly flexible coordination approaches between existing actors. Potential objectives vary from increasing the level of R&D preparedness around R&D, manufacturing, and regulatory processes to improving institutional response speeds to EID outbreaks, improving incentives for private sector participation, and ensuring access and trust in vaccines through affordable pricing and regional R&D capability strengthening in countries likely to be affected by EID outbreaks. Figure 2 illustrates constructed means–ends argument chains from perceived problem statements to anticipated benefits that enabled the structuring of stakeholder objectives and values; see Table B.1 in Appendix B in the online supplement to this paper for a full mapping of means–ends argument chains. It demonstrates that CEPI stakeholders perceived the sporadic and unpredictable emergence of EIDs and the lack of coordination and cooperation frameworks to address these as the greatest challenges in efforts to improve global health security associated with EID epidemics. They argue that vaccine R&D can contribute to better EID outbreak preparedness. However, they flagged many problems that would need to be resolved, such as misconceptions about the value of vaccines, lack of interest and infrastructural capacities to support R&D, large R&D complexities and costs, and low willingness for information access and sharing to support vaccine development, testing, and emergency use. Stakeholders predicted different types of actors—such as vaccine developers, funders, governments, regulators, and international expert organizations—could tackle several of these problems by serving different functions. These could include financing and incentivizing R&D, sharing data and know-how, scoping disease threats and setting R&D priorities, managing R&D efforts and building R&D capabilities, raising awareness of the critical issues, and improving global levels of stakeholder engagement in this space. Such functions could be provided through institutional partnerships and networks between product developers, regulators and governments, and clinical trial partners.

Table 1. Various Needs and Potential Objectives for EID Vaccine R&D Emerged from Consultations and the Literature

Clarity on operating principles around equitable access, cost, and risk sharing would be needed regardless of the coordination mechanism chosen. Such an organizational design should aim to generate a pipeline of vaccines for priority EIDs, build technical and institutional capabilities that can accelerate vaccine development and manufacturing in response to newly emerging disease threats in the future, minimize disruptions for vaccine developers, and improve the distribution of R&D capabilities and financing responsibilities across geographical regions. Stakeholders perceived the ultimate objective as securing the world from future EID outbreaks becoming humanitarian and economic crises.

Figure 3 summarizes the strategic objective hierarchy and values for CEPI identified through the objective identification and structuring process. This analysis suggests that to achieve its mission, CEPI should consider four objectives:

• improve R&D preparedness through the development of vaccines to the latest R&D stage possible, complemented by other translational R&D milestones and regulatory innovations;

• improve the speed of R&D response through the availability of manufacturing capacity on-demand, clinical infrastructure to test vaccine candidates, and rapid-response vaccine platform technologies for EIDs;

• improve market predictability through the generation of positive externalities to businesses and to the public, the minimization of disruptions to other business or public health work, and the availability of incentives for vaccine developer engagement in EID vaccine R&D;

• improve equity through the availability of vaccines to priority populations, the strengthening of low- and middle-income country (LMIC) capacity, and the promotion of shared responsibility in financing across geographical regions.

According to stakeholder preferences, equitable access, cost coverage, and risk/benefit sharing are principles, or boundaries within which they would like to see CEPI strategy operationalized; for details, see CEPI business plan (Coalition for Epidemic Preparedness Innovations 2016).

Table 2 presents the criteria definitions and their associated levels of importance considered in the DCE, because these were derived through the objective structuring process.

Table 2. A Number of Attributes and Levels of Importance Were Used in the DCE

Table 3 presents the overall effect and statistical significance of the strategic objectives (attributes in the DCE) on the attractiveness of CEPI’s strategy. As these figures demonstrate, all four strategic objectives included in the DCE are significant and should be considered in the formulation of an attractive CEPI strategy.

Table 3. The DCE Generated a Number of Overall Effects and Statistically Significant Attributes

Table 4 presents the independent effect that different importance levels assigned to each strategic objective would have on the overall attractiveness of CEPI’s strategy, based on the DCE. We define strategy attractiveness as a function of the level of importance placed on preparedness, response speed, market predictability, and equity objectives, and the means objectives’ targets associated with each of these ends and their importance levels. As the parameter estimates in the table suggest, placing low or high importance on preparedness and on market predictability would have a strong positive effect on the attractiveness of CEPI’s strategy. In contrast, placing low importance on equity and on response speed would have a positive effect on strategy effectiveness but a diminishing and statistically uncertain effect if they were given high importance.

Table 4. The Relative Importance of Strategic Objectives Varied in CEPI Strategy Choice

Based on the dominance and consistency tests included in the survey choice sets, 95% of DCE survey respondents appear to have provided a consistent response and 80% of them correctly addressed the dominance question. When the probability that the dominance question was preferred was modelled based on the choice model (Tervonen et al. 2018), we estimated that only 35% of respondents would be expected to select the dominant option, suggesting that DCE respondents attended to the task.

Figure 4 presents the predicted probabilities associated with formulating a desirable CEPI strategy given different combinations between low and high levels of importance of the strategic objectives and comparing these to a baseline strategy. These results indicate stakeholder preferences for the strategic objectives assessed. Specifically, 16 alternative strategy formulations were ranked based on their likelihood of being considered attractive. The baseline comparator was a strategy that places high importance on preparedness and market predictability, and low importance on response speed and equity. The least attractive strategy is one that places low importance on all strategic objectives. There would be approximately a 61% chance that a CEPI strategy would be desired if high importance was placed on all objectives, ignoring statistical significance values. And there would be a 10% chance that a CEPI strategy would be desired if low importance was placed on all objectives.

Given the statistical uncertainty around high importance levels preferred for equity and response speed, and accounting for objective definitions (Table 2), the above results suggest that priority should be given to the development of vaccines to the latest phase possible and at least through the end of Phase IIa (i.e., clinical safety and immunogenicity studies in humans), complemented by enabling science and regulatory innovations. Priority should also be given to generating incentives for vaccine developers and minimizing disruptions from engaging in EID vaccine R&D. Furthermore, at least one of the following activities should be prioritized under the response speed objective: developing rapid-response vaccine platform technologies, ensuring the availability of manufacturing capacity, or strengthening clinical infrastructure to test EID vaccines. And at least one of the following activities should be prioritized under the equity objective: ensuring availability of EID vaccines to priority populations, strengthening LMIC capacity for vaccine R&D, or promoting shared responsibility in financing across geographical regions.

Discussion

This study demonstrates how decision analysis can support a rational and transparent approach to strategy formulation that accounts for and ranks the preferences of multiple stakeholders in an international health policy setting. There are three key lessons and implications that can be drawn from the study. First, it is possible to combine rigorous problem structuring and quantitative preference elicitation methods to support strategy development and objective setting in a highly complex R&D planning context with many diverse and strong interests. We show how the VFT approach can be used to identify and structure stakeholder values to clarify strategic objectives in global health R&D when the diversity of stakeholder perspectives and the complexity of decision making are both high. Furthermore, the application of the DCE demonstrates how it can be used to elicit preferences over difficult strategic choices prior to their implementation.

Second, as the global governance structure for outbreak response continues to emerge, it will be important to apply these techniques to elicit clear strategic objectives and means that will frame the desired response, and in doing so, to improve EID mitigation efforts. Given the large number of stakeholders with different and sometimes competing objectives, there is a danger that more widely held values and strategic objectives can be hijacked or lost through an interest-heavy social bargaining process. The application of value-based thinking and choice trade-offs can rationalize and democratize this process in the future.

Third, decision analysis can be implemented in a dynamic way, allowing it to adapt to rapidly changing decision-making contexts. It is important to demonstrate this quality to maintain confidence in its practical, supportive function (Keeney 1996). For example, a new R&D investment strategy to combat antimicrobial resistance would likely require a different set of objectives and would suggest a different structuring of means and ends and their respective trade-offs, even if objectives appeared to be the same in name (imagine how many different meanings equity can have in health; Mooney 1987). This should reduce the generalizability of decision-analytic outcomes across different settings. However, the utility of decision analysis rests mainly on its methods for value structuring and elicitation, which, if applied appropriately, can justify differences in content while ensuring analytical rigor and transparency in a variety of management decisions.

What is Known on This Topic

Decision analysis can enhance transparency and offer an explicit measure of comparison among options to promote rational decision making. This attribute is well documented in the strategic management literature (Montibeller et al. 2006) even when social bargaining processes tend to otherwise dominate (Thomas 1984, Montibeller and Franco 2010). Even though the application of decision analysis for strategic goal setting in health is limited (Franken and Koolman 2013, Ginter et al. 2013), evidence from other sectors suggests that at least half of all strategic decisions fail as a result of poor decision-making processes (Nutt 2002, Bryson 2018). Premature dispute resolution and consensus building approaches can prevent choices from becoming apparent to decision makers and therefore promote inferior, internally inconsistent policy choices, in the absence of decision-analytic approaches that structure and address the relative importance of stakeholder values (McDaniels and Trousdale 1999, Abuabara et al. 2017). Organizations that satisfy key stakeholders’ values are more likely to enhance the legitimacy of their strategies (Ackermann and Eden 2011). However, the impact of decision analysis on commitment to action, it has been argued, cannot be proven easily at an empirical level (Montibeller and Franco 2010). Moreover, decision analysis can lose its meaning if the skills, resources, or commitment of stakeholders are lacking when engaging in deliberative strategic planning (Bryson 2018).

The evidence base from previous research and practice on strategic objectives for EID vaccine R&D is limited. Following the 2015–2016 Ebola outbreak in West Africa, the WHO created a R&D Blueprint for EID preparedness and response coordination at the global level (Kieny et al. 2016). Recent outbreaks (Ebola and subsequently Zika) also revealed that few reward systems are in place to compensate companies for the costs they incur in responding to these outbreaks, lending weight to other preexisting disincentives to private sector participation such as poor commercial prospects, uncertain regulatory pathways, and a lack of preestablished operating principles for coordination (Kamal-Yanni 2015, World Economic Forum 2015, Lucey and Gostin 2016). Equity is an important concept and common objective in global health financing organizations, such as the Bill & Melinda Gates Foundation (2018), Wellcome Trust (2018), and Global Alliance for Vaccines and Immunisation (2018). This principle has been well addressed in strategies of product-development partnerships and global health R&D initiatives in the endemic, poverty, or neglected disease space; examples include the Drugs for Neglected Diseases Initiative (2009), Medicines for Malaria Venture (2017), and Program for Appropriate Technology in Health (2018).

What This Study Adds

Our study attempts to overcome some of the challenges identified in the literature in two ways. First, definitions of preparedness and response objectives were constructed, especially as they relate to vaccine development. Moreover, this is the first time that equity and market predictability concerns for EID R&D have been explicitly addressed at the level of strategic priority setting.

Second, the systematic structuring of values and some quantitative thinking about value trade-offs has brought strategic decisions stemming from CEPI’s social bargaining processes closer to actual commitments for action, as reflected in CEPI business plan and actions taken thereafter. On one hand, CEPI’s leadership group and founding partners endorsed these means–ends strategies as part of CEPI’s business plan launch in late 2016, after numerous formal decision-making forums and deliberations informed by different versions of the decision-analytic findings (Coalition for Epidemic Preparedness Innovations 2016, Kristensen 2016, Brende et al. 2017). Not all quantitative data presented in this article (e.g., Table 4 and Figure 4) were presented in detail to the decision makers, because of both cognitive burden concerns and a perceived risk of diverting too much attention from social bargaining in an extremely time-constrained environment. However, overall analytical outcomes—such as logworth values, likelihood ratio statistics, and overall utility functions of the most preferred strategy formulations (Table 3)—were reported and offered stimuli for discussions around policy values and fundamental objectives.

In addition, since CEPI’s official launch (Reuters 2017), the organization has issued three separate investment opportunities under the just-in-case preparedness and response speed (just-in-time preparedness) objectives: two calls for proposals (CfPs) to support vaccine development against five priority EIDs (Coalition for Epidemic Preparedness Innovations 2017, Christodoulou 2019) and a CfP to support the development of rapid, multipurpose vaccine platform technologies (Coalition for Epidemic Preparedness Innovations 2018). All CfPs are supported by decision-analytic frameworks that are aligned with CEPI’s strategic goals (Gouglas and Marsh 2019, Gouglas et al. 2019). Under the just-in-case preparedness objective, CEPI has also been advancing efforts on standardized assays and regulatory pathways for emergency use through various working groups in close collaboration with the WHO (Gouglas et al. 2019). Under the equity objective, advocacy and resource mobilization efforts are under way to improve the equity in EID vaccine R&D financing across geographic regions. CEPI is working closely with several partners to improve the long-term predictability of financing EID vaccines, including through stockpile commitments, among other examined market incentives (Gouglas et al. 2019). Under the platform technology CfP, CEPI is also working with industry to leverage positive technological externalities to other vaccine areas, thus contributing to CEPI’s commitment to the market predictability objective. Retrospectively, this evidence of CEPI’s commitment to action comes in contrast with the prevailing skepticism in the literature about the lack of impact that decision analysis can have on strategic decisions in practice.

Limitations

VFT is only one of many problem-structuring techniques in decision analysis (Leon 1999, Belton and Stewart 2010, Marttunen et al. 2017). VFT assists with strategy setting because it clarifies stakeholder preferences and objectives in ill-structured decision problems (Keeney 2008, Montibeller and Franco 2010). It does not, however, enhance perceptions of the course of future events that may impact decision making, which other techniques may be better suited to stimulate (Kunz et al. 2016).

The elusive and often conflicting nature of value statements can prevent them from conforming to the classical concept of goal hierarchy that is also used in VFT (Wenstøp and Myrmel 2006). Moreover, VTF can be mentally challenging (Arvai et al. 2001), may require time and effort to be understood (Kunz et al. 2016), and can become complex in its visualization (Becker et al. 1995). A wider range and creative use of problem-structuring tools may therefore be required to identify and understand the interaction of stakeholder values within an overall analytical frame of means–ends objectives (Kunz et al. 2016). The application of a number of tools presented in our study, including evidence drawn from the literature, semistructured interviews, group discussions, and means–ends mapping, demonstrates how their use can help VFT specify and structure the objectives and then use them to inform the decision process.

This study attempted to address several drawbacks associated with DCEs. First, a systematic approach to criteria development in DCEs is generally lacking (Helter and Boehler 2016). When cognitive shortcuts are used or erroneous interpretations are made of criteria and their preferential relationships, DCEs can generate unreliable inputs for policy decisions (Ali and Ronaldson 2012). Our study has attempted to address this limitation through the use of a rigorous method to identify and structure criteria prior to DCE design.

Second, DCEs require precise criteria definitions, and ambiguity in the specification of their assessment levels can lead to less realistic or meaningful analytical outcomes (Hall et al. 2004, Ryan 2004, Mangham et al. 2009). Drawing directly from the results of the problem-structuring process, the specifications of criteria levels in our model reflected the early maturity of the organization. As CEPI strategy becomes more focused over time, future decision-analytic exercises should improve the specificity of criteria descriptions and assessment levels and should consider additional trade-offs between subcriteria where preferential independence between these is observed.

Third, the sample size for the DCE was small in relation to many DCEs commonly found in the literature (de Bekker-Grob et al. 2012, de Bekker-Grob et al. 2015), which may have influenced our findings. Given the sample size, number of choice sets, alternatives, and criteria levels included in the design, the DCE presented here was viewed as explorative (Baltussen et al. 2006). Although there is no agreement on what the minimum sample size or method for calculating this in the DCEs should be (de Bekker-Grob et al. 2015), our working assumption during the DCE design was that as few as 20 respondents should suffice to estimate broadly reliable preference data in exploratory DCE contexts (Orme 2010, Lancsar and Louviere 2008, Bridges et al. 2011a). We received 25 responses to Version 1 and 30 responses to Version 2 of our survey. Using the Johnson and Orme rule of thumb (Johnson and Orme 2003, Orme 2010), which is the most commonly applied minimum sample-size calculation rule in DCEs in healthcare (de Bekker-Grob et al. 2015), the minimum sample size required for both survey versions was met, given the number of attributes and levels included in the design.

Stakeholder preferences varied moderately, as reflected in the standard deviation estimates (Table 4), even after accounting for bias effects in the design of the survey. Preference variation was most evident around placing high importance on market predictability, response speed, and equity objectives. This variation was statistically significant in the case of market predictability but not so for response speed and equity objectives. Perhaps a larger sample size in the future could give a more definitive answer as to the expected coefficients and associated variation on high levels of importance for response speed and equity objectives.

Finally, we should acknowledge some additional, practical limitations with the method’s application. First, while the DCE allowed for a rough ordering of objectives in the face of preference variation, the methodology did not remove this variation, and it was important to engage stakeholders in thoughtful discussion without too much emphasis on quantitative data, given practical time constraints. On one hand, this meant a missed opportunity for validating stakeholders’ preference inputs into the DCE, which is generally considered a good practice in the decision analysis literature (Salo and Hämäläinen 2010, Montibeller and Winterfeldt 2015, Marsh et al. 2016). On the other hand, considering the practical constraints—cognitive burden, sample size, and timeline limitations—strategic decision-making processes are not always amenable to rigorous preference elicitation.

Second, in the context of sample-size limitations, as is often the case when working with expert and decision-making committees, there are limitations on the complexity of the value models that can be characterized by choice models, such as DCEs. However, this will be less of a concern when stakeholders whose values are of interest are a larger group, such as patients or the general population. Nevertheless, a DCE was employed because the expert and decision-making groups were quite representative of the global community relevant to EID mitigation, and logistical limitations meant that it was necessary to elicit preferences using a survey. This decision was vindicated by the results of the choice analysis, which was sufficiently precise to allow us to differentiate preferences associated with many of the levels in the choice sets. Other preference elicitation methods could also be employed, such as workshop-based swing weighting (e.g., Phillips and Bana e Costa 2007); however, such methods are generally restricted by practical constraints of time, location, and availability of stakeholders engaged.

Conclusions

The analysis reported in this study demonstrates the use of an exploratory decision analysis process to support the identification, structuring, and prioritization of strategic objectives of a new organization aimed at improving global R&D preparedness against EID epidemics. The systematic structuring of values and some quantitative thinking about value trade-offs helped CEPI stakeholders explicitly agree on a commonly preferred set of strategic commitments for action, as reflected in CEPI’s business plan, despite differences in their perspectives. In doing so, the analysis has provided a strategic narrative upon which the organization still bases its investment objectives, as reflected by several major funding opportunities issued over the past three years.

More broadly, our study highlights how formal decision analysis supports priority setting for international strategic initiatives with multiple stakeholders. It provides analytical rigor to problem structuring and preference elicitation, increasing the level of transparency and explicitness of complex strategic decision processes and outcomes in global fora. In settings where large numbers of stakeholders with conflicting objectives prevail, negotiations can devolve into social bargaining processes that do not accurately reflect the strategic objectives perceived as important by all stakeholders. As new strategies and governance structures for global health continue to emerge, it will be important to apply such techniques to elicit clear strategic objectives through democratic means. The application of value-based thinking and choice trade-offs can rationalize and balance strategic decision-making processes in the future.

Verification Letter

Dr. Frederik Kristensen, Deputy CEO, CEPI – Coalition for Epidemic Preparedness Innovations, Marcus Thranes Gate 2, 0473 Oslo, Norway, writes:

“On behalf of CEPI, I hereby confirm that the decision analytic approach presented in the article submitted for your consideration and titled ‘Setting Strategic Objectives for the Coalition for Epidemic Preparedness Innovations: An Exploratory Decision Analysis Process’ was employed in 2016 to inform the establishment of CEPI’s interim Business Plan and strategic objectives.

“The methodology employed was beneficial for structuring a complex dialogue process between multiple stakeholders with conflicting perspectives and helped determine the nature and structure of CEPI’s interim strategic objectives in an analytically rigorous way.”