December 14, 2020 in Software Survey

Decision Analysis Software Survey

Biennial survey emphasizes more robust decision analysis software capabilities.

SHARE: Share on PRINT ARTICLE:print this page https://doi.org/10.1287/orms.2020.06.04

Since its founding in 1980, the Decision Analysis Society of INFORMS “promotes the development and use of logical methods for improving decision-making in public and private enterprise.” Still a novel management technique at that time, decision analysis has evolved from straightforward statistical decision theory [1] into a holistic, multidisciplinary suite of methods that effectively combine data and expert judgment with consideration to uncertainty and risk preference.

What was once a limited technique rarely used outside the classroom, decision analysis has blossomed into a dominant analytic discipline expanding across industry domains. Decision analysis is part of the crosscutting technical management methods within the systems engineering handbook at NASA. Multi-criteria decision analysis (MCDA) is now being applied to numerous environmental studies, such as end-of-life sustainability of plastics [2], cost/benefit analysis on the reduction of carbon emission levels in metropolitan areas for stable economic growth [3] and decision-making in forest management [4]. As technology and computational power has grown exponentially over the last few decades, application of decision analysis in concert with other analytic tools has become widespread, such as incorporating GIS capabilities [5].

Naturally, as knowledge of decision analysis has fostered change within the industry, decision analysis tools have become more robust. Innovative decision analysis software components in the past, such as portfolio decision-making and sensitivity analysis, have become standard. Decision analysis software now aids not only the expert user, but novice users as well with problem definition, model structuring and training components. But the journey is by no means complete. We continue to see the need for growth in decision analysis software, expanding usability features and visualizations, and incorporating third-party tools.

This year’s software survey highlights tools that are providing an increasing capacity to effectively define, structure, solve and communicate the complex decision framework.

The Survey

The 2020 decision analysis (DA) software survey begins with an online questionnaire sent out to vendors who participated in previous DA surveys, along with any new vendors identified by INFORMS staff.

Vendors who did not participate in the 2020 decision analysis software survey by the Nov. 13 deadline may fill out and submit a questionnaire and it will be added to the online survey results.

The survey’s intention is to provide an exhaustive – or at least, as exhaustive as possible – comparison of decision analysis software for the current and prospective users’ benefit. As such, there is no preference provided here – just an objective, descriptive overview of what the software industry has to offer in the field of decision analysis.

The survey remains relatively stable each year, with minor updates as new technologies, approaches and/or features become popular within the industry. For example, the 2020 survey includes a new field within the usability features section: visual programming (i.e., code-free workflows). This type of usability feature is becoming prevalent in many data mining and predictive analytics software products, both vendor (e.g., Alteryx) and open source (e.g., Orange) alike, and is essentially the option to use widgets/tools/visuals to perform traditional code-type algorithms. We investigate if this capability has made its way into the decision analysis space through this survey.

Along with usability features, the survey asks other questions related to the capabilities of the software. This includes decision analysis applications, such as multiple competing objectives, risk tolerances and evidential reasoning; elicitation of decision analysis components, such as value functions, probabilities and strategy tables; visualization features, such as graphical sensitivity analysis for weights and probabilities, decision trees and influence diagrams; and other pertinent capability categories like decision algorithms and group elicitation. Training offerings and recognized certifications for each software tool were new questions in the 2018 survey that were carried over to 2020. 

2020 Results

The 2020 decision analysis software survey features 17 software packages from 12 vendors. Companies from the United States, United Kingdom, New Zealand and Poland are represented in this year’s survey. These companies provide decision analysis software to a variety of industries, including healthcare, finance, energy and transportation. Some of the specific applications that rely on these packages are long-range planning in industries such as mining and petroleum, high-risk project analysis within drug development, and environmental risk analysis. Although participation was down this year compared to 2016 and 2018, there is a representative sample, and we could see additional input in the online version of the survey. The following sections provide a brief overview of the survey results according to three survey categories.

Decision analysis applications: The most common applications available are multiple competing objectives (existing in 92% of the tools reviewed), uncertainty (100%) and risk tolerance (75%). The portfolio decision-making application is a major improvement from the 2018 survey, which increased from 52% to 75% in 2020. Unfortunately, the evidential reasoning application (e.g., Bayesian belief networks) is still low, only available in 17% of the tools reviewed. All 11 decision analysis applications in the survey are represented in at least two tools, with an average of being available in at least eight tools. This provides potential user flexibility when choosing decision analysis software.

Usability features and visualizations: Common usability features are the ability to interface with other software (75%), import data (83%), export data (92%) and copy/paste/move model components (75%). There is still a greater need for API, XML and tools for group elicitation, as all these features are only represented in 50% or fewer of the tools reviewed. As posited above, we do see a significant number of tools providing visual programming capabilities (58%), following the current trend in other analytics software.

For visualization capabilities, the graphical interfaces for sensitivity analysis and analytical results, and interactive interfaces that allow users to adjust analytical components, remain dominant in the tools reviewed. The visualizations that remain sparse are expected value tornado diagrams, decision trees and influence diagrams. As noted in the 2018 survey paper, relying solely on decision analysis software is secondary to the applications and usability features, because there are many tools available specifically for visualization. Furthermore, the common usability features (e.g., export data) provided in the reviewed tools support a flexible transition from decision analysis software to visualization software. We will see in future years whether decision analysis software begins to develop their own visualization components rather than relying on other software.

Licensing and training: The licensing component for the tools reviewed range from free demo or student licenses to a few thousand dollars for commercial licenses. Of the tools reviewed, 83% have a demo version and 92% provide a student version. Once you get into the enhanced/high-performance and commercial license, there is still a lot of flexibility in licensing. For example, some products have different licensing options based on single-user developer, multi-user developer or model consumer categories, discounts for volume licenses, and desktop versus network licenses.

Most of the tools provide vendor training; just over half provide classroom training; and about a one-third provide 3rd party training. Surprisingly, only 75% of surveyed tools provide online training resources. Although this percentage has grown, we could possibly see this tally increase even more by 2022 as the shift to online learning continues to grow. 

Conclusion

The 2020 decision analysis software survey provides users a snapshot of what decision analysis applications, usability features, algorithms, and licensing and training are available. The tools reviewed indicate improvements from the 2018 software survey, such as:

  • re-engineered cloud capabilities to work with Azure, Tableau and Power BI,
  • enhanced stakeholder preference tools,
  • the ability to estimate probabilities from external data and
  • the possibility to program decision tree parameters and outcomes via JavaScript.

Ultimately, we continue to see a trend in the available decision analysis software of combining multiple analytic frameworks, such as machine learning, simulation and risk analysis, and optimization; intuitive graphical interfaces and visual programming; and increased emphasis on idea generation and decision framing. Decision professionals continue to have a growing number of innovative decision analysis tools at their disposal to fit their needs.

Editor’s note: All product information has been provided by the vendors and has not been independently verified.

References

  1. Rex V. Brown, 1970, “Do managers find decision theory useful?” Harvard Business Review, May-June, p. 78.
  2. Paritosh C. Deshpande, Christofer Skaar, Helge Brattlebo, Annik Magerholm Fet, 2020, “Multi-criteria decision analysis (MCDA) method for assessing the sustainability of end-of-life alternative for waste plastic: A case study of Norway,” Science of the Total Environment, Vol.719, https://doi.org/10.1016/j.scitotenv.2020.137353.
  3. Ghaffar Ali, Sawaid Abbas, Yanchun Pan, Zhimin Chen, Jafar Hussain, Muhammad Sajjad, Aqdas Ashraf, 2019, “Urban environment dynamics and low carbon society: Multi-criteria decision analysis modeling for policy makers,” Sustainable Cities and Society, Vol. 51, https://doi.org/10.1016/j.scs.2019.101763.
  4. Esther Ortiz-Urbina, Jacinto González-Pachón, Luis Diaz-Balteiro, 2019, “Decision-making in forestry: A review of the hybridisation of multiple criteria and group decision-making methods,” Forests, Vol. 10, No. 5, 375, https://doi.org/10.3390/f10050375.
  5. Narjes Mahmoody Vanolya, Mohammadreza Jelokhani-Niaraki, Ara Toomanian, 2019, “Validation of spatial multicriteria decision analysis results using public participation GIS,” Applied Geography, Vol. 112, https://doi.org/10.1016/j.apgeog.2019.102061.

Survey directory and results

For a directory of decision analysis software vendors who participated in this year’s survey and the complete survey results click here.

Jared Beekman
([email protected])

Jared Beekman is an analyst with Innovative Decisions International, LLC (www.innovativedecisions.com).

SHARE: Share on

INFORMS site uses cookies to store information on your computer. Some are essential to make our site work; Others help us improve the user experience. By using this site, you consent to the placement of these cookies. Please read our Privacy Statement to learn more.