The ChanceOmeter

Imagine you are developing a software package that requires four final tests before release. The good news: the tests can run in parallel, and each takes six weeks on average. The bad news: you must announce a release date soon, and due to competition, the sooner the better. Miss your deadline and you’ll have egg on your face – and maybe lose your job. 

The rule that I call the “flaw of averages” guarantees that plans based on averages will be wrong on average.^1-2 Shooting from the hip, what do you think your chances are of delivering a release within six weeks? 100%, 75%, 50%, 25%, 10%, 5%, 1%? 

These are all wrong, but one is close. Click here and find the deadline that gives you a 90% chance of not losing your job. Hit the "Help" button if it is not obvious what to do.

In my experience, most people guess 50%, which seems reasonable, until you consider that all four tests must pass. This makes finishing in six weeks a bit like flipping four heads in a row on a coin. 

Stochastic Data: Conveying Chance

In the 1980s, financial engineers started running huge Monte Carlo simulations of asset growth factors, which were stored and reused in downstream portfolio decisions. The elements of such data are statistically coherent vectors of trials, which I call SIPs (stochastic information packets). Although such simulations might run for hours, once generated, SIPs can be used in downstream applications such as stochastic optimization.3 To simulate the performance of a particular portfolio, just add the associated SIPs together with vector arithmetic to create an SIP of the desired portfolio. No simulation required – just addition. Unlike static statistical data, SIPs contain chances.

Computer scientists refer to data that supports numeric-style operations as “first-class objects.”³ As defined in 1968 by Robin Popplestone, that means:

• They can be the parameters of functions.
• All items can be returned as results of functions.
• All items can be the subject of assignment statements.
• All items can be tested for equality.

SIPs obey all these properties in terms of “vector” calculations. But there is a fifth property that numbers don’t have. SIPs can be tested for the chance of meeting an inequality. For example, the chance that your portfolio will return more than 8% is determined by counting the elements of the portfolio SIP that exceed 8%, and then dividing that by the number of trials.

Foundations

In a February 2006 article in OR/MS Today, Professor Stefan Scholtes of Cambridge University, Daniel Zweidler (then the head of global exploration planning and portfolio for Royal Dutch Shell), and I wrote the foundational article on the discipline of probability management.⁴ The idea was built around stochastic data as discussed above. In our case, the SIPs represented 1,000 simulated trials of 25 years of 20 key metrics of 100 oil exploration projects, for a total of 50 million numbers, which took hours to run.

Shell was able to explore thousands of portfolios by simply adding pre-generated SIPs together. The OR/MS Today article demonstrates the interactive environment in which projects could be swapped in and out, while comparing the chances of meeting various conflicting goals in real time.

The SIPmath Standard

By 2013, Excel had become powerful enough to process SIPs with the built-in Data Table function, and Nobel Laureate Harry Markowitz and I co-founded nonprofit ProbabilityManagement.org to develop open tools and standards to spread the use of stochastic data. The organization served as a catalyst for advances in modeling uncertainty, and some important thought leaders joined us soon afterward.

Doug Hubbard, author of the acclaimed How to Measure Anything book series, created the HDR cross-platform random number generator that fits in a single cell of Excel.⁵ Then Tom Keelin, a prominent Stanford-trained decision analyst, arrived with his elegant Metalog formula, which provides a simple way to quantify uncertainty from past data and mimics a wide class of distributions. I connected the HDR and Metalog with a Copula Layer, and with help from Dan Fylstra, CEO of Frontline Systems, and his team, we developed the Open SIPmath 3.0 Standard in 2021, which can express hundreds of millions of trials in small JSON objects.^6,7

The ChanceOmeter is not just a pie chart on a web app; it is an application that extracts thousands of trials from SIP libraries small enough to be updated continuously. These libraries may be interpreted in any computer environment, including Python, JavaScript, and Excel. The nonprofit has developed an open-source add-in called ChanceCalc, which reads SIPs into Excel files, and thereafter runs thousands of trials per keystroke without the use of the add-in, making them freely shareable across virtually any platform.⁸

AI for All

AI has transformed the discipline of probability management in two important ways: the collection of statistical data and the generation of applications. I created the ChanceOmeter by sending ChatGPT a SIPmath 3.0 library for the four task durations, the graphic file for the interface button, and a plain-language description of the problem. The model generated a functional application directly from those inputs – no manual coding required.

This points to a broader shift: AI is lowering the barrier to building probabilistic tools, making it feasible for practitioners who are not software developers to construct and deploy stochastic models. For a discipline that has long faced the challenge of communicating uncertainty to non-technical audiences, that is a meaningful development.

Twenty Years On

When the foundational article on probability management appeared in this magazine 20 years ago, stochastic data required hours of computation and specialized software to interpret. Today, the same concepts can be packaged in a small JSON object, queried in any programming environment, and visualized in a browser-based tool built with the help of AI. The gap between Wall Street-grade analytics and everyday decision-making has narrowed considerably.

ProbabilityManagement.org continues to develop open standards and tools toward that goal.⁹ The ChanceOmeter and the SIPmath libraries behind it are freely available, and the organization’s webinars offer introductions to the framework for practitioners at any level.

Dr. SAM L. SAVAGE is the executive director of ProbabilityManagement.org, a 501(c)(3) nonprofit devoted to standardizing the communication and calculation of uncertainty. The organization has received funding from Chevron, Kaiser Permanente, Highmark Health, Lockheed Martin, PG&E, and others. Harry Markowitz, Nobel Laureate in Economics, was a co-founding board member. Dr. Savage is author of The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty and Chancification: Fixing the Flaw of Averages. He is the inventor of the Stochastic Information Packet (SIP), an auditable data array for conveying uncertainty. He is an Adjunct in Civil and Environmental Engineering at Stanford University.

References

1. Savage, S.L., 2002, “The Flaw of Averages,” Harvard Business Review, Vol. 80, No. 1.
2. Sam L. Savage, 2009, The Flaw of Averages, Wiley.
3. Popplestone, R., 1968, cited in various computer science literature on first-class objects.
4. Savage, S.L., Scholtes, S., Zweidler, D., 2006, “Probability Management,” OR/MS Today, https://doi.org/10.1287/orms.2006.02.17.
5. Hubbard, D. W., 2020, “A multi-dimensional, counter-based pseudo random number generator as a standard for Monte Carlo simulations,” proceedings of the 2019 Winter Simulation Conference, pp. 3064-3073, https://dl.acm.org/doi/10.5555/3400397.3400642.
6. Keelin, T., 2016, “The Metalog Distributions,” open-source standards at Probability Management, https://www.probabilitymanagement.org/30-standard.
7. Sam L. Savage, “ChanceCalc,” open-source tools at Probability Management, https://www.probabilitymanagement.org/chancecalc.
8. Sam L. Savage, 2026, “Measuring Chance: The ChanceOmeter,” Probability Management. https://www.probabilitymanagement.org/blog/2026/4/14/measuring-chance