Do You Need a Prediction or a Prescription?

Machine learning and predictive analytics immediately capture the imagination as people try to figure out how to use data to grow their business. “What would we do differently if we knew what was coming?” is a common question in the world of data analytics. However, there are a number of challenges with this problem framing that often reduce the success of data projects. In this article, you will learn about some of those challenges and a better approach to leveraging data.

First, two key definitions:

• Predictive analytics is focused on making a prediction. This is often a prediction of what will happen in the future (a forecast). But it can also be a prediction of an unknown value based on other values (e.g., whether a credit card transaction is fraud or legit). Many “machine learning” and “artificial intelligence” problems fall into this category.
• Prescriptive analytics is focused on prescribing an action. Typically, this involves creating a model of your system and the effects of possible actions. These actions often include preparing for the future (staffing/inventory choices). But they can also reflect a trade-off of side effects given unknowns (e.g., Do we deny or approve this credit card transaction?). The fields of “optimization” and “operations research” tend to study this domain.

In general, there has been a lot of focus on predictive analytics. As an example, “analytics maturity frameworks” typically suggest that companies who have already nailed descriptive analytics can grow into predictive analytics. Then, they should only layer in prescriptive analytics once they get good at predictions. However, what you will notice about predictive analytics problems is that they only impact the business when they have been translated into an action. A perfect prediction does nothing for your organization until it has actually changed something.

This is not to suggest that predictions are useless. After reading this article, you will understand the role of prescriptions and how the two can be used together to drive value for your organization.

Going from Data to Action

The key reason analytics maturity frameworks place prescription after prediction is perfectly valid. If there is a problem with the prediction, there is an opportunity for some process to compensate because a prediction by itself does not change anything. Data generally has many inaccuracies and gaps unless your organization has already spent substantial effort solving those issues. Furthermore, if you are trying to make a decision based on incomplete information, it seems to make sense that the first priority should be to improve that information. Maybe you should start with predicting anything that is unknown yet key to making your decision.

By contrast, a broken prescription seems useless. If we ask our analytics what to do based on inaccurate or unrelated data, what benefit can it provide? Unfortunately, this framing glosses over the fact that predictions guide actions. As previously mentioned, a prediction that never contributes to an action cannot improve your business. Fraud detection is classified as a predictive analytics problem; however, it is only useful if you then deny transactions that are predicted to be fraudulent.

This might seem fine because these predictive algorithms have been found to be accurate enough to link them to actions. In business, when predictions are not accurate enough, they are often presented instead as the input to a human decision-making process. A prediction that is properly trained, tested and validated will have a quantifiable “accuracy” that engenders trust in the output. It also helps to define progress for future development as data scientists try different features, algorithms or tuning parameters.

But what if we defined accuracy in terms of the actions, not the predictions? Depending on the link from prediction to action, this translation may be simple or very challenging. In the simple case, the benefits of measuring accuracy in terms of action are clear because we will be able to measure in business terms – that is, dollars of uncaught fraud or number of transactions incorrectly denied. When the link is muddy from prediction to action, it is not clear how to fairly assess the prediction. However, actions are often more forgiving than predictions. Ultimately, the measure of success for an analytics tool should be, “How did this improve my business?” Anything less is selling yourself short.

To illustrate this point, consider the problem of predicting how much a house is worth. Getting an accurate prediction is fraught with data challenges, including complex markets, changing trends and text-based features. However, a natural use of a house value prediction is to take an action of making an offer on a house. In the context of that action, a prediction is good if it improves profits. Although improved prediction accuracy can help, the context of how it will be used is critical to the value of the effort. Done poorly, prediction accuracy may be phenomenal on houses that are entirely irrelevant to your business and terrible on the ones you care about.

Fundamentally, the world is a highly uncertain place. Almost any prescription you want to make will depend on data inputs that are in the future or must be inferred (i.e., they would benefit from a prediction). However, just because it would be useful to know the future before you buy a lottery ticket does not mean you should invest effort into a “lottery prediction algorithm.” You can instead simply ask, “Should I buy a lottery ticket?” When focusing on the prescriptive problem, it is suddenly possible to quantify the value of improved information against the cost of getting it.

Putting it into Practice

The first question you should ask yourself before starting a data analytics project is “What data do we have that is relevant to our goals?” At this point in the project, you must decide what data team to bring together to solve your business problems. In a world where you have an enterprise data warehouse team that can quickly respond to new data needs from the business, it makes sense to focus on building algorithms. By contrast, an organization that has not yet worked out the kinks of defining things – such as “How many active customers do we have now and have we had at any point in the past?” – should ensure that experienced data engineers and architects form the backbone of the team. A helpful heuristic is to think about how you would train a human to do the work you are hoping an algorithm will do for you. If that training process would be unpleasant, you probably have some work to do before you can try to train an algorithm.

Once the correct team is assembled per your analytics maturity, it makes sense to clearly define the “why” of your project. Everyone on the team should be able to articulate what would change for your business if the project were to be successful. This is also your key opportunity to ensure that definition of success includes the action, not just the inputs to the action. A helpful thought experiment to distinguish between prediction and prescription is to imagine you have an oracle and can make perfect predictions. What actions would you take and how would they be different than in an uncertain world?

With a clear set of goals for the project comes the fun of the analysis phase. “Proof of concept” and “minimum viable product” are key concepts that can help your team prioritize competing elements of problem-solving. This is also the phase of the project to test competing ideas for how you might achieve your business objectives. Depending on the reality of your data and how you are trying to use it, this phase can be very quick or much slower. The quicker you can prove that the data you have can solve your business needs, the sooner you can quantify the potential return on investment (ROI) of your project.

Finally, even though this article is focused on the benefits of prescriptive analytics, in reality, people are often reluctant to give control to an algorithm. Moreover, that reluctance is frequently justified due to gaps in the data or qualitative business rules. Given that awareness, choosing the right way to add your analytics solution to the business process is critical to realizing the ROI. Nontraditional prescriptive solutions such as a “what if” calculator can be extremely effective both for communication with your users and to smoothly integrate into business processes. With involved stakeholders, you can begin to fill the gaps between your current model and the future state version where users will begin to hand over control.

Conclusion

Data is often limited in how completely it describes the things you most care about for your business. Predictions help fill the gap from your data to the things that drive decisions. However, there remains a gap from prediction to action, which can be filled in part with prescriptive analytics. By framing your business problem all the way from data to action, you can more effectively drive business impact.