Why Data Science Projects Fail: Practical Principles for Data Science Success

Data science is by far one of the hottest technology domains and job markets ever witnessed on a global scale. Chief information officers (CIOs) surveyed by Gartner consistently rank data and analytics as one of their highest strategy and planning priorities. Gartner reported that the global analytics and business intelligence software market reached $21.6 billion in 2018 [1]. This market offers extraordinary business value and economic impact that corporations can realize by leveraging data about their customers, suppliers and internal operations, combined with advanced mathematics and (cloud) computing technology.

High-tech companies, such as Google, defined as “analytical competitors,” use data science aggressively throughout their entire enterprise to sharpen operational performance and efficiency and improve customer experience in their retail and online search businesses, respectively. Companies like American Airlines pioneered the use of data and analytics in the field of revenue (yield) management in the 1980s to generate $400-$500 million in incremental revenue annually. UPS saves $300-$400 million annually with its On-Road Integrated Optimization and Navigation (ORION) application that guides their 55,000 delivery truck drivers every day. Walmart generates millions of dollars in value annually by applying predictive and prescriptive analytics to optimize its markdown pricing strategy. (The project is actually a 2023 Franz Edelman Award finalist.)

Despite these genuine success stories and many, many more examples, according to a study by Deloitte Analytics and Tom Davenport, only 20% of data science models built are actually deployed into a production system supporting a business process. Gartner reported that through 2022, only 20% of analytic insights will deliver business outcomes. This means that organizations are investing billions of dollars in analytics with minimal return – hardly a recipe for success [1].

Why do most data science projects fail to get deployed and deliver the desired business value, outcome or economic impact? A series of monthly articles will fundamentally focus on this question. We will explore and explain the organizational and individual behaviors and factors that contribute to most data science project failures, which must be addressed and consciously practiced to increase the likelihood of success. Spoiler alert: The problem is not with the mathematics and technology but rather with the actions of the people (practitioners and leaders) engaged in and the processes employed to execute and manage data science projects.

As a long-time proponent of Stephen Covey’s “The 7 Habits of Highly Effective People” [2], I find two of his habits particularly useful in the endeavor to create more successful data science projects:

1. Begin with the end in mind. Analytics expert, researcher and author Tom Davenport said, “Models make the enterprise smarter; models embedded in systems and business processes make the enterprise more economically efficient.” This should be your end goal when starting work on a data science project. You don’t want to just build a model; rather, you want to embed that model into a mission-critical system that supports a key business process such that greater economic efficiency (i.e., lower cost, greater revenue, improved customer experience) can be achieved on an ongoing basis in an automated manner with little or no human intervention, creating a flywheel effect generating business value.

2. Sharpen the saw. Abraham Lincoln once said, “If I had six hours to cut down a tree, I’d spend the first four sharpening the saw.” Most undergraduate and postgraduate education program coursework in data science (and related fields) is spent focused on mathematics and computer science methods, skills and technologies. Although this is understandable because 1) considerable training in these domains is necessary to become a data science practitioner and 2) this is what university staff know how to teach, students enter the workforce unaware of the more nuanced, subtle and harder-to-grasp aspects and dimensions of executing, managing and leading data science projects in the real world and corporate America. My intent here is to help students, practitioners, leaders and executives “sharpen the saw” and fill in the knowledge gap in their training and education that heretofore was learned only through real-world work experience.

This article series is a compendium of my own experiences, and observations of other practitioners, from more than 30 years as a practitioner, leader and executive in corporate operations research, analytics, data science, data and software engineering, e-commerce, and consulting organizations and as a business analytics and data science educator/researcher at Southern Methodist University’s Cox School of Business and other continuing and professional education programs. Given the low success rates of data science projects to deploy and deliver value, I felt compelled to share what I and others have learned with the goal of helping practitioners and leaders be more successful more frequently and avoid many of the common pitfalls associated with these endeavors.

Originally, I presented this material to the Data Science Community of Practice inside Walmart Global Tech in 2021, and then again in April 2022 in Houston, Texas, at an INFORMS Business Analytics Conference in the Leadership Track under the title “The Top 10 Reasons Data Science Projects Fail.” About 100 people attended my talk, in a room with a capacity of about 70, and the feedback was so overwhelmingly positive that several people said to me, “You should really write all this information down!” That particular invited speaker address was the genesis of this series!

The objective of the material in the series is to help make you a more well-rounded, self-aware and informed data science practitioner and leader by learning from the experiences gained by others in the field that came before in the spirit of “fail fast and learn.”

Although I utilize “data science” as a contextual delimiter, the series’ principles apply equally to related adjacent fields that utilize data and mathematics to model and solve for business problems and phenomena, such as operations research/management science, statistics, analytics, machine learning, artificial intelligence (AI), business intelligence (BI) and more.

Samuel Smiles famously said that we learn more from failure than we do from success. My approach therefore was to examine some of the primary reasons I have observed data science projects fail – a top 10 list, if you will – to highlight to data science practitioners the aspects and dimensions of their projects that are more subtle, less tangible and more difficult to grasp, but no less critical, and of which they need to be more conscious to generate successful outcomes with greater consistency and regularity.

“We learn wisdom from failure much more than from success. We often discover what will do, by finding out what will not do; and probably he who never made a mistake never made a discovery.” – Samuel Smiles

The series will comprise 10 articles, each tackling a different reason why data science projects fail and how to address them.

Now, join me on the journey to find out why data science projects fail and learn how to avoid making the same types of mistakes.

References

1. K. Troyanos, 2020, “Use Data to Answer Your Key Business Questions,” Harvard Business Review, February 24, https://hbr.org/2020/02/use-data-to-answer-your-key-business-questions.
2. Stephen Covey, 1989, “The 7 Habits of Highly Effective People,” New York: Free Press.