Change Management

“There is nothing permanent except change. All is flux, nothing stays still.” – Heraclitus

“It is not the strongest or most intelligent who will survive, but those who can best manage change.” – Charles Darwin

Despite the unassailable veracity of these age-old adages, most people, in general, don’t appear to be getting any more adept at or comfortable with accepting, embracing or effectively dealing with change.

Technology, in particular over the past 50 years, has dramatically accelerated the pace of change in business. From robots building cars to meetings being conducted via video conference to AI-based systems streamlining, automating and optimizing large-scale complex decision-making and problem-solving better, faster and more effectively than any human ever could, change is daunting for most people. The fear that jobs and livelihoods will be eliminated can be damaging to the psyche.

The simple fact is that data science is disruptive. The sheer volume of data and the dynamism and complexity of business decision-making and problem-solving mandate the use of automation and mathematical logic and intelligence. However, when we “let the data speak,” inconvenient truths are revealed. Gut instinct and heuristic “rule of thumb” planning, decision-making and problem-solving processes that sufficed for decades are now invalidated and outmoded by new and improved fact-based, data-driven and model-based solutions. Solution-embedded models, be it in a robot or a business system, radically change the way we work, make decisions and solve problems. This in no uncertain terms threatens the human beings who are used to doing things “their way, the way they have always done it for 30 years with their 150-tab Excel spreadsheet” (no exaggeration, this is an actual fact from a project I worked on).

Change management is without a doubt one of the most critical dimensions of successfully executing a data science project. The data scientist must gently, diplomatically and ever so delicately win the hearts and minds of the businesspeople who will be instrumental in designing, developing, testing, validating, approving, deploying and ultimately using the new system. If you are unable to convince and bring those folks along the journey, then you will lose their support, and your project will fail with 100% certainty. Period.

It is not enough to tell them that your “black box, math-magical computer system” is better, faster and more economical than “the old way.” You need to show them, step by step, from beginning to end and case by case by making them an integral part of the process and not the recipient – or worse, the victim – of it. Change can be a painstakingly slow process as businesspeople move through the full range of emotions and reactions to the new solution, including (not dissimilarly to the five stages of grief) denial, anger, bargaining, depression and acceptance (hopefully). You can throw in outright rejection of the new solution in favor of the incumbent. 

“It must be remembered that there is nothing more difficult to plan, more doubtful of success, nor more dangerous to manage than the creation of a new system. For the initiator has the enmity of all those who would profit by the preservation of the old institution and merely lukewarm defenders in those who would gain by the new one.” – N. Machiavelli [1]

Principles of Change Management

The first principle in change management is empathy. Be kind to the businesspeople who are your stakeholders and constituents. Your tone, body language, mannerisms and communication style all need to soften the sharp edges of the pure rationality and logic of the data, math and code. Put yourself in their position and look at the situation from their perspective. This goes back to Part 5 on Effective Communication – “first seek to understand, then be understood.” Position and present yourself as a colleague on the same team, not the enemy trying to disrupt their processes. Data science is intended to make things better, economically for all parties, not just be more change for change’s sake. 

It is important to remind folks that data science, artificial intelligence and machine learning (AI/ML) are more about augmentation than replacement (for the foreseeable future, anyway). The “human-in-the-loop” working interactively and iteratively with the model, not being replaced by it [2].

I learned all about change management firsthand in 1990 on my first project to build a new decision support system from scratch for American Airlines to schedule aircraft heavy maintenance checks and plan hangar capacity for a five-year planning window. Historically, the airline with 200 aircraft created their long-range five-year heavy maintenance and hangar plan on large sheets of paper using colored pencils, driven by calculator computations of when aircraft would be due for their respective checks. When the fleet rapidly grew to 600 aircraft, the paper-pencil-calculator solution became unwieldy and untenable, so the analysts (two at the time, later three total) switched to Excel macros. Unfortunately, the macros sometimes took as long as 10 hours to run to completion, for the larger sub-fleets, on an Apple Macintosh IIcx desktop computer (Motorola 68000 chipset), and many times they errored out prior to completion. Senior management quickly grew impatient and very, very concerned as the mystery loomed of when a new hangar might need to be built, and check yields were bleeding down to a suboptimal 80%, increasing heavy maintenance costs [3].

An industrial engineer at the maintenance base had done an analysis that demonstrated whether a system could be built to automate the maintenance check and hangar planning and scheduling process and optimize the schedule to maximize the check yields to ~100%. (Two heavy maintenance checks, $1 million each or $2.4 million today, could be avoided for each of the 227 wide-body aircraft over the life span of that sub-fleet, for a staggering cost avoidance of $454 million, or $1.09 billion today!)

A project was authorized to build the system as described, and I was assigned as the project manager and O.R. analyst to build the scheduling model and algorithm (i.e., a greedy heuristic based on job scheduling on parallel machines with firm due dates written from scratch in C), along with a software engineer who built a color-coded Gantt-chart GUI for the analysts’ Macintosh computers that emulated their wall-hanging paper schedule charts of old.

As you might imagine, there was quite a bit of skepticism from the analysts about the ability of a computer to generate higher-yield, more efficient five-year maintenance check and hangar schedule plans better than they could. Their skepticism turned first to incredulity and then quickly to unmitigated fear and dread when my partner and I delivered an early version of the software within a few months that, with the right input parameters (running on a Mac IIcx desktop computer), could generate a five-year, 600-aircraft fleet maintenance and hangar plan schedule with optimized ~100% check yields in about 18 minutes (a process that used to take two or three people weeks to generate one feasible plan with 80% check yields)!

At that point, the analysts took me aside and said something to the effect of, “You are going to put the three of us out of work with that computer program of yours!” I not only assured them that was not the case but also predicted (and bet them a steak dinner) that they would all get promoted as a result of their ability to use the new system to create more efficient, cost-effective maintenance plans in a far timelier manner than before.

Interactive Optimization

As it turned out, the system we created was very much a case of (AI) augmentation rather than replacement. The system employed a design framework (conceived at Georgia Tech in the late 1980s, where I earned my M.S. degree) known as interactive optimization. The approach combines prescriptive optimization-based techniques, including heuristics when appropriate, and an evaluative simulation-based approach to quickly generate optimized schedules interactively with a human-in-the-loop iteratively providing the necessary inputs and feedback to guide and push the algorithm in the right direction toward an optimized solution. Therefore, the human and system work together, leveraging their respective strengths to generate better solutions quickly that neither would be able to deliver on their own. Humans can more easily inspect a graphical Gantt-chart representation of the schedule and see where hangar capacity needs to be added or excess capacity taken away to optimize check yields. A computer can add and subtract, albeit really quickly, and store information, and an algorithm can be programmed to automatically generate maintenance plans and schedules the same way a human would, but far faster.

Suffice it to say the project was a success. My software engineer and I, both working full time, delivered the first production version of the system in about six elapsed months (12 labor-months) and demonstrated how we would achieve the originally targeted benefits over time, i.e., $454 million in maintenance cost avoidance through increased wide-body aircraft check yields, along with multiple additional unforeseen benefits. By optimizing yields and, in effect, pushing aircraft maintenance events out later in time, but still within the FAA’s legal limits, the analysts used the model to open up additional hangar space, which allowed:

1. Aircraft maintenance work that had been contracted out to a third party, owing to a perceived lack of in-house hangar capacity, to be brought back in-house and avoid incremental costs.
2. American Airlines to bring in maintenance work from other airlines that didn’t have ample hangar capacity (and that work was done at a profit).
3. One narrow-body aircraft to be returned to the fleet and revenue-generating service for a period of one year after an entire maintenance line was deemed superfluous and then shut down.

The analysts not only received promotions but also became a valued, trusted resource to the executives, including the senior vice president, as a result of their ability to “see into the future” with confidence and accuracy and to evaluate all manner of various planning scenarios with the new model/system that they never could have dreamed of doing before.

Keys to Success

What were the key change management factors that made the project a success? There were clear goals, objectives and a well-defined project scope, including a tangible business value target. To start with, I personally spent the first 6 weeks of the project literally sitting and working side by side with the analysts at the maintenance base in Tulsa, Oklahoma, learning about and understanding the art and science of scheduling aircraft maintenance and hangar facilities, the data and decision-making until I could do the job myself. I listened two-thirds of the time and asked questions the other third.

As a team, we had regular status and update meetings every time my partner and I hit a noteworthy milestone and deliverable (what today we would call Agile-Minimum Viable Product) at each stage of development of the model, algorithm and schedule GUI. There was ample two-way communication – i.e., we demonstrated what we had done in detail, and the analysts provided constructive feedback and guidance to validate the model’s performance and results. I continually reassured the analysts that the system was designed not to operate “completely autonomously” but rather for them to operate it and “drive it” iteratively and interactively, much like a driver directs an automobile with inputs from the gear shift, accelerator and brake pedals, and steering wheel to reach their destination.

The changeover from a cumbersome, manual, spreadsheet-based process to a streamlined, automated and interactively optimized process was orchestrated to reduce fear of and instill confidence in the new solution. We endeavored to make the transition to the new system as seamless and stress-free as possible by reusing all of the same data, terminology, scheduling logic, KPIs, report formats and familiar visualization tools in software GUI, such as the Gantt chart from the historical wall-hung paper maintenance and hangar schedules. That way, the learning curve on the new system was not very steep at all.

The interactive optimization approach, based on the analysts’ own step-by-step processes, also made the analysts feel much more comfortable with the solution, rather than being a “black box” that they didn’t understand. One of the analysts even referred to the new system as “a big calculator” that could enter the input data and output an optimized five-year maintenance schedule and hangar plan. A great metaphor indeed [3]!

Justification for Change

“One of the things I have learned well from many real-world modeling engagements is that finding the supposedly ‘optimal’ solution is often not nearly as important as putting the solution values into a form that the client is accustomed to seeing.” – R.E.D. “Gene” Woolsey, Ph.D., Professor, Colorado School of Mines and Operations Research Academic, Practitioner & Consultant

The best way to “grease the skids” of change management is to deliver significant, tangible, measurable business value that can be categorically attributed to the new model/solution as demonstrated by before-and-after experiments. We did that in this case, and the “after scenario” delivered far more and better solutions faster than the analysts could have ever imagined. This made for a super easy justification for change. It’s rarely that easy, but sometimes it can be. (Promotions, raises and escalation of one’s status in the organization goes a long way toward acceptance!)

I went on to use this exact same approach multiple times during my career, including once at another airline to build a new jet fuel supply chain purchasing and inventory management optimization system. As with the maintenance scheduling scenario, the science and technology were sophisticated and substantive, leveraged augmentation versus replacement, and got the job done. That said, it was the “soft skills” that really made the difference. The jet fuel supply chain business folks were quite attached to their 150-tab Excel spreadsheet that they had been using for 30 years, and they did not necessarily want to trade it in for a “new and improved” data-driven, analytically based forecasting, purchasing and inventory optimization model suite. In fact, they initially put up quite a fight. However, when the results of a head-to-head “bake-off” between the spreadsheet and the new models were validated, the supporting case for the new models/system was made: an eight-figure annual cost avoidance opportunity generated by the models in a matter of minutes, versus days and weeks by the status quo process!

The outcome was quite similar with significant business value and economic impact, and satisfied business stakeholders benefitted from a continuous close engagement with my team from the start of the project. A smooth, seamless transition and a change management process focused on large doses of communication, mutual understanding and empathy, and iterative testing and validation made all the difference.

Notes and References

1. Machiavelli, Niccolò (1469-1527), 1981, “The Prince,” New York: Penguin Books.
2. Davenport’s “The AI Advantage” (2018) does a great job of explaining this concept.
3. Doug Gray, 1992, “Airworthy: Decision support for aircraft overhaul maintenance planning,” OR/MS Today, pp. 23-29.