Analytics and Optimization Reduce Sewage Overflows to Protect Community Waterways in Kentucky

Introduction

Safe and clean drinking water and sanitation are essential to health and the full enjoyment of life (United Nations General Assembly 2010). The water and wastewater needs of most people in North America are served by publicly owned water and sewer utilities. However, the average person does not give much thought to where all the dirty water goes after washing dishes, doing laundry, or taking a shower. But someone has to think about it; for the residents of Louisville, Kentucky, the staff members at the Louisville and Jefferson County Metropolitan Sewer District (Louisville MSD) have assumed this responsibility.

The Louisville MSD staff works to achieve and maintain clean, environmentally safe waterways for a healthy and vibrant community of more than 750,000 residents. The organization’s more than 650 employees provide wastewater management and drainage and flood protection services across the 376 square miles of the Louisville metropolitan area. In addition to operating and maintaining the area’s wastewater collection and treatment system, and its flood protection and stormwater system, Louisville MSD has invested nearly $925 million in infrastructure improvement projects over the past five years. Each year, it also plants more than 1,000 trees and other vegetation to enhance water filtration and reduce runoff, and it provides numerous outreach programs to inform and educate the community about protecting the waterways. The Louisville MSD staff works around the clock—24 hours a day, every day of the year—to achieve safe, clean waterways for its community.

Tetra Tech, a leading global provider of consulting and engineering services, has been working with Louisville MSD in various capacities since the 1990s. The services it provides include planning assistance, manhole and sewer inspection, rehabilitation design, and construction support. Tetra Tech was selected by Louisville MSD to evaluate and help implement an innovative strategy: utilizing existing infrastructures and real-time control (RTC) to reduce combined sewer overflow (CSO) and sanitary sewer overflow (SSO) discharges while reducing the costs of conventional infrastructure and achieving better operational and environmental goals.

The history of sewage treatment in North America is relatively short. The first sewer systems in the United States were built mostly as combined sewers that carried both stormwater and sewage. They discharged the wastewater into rivers, lakes, and the sea without providing any treatment to the wastewater. The first large-scale sewer systems in the United States were constructed in Chicago, Illinois, and Brooklyn, New York, in the late 1850s; construction in other major U.S. cities followed (Burian et al. 2000). Wastewater treatment as we know it today started much later.

The nearly 1,000-mile-long Ohio River has the second-highest volume of discharged water of all rivers within the United States. Situated on the banks of the Ohio River, the city of Louisville has had a long history of urban and suburban development since the early 1800s. As Louisville grew and began industrializing, running water inside buildings became available. Town residents no longer needed to use chamber pots and outhouses for sanitary waste disposal. Potable water was being used at a faster rate, which meant a more efficient method of removing the waste was also needed to minimize exposure to diseases such as typhoid. Pipes were installed to connect sanitary waste straight into the piped creeks; the waste was mixed with stormwater drainage as the combined wastewater was delivered to creeks and rivers without treatment. In 1958, the Morris Forman Wastewater Quality Treatment Center (WQTC) became the first operational treatment plant in Louisville, and it remains the largest in Kentucky. Originally, this treatment plant was designed and constructed to treat only “dry-weather” sanitary waste. Diversion structures were built inside the pipes to divert the correct amount of flow to the plant for treatment. In dry weather, all the flow would flow through the diversions; however, as rainwater made its way into the pipe and mixed with the sanitary waste, the level of flow inside the pipe rose. The diversion structure was sized to allow flow up to a specific amount to enter the plant for treatment. The wastewater exceeding this amount was directed to the waterway as before. This was done to prevent flooding of the homes and businesses connected to those pipes. The excess flow became known as CSOs. These CSOs would contain sanitary waste from, for example, households, industry, hospitals, and slaughterhouses, as well as rainfall runoff (i.e., stormwater) from roads and other surfaces that contain pollutants such as heavy metal and hydrocarbons. The CSOs would be released straight into waterways and flow downriver to the next cities’ drinking water intakes.

Technical, economic, and social challenges are inherent in the management of water in an urban environment to address sewer overflows and also maintain adequate clean water for the protection of public health and the environment. When a large rainfall event inundates wastewater and stormwater system capacities, it can lead to problems such as urban runoff, pollution, sewer overflows, localized flooding, drinking water contamination, and erosion. Moreover, these challenges are exacerbated by a changing and uncertain future climate, an accelerated urban population growth, impervious surfaces, aging infrastructures, limited resources, and increasingly complex water quality issues.

In the past, sewer system collection and treatment-capacity challenges were addressed mainly by investing in the construction of new infrastructure, such as sewer separation, new storage facilities, and system expansions. However, the capital expenditures for these traditional solutions are becoming cost prohibitive, and feasibility and space availability for construction in urban environments are becoming more limited.

Other challenges facing utilities relate to efficient operation and workforce development. Sewer networks are generally managed by licensed operators, who often base their flow-management decisions on limited information and on their relevant experience, which may vary from one operator to another. Collection and treatment systems are often operated in silos, which reduces the efficiency and the performance of the overall system. Finally, operators can only react to the rapidly changing system conditions in wet weather and historically have had only limited tools to manage the system to avoid unnecessary sewer overflows into local waterways.

Louisville MSD has the added challenges of operating and maintaining a very complex and extensive wastewater and stormwater system as well as the Ohio River flood protection system. The sewer systems consist of both sanitary sewers, which transport only sewage, and combined sewers, which collect rainwater runoff, domestic sewage, and industrial wastewater in the same pipe. The system has more than 3,200 miles of wastewater collection sewer lines, 75,000 manholes, 5 regional wastewater treatment facilities, more than 280 pump stations, and 27 long-term monitoring network locations for tracking stream water quality. The Louisville MSD service area encompasses 11 watersheds in the Ohio River Watershed spanning approximately 385 square miles. Figure 1 presents a map of the Louisville MSD sewer system.

A portion of the separated sanitary sewer area is a tributary to the combined sewer system (CSS) area. All the dry-weather flows from the CSS are conveyed to the Morris Forman WQTC to remove pollutants before discharging to the Ohio River. During wet-weather conditions, when the capacity of the CSS is exceeded, the excess flow, a mixture of sewage and stormwater runoff, is discharged to the forks of Beargrass Creek and the Ohio River through 101 active combined sewer overflow outfalls. There were nearly seven billion gallons of average annual overflow volume per typical year prior to this project’s implementation. These overflows were due to several issues, such as the lack of a storage and treatment capacity, operation and maintenance, and an aging infrastructure, which stem back to the construction of the first sewers in Louisville during the early 1800s. This paper focuses mainly on improving the operation of flow management during wet-weather conditions. For Louisville MSD, there is much interconnectivity within the sewer system and flow limitations from local sewers and pump stations. At any time, an operator could be required to make a decision to open or close a flow diversion gate to route flow to one treatment plant or another, or to modulate a flow regulator gate or a sewage pump to control a desired flow rate into an interceptor sewer. With limited real-time information about the sewer system condition and upcoming rainfall, operators often manage the system conservatively and rely on their experience to make decisions. Sometimes, overflows would occur when the wastewater system still had available capacity in terms of conveyance, storage, or treatment.

Louisville MSD Waterways Protection Commitment

Despite the enormous challenges, Louisville MSD has never been deterred in its commitment to advocate on behalf of the community for public health, safety, and protection and to continually improve the environmental quality of local waterways. Since the late 1990s, it has annually updated its five-year capital improvement program (i.e., list of projects) in order to implement its long-term control plan (LTCP) to address CSOs. Louisville MSD strives to “think outside the box” and “find a better way.” Its staff embraces technology in the development of solutions and designs to go beyond the status quo of traditional CSO controls.

Overflow Control Solutions Considered

Louisville MDS’s LTCP also utilizes many infrastructure solutions, such as (1) preventing stormwater from entering the sanitary sewer; (2) building multiple storage basins near CSO outfalls to capture the overflow before it gets to the waterway, holding it, then slowly releasing it back into the system for treatment; and (3) increasing the capacity of treatment facilities to handle the additional flows.

Some of the oldest sewers in Louisville were designed as storage tunnels capable of continuously carrying wastewater and stormwater during rain events. Louisville MSD was an early adopter of RTC technology in the United States, using these large tunnels as in-line storage facilities. The first installation was its Snead’s Branch relief drain, equipped with an inflatable dam under local reactive control (U.S. Environmental Protection Agency 2004).

RTC can be broadly defined as a system that dynamically adjusts the operation of wastewater and stormwater facilities in response to online measurements in the field to maintain and meet the operational objectives, during both dry- and wet-weather conditions (U.S. Environmental Protection Agency 2006). Although the operational adjustment is not necessarily instantaneous to the online measurements because of data collection and communication delays, and operational changes must be limited to prevent excessive wear and tear of equipment, RTC is an acceptable term in the wastewater industry and does not imply that an immediate real-time response (e.g., a response within seconds) is necessary. The conventional design practice of wastewater systems is conservative and includes significant safety factors that at times result in the oversizing of assets. In addition, because of spatial variation in rainfall distribution, runoff response, and flow travel time, unused capacity in conveyance, storage, and treatment is often available for short periods. This presents opportunities to optimize the full utilization of system capacity through operational strategies. Both existing and proposed facilities can benefit from RTC. Potential benefits include receiving water quality protection, flow equalization, reduced flooding, energy savings (Tan et al. 1988), integrated operations, and better facility planning (Gonwa et al. 1993).

One of Louisville MSD’s core values is its accountability to ensure public trust through transparency, financial responsibility, and stewardship. The use of RTC technology presented an opportunity to maximize its available funds and thus keep billing rates affordable for rate payers. Louisville MSD partnered with Tetra Tech to investigate a more sophisticated RTC strategy to maximize system-wide sewer capacity and further reduce overflow abatement costs.

Innovative Features of the Csoft Software Used for RTC at Louisville MSD

Tetra Tech started the development of Csoft^®, an RTC-specific software solution, in the mid-1990s to provide decision support to efficiently manage sewer networks globally based on rain forecasts, model predictions, and sensor readings (Grondin et al. 2002).

The operations research (OR) and analytics that Tetra Tech has integrated into the Csoft software for the Louisville MSD RTC system go beyond solving an optimization problem. The solution harvests the unused capacity in the Louisville sewer system and utilizes it for the benefit of the community. This may seem like a simple challenge—use millions of gallons of excess capacity in the sewer to store wastewater during a storm. But this capacity is spread across 3,200 miles of underground pipe throughout the city. Not only does the solution route the wastewater flows optimally, much like traffic signals route traffic, it also takes into consideration storm sizes, such as long slow rainfalls and rapid thunderstorms, and the path of the resulting water flowing through the system. An operational model represents this complex pipe system and its operation.

The sewer system conditions are monitored constantly, and data are collected and centralized every five minutes. The weather forecast information is also collected every five minutes. The optimization problem is launched rapidly and solved every five minutes with a prediction horizon of six hours. This provides Louisville MSD with the ability to adapt operational decisions for gates and pumps to current sewer system condition changes and to anticipate changing weather conditions in the future.

The solution has a framework to maximize the use of existing and future infrastructures in an application where models, measurements, and load prediction inaccuracies significantly affect the decision-making process. Tetra Tech configured Csoft specifically to meet the following requirements for Louisville MSD’s RTC system:

• • achieve a level of performance that reduces flood risks and CSOs during rainfall events while taking climate changes into consideration;

• • provide flexibility to meet various operational objectives;

• • reduce the computing times to adapt to the expansion of Louisville MSD’s RTC system over the years; and

• • ensure robustness for the safe and reliable operation of the sewer system.

In 2000 and 2001, Tetra Tech collaborated with Louisville MSD to conduct two RTC feasibility studies. The studies identified potential RTC sites, such as a pump station, a storage facility with one or multiple gates, flow regulation or diversion structure, or in-line storage using gates and/or dams, which temporarily store the flow in a large existing sewer pipe. The cost-benefit analysis from the studies showed a relatively low unit cost ranging between $0.006 and $0.021 per gallon of CSO reduction per year by maximizing the existing collection and treatment system. The unit cost refers to the associated cost of adding the RTC layer to retrofit the existing facility to capture more flow, resulting in the reduction of CSO volume. Examples of these costs include monitoring, programmable logic controllers (PLCs), gate actuators, pump variable frequency drives, and Csoft. This unit cost is 4–10 times lower than that of constructing traditional storage solution alternatives. In particular, the RTC solution is well adapted to the complexity of the Louisville MSD sewer system, which has multiple large facilities with available capacity for control, flow transfer ability through existing diversion chambers, variable flow travel time between control sites, and heterogeneity of rainfall over the service area. During rainfall events where all system capacities are maximized, Csoft has the ability to prioritize where overflow occurs based on the water quality sensitivity of the receiving waterways. For example, CSO is prioritized to overflow into the Ohio River with a much larger flow volume than a local creek that runs through residents’ backyards and local parks.

Balancing Model Accuracy with the Need for Fast Run Times

In the technical literature, most RTC applications for the management of combined sewer systems use local or global rule-based control. These heuristic approaches can achieve good control performances in small sewer systems during homogeneous rainfall events, but performances are limited when applied to large and complex systems, such as Louisville MSD’s sewer, where optimization-based algorithms would yield better performance results (Lund et al. 2018).

The Csoft algorithm uses model predictive control (MPC) with a feedback structure that repeats the optimization recursively in a finite time horizon. The Gurobi mathematical programming solver computes the flow set points (target values for flows) for future control actions such as opening, closing, or modulating gates and activating or deactivating pumps.

The repeated optimization updates the initial conditions every five minutes based on field data. Updated data include rainfall measurements, rainfall forecasts, water levels, flows, gate positions, pump rates, treatment capacities, and alarms. This method ensures that past process behavior (model) inaccuracies and load trajectories (predictions) do not affect the control decisions, and future prediction inaccuracies are reduced because they are updated almost continuously. For example, if the actuator of a controlled gate fails, Csoft receives the malfunction alarm and automatically modifies the gate control rule, changing it from “optimal” to “static,” and regenerates the gate’s rating curve based on the measured gate opening.

One of the main challenges related to using MPC is finding the right trade-off between model accuracy and computing speed. On one hand, a simple linear model reduces computing time, but it introduces a considerable amount of model inaccuracies that decrease the performance of the control system. On the other hand, a complex nonlinear model is more accurate but greatly increases computing times, which can be prohibitive in RTC applications.

The Louisville MSD RTC system is updated based on five-minute data sampling intervals (i.e., the intervals between the reoptimization of control decisions) to reduce or eliminate past inaccuracies in terms of model and load trajectories. This computing time constraint eliminates the necessity of a stochastic formulation of the optimization problem accounting for model and prediction uncertainty. It also eliminates the use of a nonlinear model based on Saint-Venant equations.

To reduce model inaccuracies without introducing nonlinearities, the problem is formulated as a mixed-integer program (MIP). This approach piecewise linearizes the nonlinear hydraulic behavior associated with gates, pumps, and storage facilities. As a result, the piecewise linearized equations describing the hydraulic behavior of the sewer facilities can be made as close as desired to the nonlinear ones by increasing the number of linearized sections. The MIP formulation also accurately reproduces discrete behaviors, such as that of on/off pumps.

Figure 2 shows the nonlinear and the piecewise linear behavior (flow-level relationship) of a fully open undrowned (i.e., no impact from downstream hydraulic conditions) rectangular sluice gate three feet wide and three feet high with three linearized sections. The higher the upstream water level is above the invert (floor) of the gate, the greater the flow. The piecewise linear discretization of a function of one variable can be found in D’Ambrosio (2009).

To further reduce computing times and model uncertainties, only the controlled section of the sewer system (i.e., the section that is affected hydraulically by the actions of a dynamically controlled facility) is modeled in the optimization problem. Integrated into the optimization feedback structure, a simulation-oriented model reproduces the uncontrolled section of the network. This model is simulated with updated rainfall measurements and forecasts at each data sampling interval within the prediction horizon. The rainfall forecasts are provided by a nowcasting system based on a linear extrapolation of radar reflectivity. The outputs of the simulation-oriented model provide the load predictions to the optimization model. Louisville MSD uses the InfoWorks ICM hydrologic and hydraulic model as the simulation-oriented model. Figure 3 shows Csoft’s architecture and its calculation core loop.

Another innovative feature to minimize model inaccuracy is online calibration. This calibration is achieved at each data sampling interval by solving a linear programming problem, where the cost function is defined as a trade-off between minimizing model flow errors and keeping the model parameters inside their physical boundaries (e.g., positive flow rates, mass balance closure). Calibration is achieved in a forward-moving time window using a past finite time horizon. The calibration horizon is large enough to avoid bias in the parameters’ identification but small enough to capture the time-varying nature of the physical process.

Balancing Priorities

To efficiently manage the sewer system under different rainfall and hydraulic conditions for various operational objectives, Csoft has a novel concept of “management strategy” using a cost function. Operators can either manually select the active management strategy or leave it on automatic where a strategy is triggered based on predefined conditions (e.g., rainfall, water level).

In a combined sewer system, during dry weather and small rainfall events that do not cause overflows, the management strategy will maintain a constant flow to the WQTC to maximize treatment efficiency, through optimal flow routing and use of storage when necessary, to balance flow in the collection system. During larger rainfall events, the management strategy will minimize overflows by storing excess flow and only releasing the accumulated volume when rain subsides, and treatment capacity becomes available. Finally, during extreme events that may cause flooding, the management strategy strives to respect maximum water levels; in such cases, overflows could be allowed to reduce basement or street flooding risks,

The cost function defines several penalties that apply to flood volumes (i.e., those volumes of wastewater flow or storage level that cause flooding), water levels, flows above or under a given threshold, overflow volumes, flows receiving only a primary treatment, unused secondary treatment capacities, stored volumes, and flow set point variations (i.e., variations inside the prediction horizon and between two data sampling intervals). These penalties ensure sewer operation efficiency under the active management strategy.

To facilitate the configuration of penalties for users, the cost function defines the variables in volume and the penalty weights in dollars per volume unit. The heaviest penalty weights are assigned to the most important operational objective.

The operational objectives of Louisville MSD’s RTC management strategy are defined as follows, in order of decreasing priority:

• 1. Minimize water levels or flows exceeding a flooding threshold.

• 2. Minimize overflow volumes and prioritize overflow locations to protect the most sensitive waterways.

• 3. Maximize flows receiving full treatment and prioritize treatment at plants providing the highest treatment efficiency.

• 4. Minimize sewer dewatering time.

• 5. Minimize gate movements and pump start/stop cycles.

Louisville MSD’s RTC management strategy also prioritizes flows between the two water quality treatment centers—the Morris Forman WQTC and the Derek R. Guthrie WQTC. In addition, when overflows are unavoidable, Csoft prioritizes overflows at outfalls to the Ohio River, which results in a lesser environmental and health impact than if it prioritized overflows to the (much smaller) Beargrass Creek.

Reducing Computing Time Through Warm Start and Iterative Solving

Csoft’s rapid computing speed differentiates it from other solutions in the literature. Over the years, several algorithms have been programmed to reduce the computing time needed to solve the MIP. This became necessary because of the increasing number of control facilities in Louisville MSD’s RTC system since 2006, and the additional sites that will be incorporated based on the overflow abatement plan. Currently, the time allowed for the entire Csoft step is 150 seconds, of which 90 seconds are used to solve the MIP.

To satisfy this time constraint over a prediction horizon of six hours (i.e., the minimum prediction horizon necessary to optimally manage the most upstream RTC sites), Tetra Tech programmed an algorithm that builds an MIP start file for warm starting. This file contains an initial estimate of every binary variable, computed from the previous set points. This approach considerably reduces the computing time required by Gurobi to solve the optimization problem because it guarantees a feasible solution that is generally close to the desired optimal solution.

We use a second algorithm to decrease the time needed to solve the MIP. This algorithm is based on the general idea that solving an optimization problem with few binary variables several times takes less time than solving a similar optimization problem with many binary variables once. Based on a comprehensive understanding of the hydraulic behaviors of the Louisville MSD sewer system, the algorithm fixes the binary variables related to piecewise linear discretizations to the values from the MIP start file. It then optimizes the MIP including the remaining binary variables and analyzes the resulting solution. If the piecewise linear discretization criteria are met, and the cost function does not change significantly compared with the previous solution, then the optimization process stops. Otherwise, the algorithm selects a new set of binary variables and optimizes again. This process continues iteratively until convergence on cost is achieved or until the total optimization time exceeds the prescribed 90-second limit.

In Table 1, we compare the computing times and sizes of the Louisville MSD MIP at eight different moments during a rainfall event. At each time step before the Gurobi presolve, the MIP contains 78,518 continuous variables, 33,286 binary variables, and 102,938 constraints. The presolve step reduces the number of binary variables to a range between 3,651 and 6,223 when solving in a single call and to a range between 284 and 2,501 when solving with the iterative algorithm. Fixing some binary variables also allows the removal of some constraints and their associated continuous variables if they are not used elsewhere, further reducing the resulting MIP.

Table 1. Solving Subsets of the Louisville MSD MIP Iteratively Considerably Improves the Solver Run Time with Only a Modest Increase in the Optimal Cost When Compared with “One-Call” Solving of the Entire MIP

Gurobi Version 7.5.2 is used with a time limit set at 20 minutes for these test runs. As we illustrate in Table 1, the iterative optimization algorithm dramatically reduces the computation time required to solve the problems with only a minor impact on the resulting solution’s cost. These results were obtained on an Intel Xeon 3.20 GHz 32 GB RAM computer with four processors and running Windows 10.

Protecting Against Data Errors

Csoft includes robustness features that take into consideration the uncertainties associated with models, measurements, and forecasts for the RTC of sewers.

Robustness is integrated into the cost function through the introduction of temporal attenuation factors on the penalty weights to decrease the impact of the process variables closer to the end of the prediction horizon. In the decision-making process, this parameter gives more importance to the flows and levels predicted in the near future than those predicted in a distant future and based on rainfall forecasts.

The application of level penalty weights also adds robustness. For example, the Louisville MSD RTC system uses a level penalty when the water level within a storage facility approaches two feet of the maximum allowable level. This introduces a safety margin to factor in possible errors in monitoring measurements and in model simulations.

Another robustness feature is the flow set point relaxation algorithm. This algorithm postprocesses the flow set points computed by the MPC algorithm by increasing their values when the objective is to not restrict flow. This avoids undesired storage and possible overflows that could arise from an underestimation of the prediction loads.

Most important, Csoft includes a data validation algorithm that detects and addresses measurement quality issues. It is based on the following two assumptions: (1) the difference observed between the values of two sensors measuring the same process variable belongs to a distribution of zero mean and known standard deviation when both sensors are operating normally, and (2) the difference observed between the values of the two aforementioned sensors belongs to another distribution when at least one sensor is faulty. From these assumptions, the differences between the measurements of two redundant sensors are characterized and compared with a threshold value, and a decision is made with respect to the validity of the measured values. For the simple case where only two redundant sensors exist, both measurements are validated at a given instant if their absolute difference is lower than the threshold value; otherwise, they are both assumed to be invalid.

Implementation

Since 2006, the Louisville MSD RTC system has been implemented in phases, and this phased implementation continues. Louisville MSD and Tetra Tech staff members collaborate closely from planning to design and through all stages of implementation to commissioning.

On the basis of the results of feasibility studies, Louisville MSD decided to implement phase 1 of the RTC system, which consisted of retrofitting five existing control sites. These included two flow diversions for optimal flow routing, one in-line storage to maximize sewer utilization, one outlet gate control for two off-line stormwater storage basins where 50% of the volume is used for CSO control and the remaining capacity is reserved for flood mitigation, and one pump station for efficient dewatering of upstream storage.

The most impressive RTC site is the control of the large gates at the end of the 24-foot-wide by 27-foot-high Southwestern Outfall Sewer (SWOR1) (Figure 4), which was originally built for flood relief purposes and had available capacities during normal wet-weather operations. RTC is applied to maximize storage within the pipe for a wide range of rainfalls to eliminate unnecessary overflows downstream. With a wet-weather storage capacity of 14.6 million gallons, the SWOR1 RTC site is a featured example of how Louisville MSD successfully manages in-line storage with RTC.

Louisville MSD has a supervisory control and data acquisition (SCADA) system and a highly skilled in-house instrument and control (I&C) staff. The SCADA system serves as the necessary backbone to the RTC system, and the I&C team is instrumental in the successful operation of the RTC system during wet weather. Implementing RTC at existing facilities allowed for rapid deployment and rapid overflow reduction results.

The first nine months of operation results from the phase 1 implementation showed sewer overflow reduction of 620 million gallons (MG), which matched the study prediction range of overflow volume reduction across those five sites. The following year, the system again prevented more than 600 MG of sewer overflows.

Although the initial implementation was relatively short, the overall sewer overflow program takes a long time over a phased implementation to design and construct the new infrastructure improvement projects. Once each of these facilities is built, it is integrated into the RTC system.

This phased implementation model prioritizes the control sites that provide the most overflow reduction advantages, aligns with Louisville MSD’s adaptive management approach to assess projects on performance results and a cost-benefit basis, and allows the involved stakeholders to gain knowledge and confidence in the system and the process to ensure full staff buy-in of the technology.

The implementation added robustness with the addition of redundant key equipment to minimize downtime, fail-safe modifications (e.g., emergency weir, backup power), data validation, and a set of local control rules to allow each RTC site to operate automatically and safely during failure conditions (e.g., sensor failure, actuator breakdown, communication loss). The Louisville MSD I&C team and Tetra Tech developed process control narratives that performed the physical local station programming.

Transforming Louisville MSD’s Organization

The main impact of the project is that Louisville MSD went from the status quo, which was a reactive operation, to proactive and predictive control. Throughout the implementation, it fostered organizational culture change that includes the involvement of its operations staff in the planning, design, construction, programming, and commissioning of the RTC system. The support from Louisville MSD executive management to communicate the vision and importance of total system optimization was critical to the program’s success. Through monitoring and data analytics, Louisville MSD gained a better understanding of the system and was able to provide better customer service to the rate payers. By continually monitoring the underground sewer network, the software provides the operations staff with critical information, enabling better system operational decision making in both dry- and wet-weather conditions.

In the initial days of the project, operational staff members feared that adding artificial intelligence to wastewater management would eliminate the need for some staff. Gaining the confidence of the operators in the RTC system was a major challenge. Automation and optimization brought drastic change in the operation of some facilities, compared with manual operation limitations. The main concern was that equipment malfunction would lead to major system failures and flooding of sewer water onto streets or into residences. This concern was lessened when operations during many heavy rainfall events demonstrated the fail-safe design of the system and its robustness at handling malfunctions. Louisville MSD also adopted a gradual approach when commissioning a new facility. The facility was first operated with a large safety margin from the optimal operating conditions. As operators became comfortable with the RTC behavior and performance, the margin was reduced to achieve better environmental performance. An example of this is the maximum water level allowable for in-line storage. At first, the target water level is set lower than the design value, and as performances are proven, it is raised to the optimal value, providing the maximum storage volume for CSO abatement. Another key feature for acceptance is that the RTC system is designed so that operators not only supervise the facilities remotely but can also take over full control of the facility at any time through the SCADA system.

The implementation of phase 1 involved existing facilities at which the original infrastructure design occurred before the availability of the RTC approach. It required retrofitting the facilities—for example, adding redundant instrumentation and replacing local PLCs with enough computing power to operate facilities under a variety of control modes rather than only remote/local/manual operation. These considerations have since been incorporated into current design standards.

Data analytics are instrumental to a successful implementation and sustainable long-term operation. The RTC system is fully automated in that it responds every five minutes with a new set of instructions for the next six hours. Csoft computes set points, and the SCADA system sends them to the local facilities where PLCs control the equipment to comply with the set points. Flow management is therefore improved and makes better use of available conveyance, storage, and treatment capacities throughout the system. The sewer system is actively monitored, and all field data (e.g., monitored levels and flows, equipment status, set points, operational decisions) are collected in the central station database. Alarms are raised when malfunctions are detected or when water levels are outside normal operating ranges, thus improving emergency responses. The Louisville MSD operators supervise the RTC system and actively respond when they receive an alarm or detect an anomaly. Analyses following the occurrence of an event detect system defects, assist with the stabilization of the optimal control set points, resolve specific problems, and provide continuous improvements to the overall system performance.

To improve on model and monitoring accuracy, Louisville MSD has invested in extending the rain gauge and coverage of level and flow monitoring, as well as in the development of a detailed hydrologic and hydraulic model to better represent the system conditions. It calibrates and validates the model on an ongoing basis. However, Csoft’s use of online model connectivity reduces the number and extent of network monitoring required for system-wide optimization.

In addition, Tetra Tech developed a set of web-based training tools for the Louisville MSD staff to use for knowledge transfer. Workshops and meetings with Louisville MSD’s engineering and operations division management and staff facilitate ongoing coordination and program support. Control site commissioning and start-up provide on-site training opportunities for staff as well. Louisville MSD and Tetra Tech update the operating manuals and standard operating procedures manuals regularly to describe system operations under all conditions when each facility comes into service. Dashboards presenting key performance indicators for different levels of stakeholders are being developed to improve the understanding of the RTC system and collaboration between stakeholders, while assessing system performance every few minutes.

Throughout the initial implementation and subsequent new-facilities integration period, Louisville MSD and its operators saw the benefits of OR and RTC. In the technology-dependent world in which we live today, operational staff are still needed. Now they can better focus on the operational and maintenance needs of the system guided in part by a smart system. The skill sets have changed, thus making the wastewater operator jobs more technically oriented. This creates opportunities for workforce development and new employment opportunities that are attractive to the current generations in the workforce and to those just graduating from school.

Key Deployment Milestones and Program Cost

The initial feasibility studies took place from 2000 to 2001. The design of the phase 1 pilot implementation was initiated in 2002. Testing and verification of the RTC system indicated that in a typical year, the annual overflow volume would be reduced by more than 10% when compared with volumes prior to the retrofit. The “smart system” became operational in 2006.

After an initial period of supervised control and tuning, Tetra Tech transferred control of the system to the Louisville MSD collection system operators at the end of June 2006.

The implementation of phase 1 resulted in significant overflow reductions at seven major CSO locations. Analyses of the pilot project operational data indicated that 600 MG of overflow reduction occurred annually. This is almost equivalent to 1,000 Olympic-size swimming pools of sewage water that did not go into the surrounding rivers. The real-world results were consistent with the estimations from the feasibility studies.

Because the RTC system is actively monitoring and controlling the sewer system, Louisville MSD began to gradually change its organizational culture through the testing and eventual acceptance of the new technology. It began to foster agency-wide change to set performance expectations for managing wet weather; to promote cross-silo collaboration among engineers, designers, operators, and maintenance staff; and to embrace a progressive vision of total collection and treatment-system optimization.

In 2008, Louisville MSD executives decided to embrace this smart solution as an integral component of their long-term plan to mitigate sewer overflows and manage the system on behalf of its rate payers. The long-term plan was updated to rightsize the overall need for new storage basins projects with the implementation of Csoft.

Phase 2 of the RTC system went online in December 2008. This phase included the integration of a main diversion structure and Southwestern pump stations, the retrofitting of flow regulators, and the construction of SWOR2, a new in-line storage site where an inflatable rubber dam and sluice gate provide 18 MG of additional in-line storage in the 24-foot Upper Dry Run Sewer Trunk and Mill Creek Sewer Trunk. After phase 2, RTC became an integral part of the overflow abatement plan, and new facilities constructed under the plan were gradually integrated into the RTC system as they came into service.

Projects under the overflow abatement plan will continue and are expected to improve water quality in both the Beargrass Creek and the Ohio River through and below Jefferson County. The current RTC system includes nearly 30 control facilities, such as two stormwater retention basins, multiple in-line and off-line storage facilities, flow diversion controls, and pump stations.

Benefits of RTC Implementation

The innovative RTC solution pioneered by Louisville MSD and Tetra Tech has had extraordinarily favorable impacts in terms of operation flexibility; operation efficiency (cost); regulatory compliance; and safe, clean waterways for the community.

Improving the Work Lives of Louisville MSD Operators

The implementation of RTC brings many qualitative benefits. It ensures the consistent and proactive operation of Louisville MSD’s sewer system with the flexibility to optimize and adapt to different types of actual and changing rainfall events compared with the traditional manual operation. Operators have more confidence in the system and can focus on the operation and protection of critical infrastructures during extreme rainfall events.

RTC and the implementation of the overflow abatement plan is an integral part of Louisville MSD’s commitment to continuous improvement through enhanced sewer system monitoring, analysis, and understanding. This supports the overall sustainability of the local community and the region by proactive operation and maintenance to enable reliably clean water and flood protection.

Adding more sophisticated instrumentation, control, automation, and advanced analytics to operations and maintenance, engineering, and planning helps Louisville MSD with continuous workforce development and the ability to operate into the future. It also attracts a new generation of local engineers and operators who are interested in intelligent control and smart ways of doing business. An added benefit of this technology is that it gives Louisville MSD the ability to redirect flow to other parts of the system to allow access for maintenance and troubleshooting. The additional flow monitors and level sensors inside the sewer pipes provide continuous information displayed on control panels with built-in alarms that indicate potential problems. This information allows troubleshooting to occur in a timelier and safer manner for the staff.

Cleaner Water

The impact of innovation goes beyond financial and environmental benefits. Louisville is a beautiful river community. From boating and fishing to wading and swimming—on the mighty Ohio River and the multitude of inland streams—the use of local waterways is a normal part of life here. This project directly impacts the quality of life of Louisville’s 755,000 residents in this community. Louisville’s goal is to be a world-class destination city that is full of economic growth and outdoor recreation opportunities for those who that call this home today and will reside here in the coming decades.

The Louisville MSD RTC system improves the environment by managing, monitoring, and reporting sewer overflows and any water quality violations under regulatory compliance. This system is key to maximizing the conveyance, storage (in-line and off-line), and treatment capacities of both previously existing and newly constructed projects under the overflow abatement plan through automated decisions for storing and routing flows in the sewer system.

Because the Louisville MSD system is changing with the implementation of new projects, and the amount of rainfall differs every year, Louisville MSD uses the continuous model simulation of a typical year for the purpose of demonstrating CSO and SSO reduction performance. Results show that prior to RTC and the implementation of the overflow abatement plan, the Louisville MSD system had an estimated average annual overflow volume of nearly seven billion gallons during a typical year. As of June 2018, the system, with completed overflow abatement plan projects and RTC, is achieving an average annual overflow volume reduction of four billion gallons during a typical year. After the construction of all the planned projects, postoverflow abatement plan model simulations showed that RTC will allow Louisville MSD to capture and treat 17 billion gallons of wet-weather combined sewage based on a typical year.

The RTC system has been performing consistently to reduce combined sewer overflows through a better use of its storage facilities. Monitoring data at RTC-controlled in-line and off-line storage facilities showed consistent results of billions of gallons of CSO volume stored and later treated at a WQTC every year. The actual system performance results vary annually because of variations in actual annual rainfall event types, volume, and intensity; the addition of control sites; and the unavailability of conveyance, storage, and treatment capacities because of maintenance, construction, breakdowns, and repairs.

Overall, the RTC system has contributed directly to the mitigation of more than two billion gallons per typical year of combined sewer overflows to date. Another two billion gallons of average annual overflow volume reduction per typical year can be attributed to other overflow abatement plan projects, such as sewer separation, green infrastructures, inflow and infiltration reductions, and operation and maintenance improvement efforts.

Lower Cost

Louisville MSD achieves regulatory compliance in an innovative and cost-effective way with RTC and its efficient use of resources. Maximizing the existing and future collection and treatment system through RTC not only reduces the overflows significantly but also reduces the need for large storage facilities across the community to reach the same environmental objectives.

Financially, Louisville MSD has conservatively estimated its capital cost avoidance from applying OR to be $200 million to $300 million. This was achieved by downsizing the overall need for new storage basins projects as initially identified. The use of the Csoft optimization software allowed better use of the existing system capacity while achieving the same overflow reduction. The cost avoidance is calculated as the capital investment reduction from the update of the initial long-term overflow abatement plan where some storage projects were either eliminated or reduced in volume.

Once Louisville MSD embarked on a full system integration with Csoft, it did not perform any cost-reduction evaluations to compare the savings prior to and following the integration. Because it is actively monitoring the system, updating the model, and analyzing data from actual operations, it is learning more about its sewer system and applying adaptive management to invest in improving the system through the implementation of various projects. Louisville MSD is a public utility funded by the rate payers to protect public health and safety; therefore, avoiding $200 million to $300 million from implementing RTC represents 20%–30% of the total estimated cost, which enables the rate payers of the Louisville metro area to have more affordable service for safe, clean waterways. Louisville MSD also provides two other core business functions—providing stormwater management and Ohio River flood protection services for the community—thus keeping the costs to the rate payers at an affordable level, contributing to the overall economic health of the local community, and helping local suppliers stay in business and innovate.

Executive View

The adoption of RTC technology requires organizational commitment and staff buy-in. Utilities need to consider operational issues and constraints when selecting the appropriate level of RTC implementation. Involving system operators early in the planning process is important, as is identifying and communicating roles and responsibilities at every stage: design, construction, commissioning, and postconstruction performance monitoring. Developing and implementing standard operating procedures, monitoring system performance, and doing analyses after an event are critical to the proper operation, maintenance, and improvement of the RTC system.

The smart use of RTC technology has allowed Louisville MSD to maximize programmatic benefit and cost reductions while also improving the water quality of receiving waterways. It continues to improve and expand its RTC system as new storage and treatment facility improvements are constructed under its overflow abatement plan. According to Angela Akridge, chief engineer of the Louisville MSD and one of this paper’s authors, real-time control is an important component of Louisville MSD's long-term plan to mitigate untreated combined sewer overflows into Beargrass Creek and the Ohio River. It is an innovative, cost-effective, and sustainable management strategy that helps to satisfy regulatory requirements while also improving overall sewer system operability.

Transportability to Other Communities

Louisville MSD has often shared its experiences and demonstrated this new technology to other communities so that others can benefit and adapt Csoft into their efforts to reduce overflows. In the United States, more than 800 river communities like Louisville can benefit from this technology.

The OR developed in Csoft makes it an adaptable and robust solution, complementary to the traditional engineering methods, to address the various challenges in urban water management, which are universal; examples include sewer overflows, stormwater runoff, water quality, flooding, and droughts. This solution has been successfully implemented in other communities in the United States, Canada, and France, saving each community between 25% and 75% in capital expenditures by optimizing the use of system capacities while reducing additional infrastructures required to achieve multiple operational and environmental objectives. Communities such as Wilmington, Delaware, in the United States; Montreal and Quebec City in Quebec in Canada; and the Paris Agglomeration Sanitation Bureau and the Community of Bordeaux in France have adopted Csoft. In total, these implementations have saved nearly a billion dollars in avoided costs for these communities while also improving their water quality.

OR will continue to innovate and evolve the Csoft solution to other water management applications, further capitalizing on the availability of monitoring, rapid data transfer, and OR innovations. This system has the flexibility to adapt and further the objective of optimizing the systems we use to manage our water and wastewater systems.