Case Article—Analysis of Call Center Data at Patelco Credit Union

Introduction

All too often, students as well as practitioners, despite prior course work in statistics, find themselves at bay when faced with real data in practical situations and struggle to effectively communicate data and insights or draw statistically significant conclusions. This case study provides a context, adapted from a real-world situation, that instructors can use to introduce students to a systematic and statistical approach to the analysis of data that might be needed to address operational challenges in any environment. In particular, the focus is on operational challenges at a customer call center of Patelco, which is a California-based credit union. Customers face long delays when they call the call center, and the duration of the calls is long for many of these customers. This has an adverse effect on customer experience. Also, internally, there is pressure on the bank (as on most other financial institutions) to control costs, so staffing levels at the call centers is an important area of concern.

Customer service and operational efficiency are particularly important for credit unions because they are member owned and, by their charter, operate to serve their members rather than maximize profits. Because call centers are the primary interface between customers and the credit unions on an ongoing basis, customers’ experience when they call service agents is critical. Therefore, operational efficiency of the call centers can be a key factor in defining success for these institutions, making analysis of call center operations critical for credit unions. The focus of this case is primarily on understanding the raw data about incoming calls handled by the agents at one of Patelco’s call centers. This is important because analysis of the variability in the call duration times as well as patterns in arrival rates of these calls during a day or across days of the week are important inputs to any analysis of customer delays or staffing levels. Optimization of staffing levels is not considered in this case study but is the subject of a follow-on case study. Although the focus here is on Patelco, the discussions are relevant to the design of call centers in other contexts as well as analysis of data in general.

Founded in 1936 as a not-for-profit financial institution with initial assets totaling $500, Patelco originally served only the employees of the Pacific Telephone & Telegraph Company. By 2019, Patelco was the 22nd largest credit union in the country and 6th largest in California, with over $7 billion in assets and serving over 360,000 members nationwide. They serve communities across Northern California, including the Bay Area, Sacramento, and San Jose as well as the employees of over 1,100 large and small businesses throughout the United States (Patelco). Customers can access their accounts from 38 local branches, over 30,000 ATMs, and over 6,000 shared credit union service centers nationwide. In addition to the usual banking services, they also offer home loans, home equity loans, auto loans, personal loans, credit cards, and home equity lines of credit.

The banking industry is a particularly interesting context to study in a course on statistics or operations management (Frei and Harker 2000, Campbell and Frei 2004). Given the financial and banking crises that the United States has gone through, retail banking provides a very relevant industry for examination. Although the majority of the popular press and management literature has focused on the financial causes of these crises, in many cases, operational factors are equally important in explaining inefficiencies in this industry. Indeed, bankers often realize that the economics of retail banking are such that the cost of customer acquisition far exceeds that of customer retention. The latter, of course, is driven by customer satisfaction resulting from operational performance.

The call center described in this case study is an example of an inbound call center that handles calls initiated by a customer calling the credit union. These calls may relate to customer queries about their accounts or about the various products that are offered by banks. In contrast, outbound call centers handle calls initiated from the center, typically for telemarketing and survey type of activities. (Gans et al. (2003) provide an excellent review of the academic literature on call centers.) Capacity planning tends to be more challenging for inbound call centers.

Overview of the Case and Teaching Objectives

The protagonist in the case has been charged with investigating the operational performance of Patelco’s call centers. As a precursor to this analysis, he is interested in analyzing the raw data related to calls that have been handled by call center agents. Therefore, students are placed in the role of an analyst tasked with such an analysis.

In the operations management literature, and in practice, a number of modeling assumptions are made to facilitate tractability in quantitative analysis. However, these assumptions are often not statistically validated. This case study gives students an opportunity to test such assumptions. They are required to graphically display the variability in the call duration times. Because variability in call duration times is a key driver of waiting times for callers, such an investigation must precede any analysis to optimize capacity in call centers. Useful managerial insights can result from such graphical visualization. For example, the cumulative distribution function can be used to quantify what fraction of calls are being completed within a certain target length of time. This context also provides an opportunity to preempt a conversation about factors that might drive delays in any service system, such as variability in arrival or service times, insufficient service capacity, or long service times.

Students are asked to analyze if the exponential distribution can be used to model the call duration time. Because this assumption is commonly used in queuing analysis models, it must be verified. Again, graphical comparison of the empirical cumulative distribution function with an actual distribution can be illustrative but not conclusive; hence, a statistical test, such as the Chi-square goodness of fit test, should be used. Even if the test results prevent us from concluding that the exponential distribution is a good fit, we might still use this assumption in some instances if the resulting analytical tractability is compelling and if the loss of accuracy is minimal. This is a useful insight for students.

Students are also required to compare the performance of call center agents and variations in the number of calls received across the days of the week and during any day. This sort of investigation must precede the step of optimization of staffing decisions over time—within a day or across the days of a week or assessment of staff performance. Students will learn how to perform such analysis by formulating appropriate research questions and designing testable hypotheses. Indeed, data collection should commence only following the specification of such research questions and hypotheses, a sequence often inverted in practice. In particular, students will get to apply useful concepts, such as confidence intervals and the unequal variance t-test (also known as the Welch’s t-test). Students learn how the sample data can be used to compute a confidence interval for the average call duration time. They are surprised to learn that even though the average of the call duration time of one agent is less than that of another, this difference may not be statistically significant. Importantly, the case also provides an opportunity to discuss practical considerations in interpreting the results of such tests.

Students will realize that the data set accompanying this case study does not help them answer all the relevant questions about analysis of the data center operations but can address some important ones. At the same time, they will learn how to deal with data formatting issues and how to exercise judgement in order to gain the most insight from real-world data sets.

After working on this case, students should be able to

• Graphically visualize data using x-y scatter charts, histograms, and tables and derive managerial insights from such summaries;

• Compute confidence intervals and compare sample means using unequal variance t-tests, and assess practical implications of the results;

• Apply the Chi-square goodness of fit test to evaluate statistical distributions and assess practical implications of the results;

• Formulate tractable research questions (e.g., evaluation of differences in staff performance metrics, such as average call duration time); and

• Recognize challenges associated with the use of real-world data (e.g., values of zero, insufficient data for analysis, Excel date and time formatting).

Teaching Suggestions

This case has been used in graduate and undergraduate classes in operations management, supply chain management, and service operations to review statistical concepts related to the topic of queuing analysis. It also works well in executive MBA classes. In general, undergraduate and some MBA students tend to focus more on the statistical tests. The more experienced MBA and executive MBA students engage in a very interesting discussion related to practical considerations concerning data analysis and how best to draw insights and communicate data and results.

This case can be taught as a standalone case if the objective is to teach students about data analysis and statistical tests using a real-world context. It can also be taught prior to using the follow-up Patelco (B) case, available from the author, where the goal is to teach the students how to make capacity planning decisions in a call center.

Depending upon the background of the students, they may or may not have been previously exposed to the statistical concepts discussed in the case (descriptive statistics, probability density function, cumulative distribution function, exponential distribution, central limit theorem, confidence intervals, and tests of statistical hypotheses, such as Chi-square goodness of fit test and the unequal variance t-test). If they have studied these before, then a quick review before or simultaneous with the application is helpful. If not, then this can be used to motivate the need for and value of such tests, with references provided to appropriate materials from statistics text books used at your institution. In this case, because the t-test has been used both in Problems 2 and 3, the instructor may choose to focus only on one of these problems for illustrating the methodology and de-emphasize the methodology in the other problem.

This case can also be taught in a statistics class. Because the case uses a range of statistical concepts, it may be appropriate to use this toward the end of the class to illustrate how all of the methods can be applied in practice. As noted above, if some of the concepts have not been covered in class, then this can be used to motivate the need for and value of such tests, with references provided to appropriate materials from statistics text books used at your institution. Finally, it is also possible to introduce students to the context and data earlier in the class and use the data to illustrate specific methodologies as and when needed. For example, the concept of probability density function and cumulative distribution function can be illustrated using the data earlier in the course. Later, when students are taught confidence intervals, or hypothesis testing, the same data set can be used again.

The case has been designed to be used as an in-class activity, although it can be treated as a homework assignment for credit. Students can work on the case independently, in pairs, along with the instructor, or a mixture of these options.

If used as an in-class exercise, the case can be scaled down depending on the available time and depending on the specific topics that the instructor wishes to cover. To cover all of the analysis in the case will require about two hours in class.

Classroom Experience

There are four types of challenges that students run into as they analyze the case study, each of which becomes a learning opportunity. The first involves learning about how date and time data are stored in Excel. In the data set, information about the start time of calls is time stamped using the mm/dd/yyyy hh:mm:ss AM format and that for call duration is in hh:mm:ss. Many students are unable to perform any calculations using these data because they are unfamiliar with how to convert this into a usable format. This is new learning for many. Second, we find that even though students have had a course in statistics, when faced with data in practice, they are often unable to remember the importance and knowhow of the required statistical methodologies. This leaves them either stumped or with imprecise conclusions about the data. When the appropriate techniques are demonstrated, it leads to important aha moments; students appreciate the power and insights resulting from statistics. Third, students grapple with several practical considerations that become relevant when using real data. For example, what should one do with data that may be erroneous? How does one decide the right level of granularity with which to analyze data? What would one do if a hypothesis is rejected? Thus, this case study offers the instructor the opportunity to reinforce the lesson that there is a substantial amount of art involved in the science of quantitative analysis and that the experience obtained from repeated use is invaluable. Finally, students sometimes struggle with how to visualize the results of their analysis in a way that conveys insights easily in practice. Thus, the instructor can teach the power of visual communication of complex analysis.

Preparing for the Case

If the case is used for class discussion, it is helpful to assign students a preclass assignment aimed at familiarizing them with the data before they come to the class. The assignment can require them to determine simple summary statistics of key data, so that they are familiar with the nature of the data and the environment. This can also expose them to some of the data formatting–related challenges, which can then be discussed in class.

After the discussion in class, students should be encouraged to review teaching materials related to descriptive statistics, probability density function, cumulative distribution function, exponential distribution, central limit theorem, confidence intervals, and tests of statistical hypotheses, such as the Chi-square goodness of fit test and the unequal variance t-test. If they have not seen these materials earlier, then this can be a segue into that discussion.