Case Article—Moneyball for Murderball: Using Analytics to Construct Lineups in Wheelchair Rugby

1. Introduction

This case provides students with an opportunity to explore descriptive, predictive, and prescriptive analytics techniques in the novel application domain of wheelchair rugby (WCR). A major theme in sports analytics focuses on determining player value and constructing optimal lineups to maximize a team’s chance of winning a game. Estimating player value typically involves a combination of exploratory data analysis and statistical modeling, whereas lineup construction is accomplished using mathematical optimization. The hypothetical client in this case is the coach of Canada’s WCR team, who is interested in using analytics to help inform lineup decisions.

Wheelchair rugby, colloquially referred to as “murderball,” given its aggressive and full-contact nature, is a mixed-gender sport that is played on a standard-sized basketball court. The objective of the game is to score more goals than your opponent by carrying the ball across the other team’s goal line. Four players from a team are allowed on the court at the same time; the exception is when a player is sent off for a penalty. A novel twist compared with able-bodied team sports is that in WCR there is a well-defined knapsack constraint that limits which players can play together at the same time. This constraint considers the physical ability of each player, which is summarized by a numerical physical rating assigned to every player. Athletes with less physical impairment have higher numerical physical ratings. The knapsack constraint ensures that lineups cannot have too many athletes with low physical impairment. More importantly, it promotes the participation of all athletes, independent of their physical impairment.

The activities in this case are divided into three modules. In the first module, students analyze a synthetic data set (derived from a real data set) of WCR game data. Students build familiarity with relevant statistics such as goals scored and player physical ratings. An important statistic that is introduced is a player’s plus-minus, defined as goals scored by their team minus goals allowed by their team when the player is on the court. However, raw plus-minus values can be misleading in terms of ranking player performance. Indeed, classic “Moneyball” analysis demonstrates that proper statistical modeling is needed to untangle the effect of other players on one’s plus-minus. So, in the second module, students develop linear regression models to estimate adjusted plus-minus (APM) values, which account for the other players on the court, as well as factors such as home team advantage. APM was first popularized in basketball (Rosenbaum 2004) and subsequently applied in other sports, including hockey and soccer (Hvattum 2019). Finally, in the third module, students use the estimated regression coefficients as parameters in linear and mixed-integer optimization models. By taking the output of the regression and using it as input in an optimization model, this case can be used to introduce students to the popular “estimate-then-optimize” or “predict-then-optimize” paradigms. Students will also explore specific optimization concepts such as understanding feasibility, formulating context-specific constraints, and identifying multiple versus unique optimal solutions.

2. Literature Review

Sports analytics provides fertile ground for pedagogy in operations research (OR). For example, baseball has been used to teach machine learning (Barnes and Bjarnadóttir 2019), Markov processes (Bickel 2004, Sokol 2004), and optimization (Cochran 2004). Other sport-specific cases include using simulation and stochastic processes to analyze hockey (Ingolfsson 2004), basketball (Ammar and Wright 2004), and soccer (Chu 2003); investigating scheduling problems in various soccer leagues (Birge 2004, Goossens and Beliën 2022); exploring business analytics in football (Huggins et al. 2020); and applying integer programming models in cycling (Beliën et al. 2011). There are numerous other examples in which sports has been used in OR/management science classrooms (Trick 2004, Kvam and Sokol 2004, Willoughby 2004, Roth and Matherne 2021).

Another major stream in the literature includes teaching a combination of analytics techniques (e.g, predictive and prescriptive), either throughout a course or within a case study. For example, students are introduced to forecasting, allocation, and overbooking problems in undergraduate and graduate courses in revenue management (Anderson et al. 2009). Machine learning has been combined with optimization in an undergraduate OR course focusing on real-world applications (Boutilier and Chan 2021). In terms of cases, different types of analytics have been taught in airline revenue management (Shumsky 2009), staffing (Pachamanova 2015), and railroad company management (Gorman 2021). Games such as “Pass the Pig” also provide an opportunity to apply multiple analytics techniques (Gorman 2012).

Our case bridges these two areas in the literature by exposing students to a combination of analytics techniques within the realm of sports, particularly Paralympic sports. Whereas able-bodied individual and team sports have been well-represented in scholarly work in sports OR (both research and teaching), Paralympic sports have not. We believe it is important to increase the representation of Paralympic sports and athletes in our scholarly endeavors, for the sake of advancing the sports themselves, as well as for generating interest among students and the general public. We hope this case contributes to that cause.

3. Learning Objectives

Upon completion of this case, students should be able to:

1. Clearly distinguish between descriptive, predictive, and prescriptive analytics techniques and combine predictive and prescriptive models to generate novel insights.

2. Use appropriate software to:

   • (a) Perform basic data analysis such as data parsing, data manipulation, and data visualization on a realistic sports data set.

   • (b) Fit ordinary and weighted least-squares linear regression models and interpret the model output.

   • (c) Formulate and solve linear and mixed-integer optimization models, including those with piecewise linear objective functions.

3. Identify opportunities in sports and other application domains where these techniques can be applied.

The first objective is broad and should be achieved by students reflecting on how the modules of this case fit together. The second objective covers skill development specific to each module, with a focus on developing hands-on programming skills. Although other languages could be used, our recommended language is Python, given its pervasiveness in industry and research. As such, the content of the case has been developed with Python in mind. The third objective can be achieved through discussion following the completion of the case, guided by the instructor, to synthesize and generalize what the students have learned.

4. Required Background

This case assumes that students have basic data analysis, regression modeling, and optimization modeling skills. In this section, we highlight specific knowledge and skills required. A complete Python notebook for this case is available to the instructor. The case can be adapted to teach these concepts in an introductory course, with a focus on the more basic concepts.

4.1. Basic Python and Data Analysis

Students should have a good foundation in Python and be familiar with common packages such as $\mathsf{Pandas}$, $\mathsf{NumPy}$, and $\mathsf{Matplotlib}$. In terms of data analysis, students are expected to join data from multiple tables, appropriately group and aggregate data, as well as perform simple visualizations such as scatter plots and box plots.

4.2. Regression and Optimization

Students should have an understanding of regression analysis, including identifying appropriate dependent and independent variables, assessing model fit, and interpreting coefficient estimates. Moreover, students should be able to understand the differences between ordinary and weighted least-squares regression. Within Python, regression models will be implemented using the $\mathsf{statsmodel}$ package. Regarding optimization, students should know how to formulate linear and mixed-integer programs, as well as understand how to transform maximization problems with piecewise linear concave functions into equivalent linear programs. The optimization models within Python will be solved with the open-source $\mathsf{cvxpy}$ package. The $\mathsf{PYTHON}\_\mathsf{MIP}$ module from the $\mathsf{mip}\_\mathsf{cvxpy}$ package is also required to solve the mixed-integer programs. Instructors and students can use other solvers, such as $\mathsf{GUROBI}$ or $\mathsf{CPLEX}$, if they wish. Some of the more advanced concepts such as weighted least-squares regression or transforming piecewise linear objective functions could be taught using the case, as long as students have a basic foundation in ordinary least-squares regression and linear/integer programming, respectively.

5. Teaching Plan

5.1. Scope

The technical content of this case is divided into three modules:

1. Descriptive analytics: exploratory data analysis and visualization

2. Predictive analytics: regression modeling

3. Prescriptive analytics: optimization modeling

Given the breadth of material covered, this case can be used in several types of courses (see Table 1).

Table 1. Mapping from Case Modules to Different Types of Courses

In an introductory statistics course that covers regression, modules 1 and 2 can be used to reinforce data analysis and regression modeling skills. Extensions to regularized regression and other advanced topics are possible. In an introductory operations research course covering deterministic optimization, modules 1 and 3 would be appropriate. In order to solve the optimization models numerically, the instructor would need to provide the regression coefficients from module 2 to the students to use as optimization model parameters. These parameters are given in the Teaching Note. Finally, in an analytics course that covers both statistics/machine learning and optimization, all three modules can be used. In this scenario, students would develop skills building a complete pipeline that links the output of a regression model with the input of an optimization model.

5.2. Delivery

This case is meant to provide students with hands-on experience in data analysis and modeling. As such, we expect students to spend serious effort modeling and coding before examining the results of the case with their classmates and instructor. Given this perspective, our preference is to present the case over two lectures (assuming 2–3 hours). However, it is also possible to cover it as a single-lecture case (assuming approximately 1–1.5 hours). For a statistics-only or optimization-only course, the included modules can be extended to consider advanced topics if the case will be covered over two lectures, or consolidated as is into a single-lecture presentation. The two-lecture format is particularly suitable for an analytics course covering both statistics and optimization, given the breadth of material covered. Detailed teaching plans are given in the Teaching Note.

5.3. Data, Python Notebook, and Case Questions

Students are given two data files to work with: $\mathsf{player}\_\mathsf{data}.\mathsf{csv}$ and $\mathsf{stint}\_\mathsf{data}.\mathsf{csv}$. The first file contains a list of all players and their physical ratings. The second file contains detailed information on each game.

We also provide a complete Python notebook ($\mathsf{WCR}\_\mathsf{case}.\mathsf{ipynb}$) as a companion to the Teaching Note. The notebook is divided into three sections, corresponding to the three modules shown in Table 1. It covers the technical questions presented in the Teaching Note and the code-based solutions. This notebook can be used to evaluate student submissions. It can also be modified and presented to students as a “skeleton file” that guides them toward a particular model. The notebook can be run in any regular Python coding environment (e.g., Jupyter notebook, PyCharm, Google Colab, etc.). The Teaching Note includes a comprehensive set of 25 questions that the instructor can assign or use to guide the development of the case. The questions cover a range of data analysis, regression modeling, and optimization modeling topics.

6. Classroom Experience

This case has been used in different forms in three undergraduate engineering courses—an analytics course (twice, Fall 2020 and Fall 2021) and two case-based courses (once each, Winter 2022)—at two institutions. All courses are electives and have a fairly even mix of third-year and fourth-year undergraduate (primarily OR/industrial engineering) students. The content for this case was initially developed as a laboratory for the analytics course, which is the course in which the authors have experience teaching the case. Subsequently, we expanded the content and developed an entire case around it for the two case-based courses, which were delivered by colleagues.

6.1. Courses Where This Case Was Used

6.1.1. Analytics Course.

This course covers machine learning and optimization, with an emphasis on applications. Enrollment in the two years the case was offered averaged 80 students. Almost all students taking this course have already taken an optimization course but not a machine learning course. Every week there is a lecture, tutorial, and three-hour computer laboratory. A comprehensive description of the course is provided here (Boutilier and Chan 2021). Students work through parts of all three modules during the laboratory, with the aid of teaching assistants roaming the computer laboratory. However, given the time constraints of the laboratory, only a small subset of the case content is covered. In particular, advanced topics in the regression and optimization modules are omitted. A skeleton of the Python notebook is provided to each student, who answers a set of questions by filling in code blocks and interpreting the results. As such, students are guided through the case much more directly than in the two following case courses.

6.1.2. Case Course 1.

In this course, students work through a different case every week, which is posted a few days ahead of time. Enrollment was 13 students in Winter 2022. The weekly structure is as follows: (1) a first lecture (one hour) at the start of the week to build a common understanding of the key details of the case and potential modeling approaches; (2) a few days to complete the case in teams of two to three students; and (3) a second lecture (two hours) where students present their results, compare approaches, and engage in broader discussion. The day before the first lecture, students must submit a one-pager describing their proposed approach to solving the case. The day before the second lecture, students submit a write-up of their solution. The lectures are facilitated discussions, and students receive points for participation. A comprehensive description of the course is provided in Frances and Terekhov (2018).

To cover the entire WCR case, it was split into two parts, covered over two weeks. The first week focused on the first two modules, mainly the regression, and the second week focused on the third module. This course is taught at the same institution as (and typically taken after) the analytics course described above. However, the case represents a significant expansion over the laboratory content from the analytics course, so the overlap is minimal. Also, as both courses are electives, to date, there has not been significant overlap in the students (i.e., fewer than half of the students in the case course had taken the analytics course). To address the potential advantage that students who had taken the analytics course might have, the case course instructor ensured that each team had exactly one student who had previously taken the analytics course.

6.1.3. Case Course 2.

The structure of this course is similar to the other case course, except that students work through a case every two weeks and there is a single lecture per week for three hours. Enrollment was 17 students in Winter 2022. After the first lecture, students have an entire week to complete the case and submit a report. Like the other case course, this course emphasizes identifying appropriate solution methods and real-world considerations (e.g., missing data, client needs, timelines, assumptions, limitations, trade-offs, solution sensitivity, recommendation to the client on how to proceed).

All three modules were covered with emphasis on the regression and optimization. Unlike the other case course, this course tackled all elements of the case at once. Given how the course is structured, the case took the same amount of time (two weeks), but students did not have the benefit of regrouping to discuss the regression results before embarking on the optimization modeling. The students did not have prior knowledge of some of the advanced elements of the case. So, much of the first class—typically reserved for discussion—was used to help them set up the regression properly and formulate the objective in the advanced optimization models.

6.2. Instructor Comments

Instructors of the case courses thought that the case was great for teaching sports analytics, as well as advanced optimization and regression ideas. They noted, however, that it was difficult to do so in a course meant to apply existing knowledge, unless there was additional time (e.g., a tutorial) outside of the discussion classes that could be used to teach the advanced topics. All instructors found that students had an easier time with the optimization module than the regression module, likely because the departments in which these courses were offered emphasize optimization modeling more in their curricula.

6.2.1. Analytics Course.

This course focuses heavily on Python, and students had exposure to both regression and optimization modeling earlier in the course, before the WCR laboratory. Hence, they were well prepared to tackle the technical aspects of the case. The WCR laboratory was the students’ first exposure to the predict-then-optimize paradigm, which is a major focus of this course. Being grounded in the real-world context of WCR helped students understand the differences between prediction and optimization and, more importantly, how they can fit together in an end-to-end pipeline.

Students were very curious about and invested in modeling piecewise linear functions as part of the advanced optimization models. Lots of questions during the laboratory revolved around this issue. The students were also interested in discussing how optimization models could be used in practice and what could be done to the models to make them more realistic to coaching staff.

6.2.2. Case Course 1.

The instructor thought that the regression module was an excellent vehicle for students to practice their regression modeling skills using Python. There was a good discussion around the interpretation of the regression coefficients, the meaning of positive versus negative values, and how to modify the regression so that the coefficients were all positive. Students did not know how to do this on their own, but figured it out through the class discussion.

Regarding the optimization models, almost all students managed to formulate and solve the basic models. One team that had trouble proposed a greedy algorithm instead, which led to a discussion between exact and heuristic methods. The advanced models required more hand-holding. When students compared solutions, they realized the presence of multiple optimal solutions and discussed whether some were better than others based on criteria beyond the objective function value.

Overall, the instructor thought that the complementarity between regression and optimization was a strength of the case. He also appreciated the fact that the case could be broken down into well-defined steps.

6.2.3. Case Course 2.

Students in this course had less exposure to Python than in the other courses. Hence, many students used Excel for the regression. They found that the data set was too big for Excel, so they used a subset of the data to fit the regression model. It was a challenge for them to deal with the negative regression coefficients, and many students simply added a constant value to make them all nonnegative. Students found the basic optimization models straightforward to implement, but they were challenged with the more advanced models. The instructor commented that it was nice to have a range of difficulty, especially for stronger students, so that they would have an opportunity to stretch their skills.

The postcase discussion was rich. For example, students realized that solutions that require players to be substituted on and off very often were not realistic, which helped them realize the importance of looking out for multiple optimal solutions when interpreting optimization output. The application context was easy enough to understand so they could provide relevant critiques to various assumptions. For example, students noted that plus-minus values were averaged over lots of opponents as opposed to being personalized to the opposing team. Also, they noted that more realistic fatigue curves should account for the effect of rest. Some students pointed out that a dynamic model would be important for reoptimizing lineups as the game progress. One student commented on the interplay between regression and optimization, aptly noting that errors in the regression could significantly influence the optimization results.

Given the scope of the case and some of the advanced skills needed to complete it fully, the instructor noted that breaking it into two cases might have been more effective. The instructor also mentioned that the case could be used as a class project in some courses. This way, it could be used to integrate all the techniques they learned, for example, in a course that covers regression and optimization in a Python environment.

6.3. Student Feedback

We conducted an optional postcase survey across the three courses to collect student feedback. The instructors also reported anecdotal feedback from students obtained during their discussions.

6.3.1. Analytics Course.

We received 35 responses across two years of using the WCR laboratory.

• 91.4% agreed/strongly agreed that it helped teach an end-to-end, predict-then-optimize framework, and explain predictive and prescriptive analytics.

• 94.3% agreed/strongly agreed that it helped convince them that analytics/OR can be applied to a wide variety of real-world problems

• 56.5% agreed/strongly agreed that this laboratory was more interesting than the others.

• 52.5% found it more difficult than the other labs.

• 82.6% agreed/strongly agreed that it helped them better identify potential problem applications compared with other labs.

Some student quotes:

• “Predict and optimize as a topic was interesting to learn about, so I enjoyed the lab as it was a new concept.”

• “[The wheelchair rugby lab] was the one I felt engaged in most.”

In talking to the students, they liked the fact that they could work on a data set derived from real data obtained from Canada’s WCR team. Overall, they thought WCR was an interesting sport and application area.

6.3.2. Case Course 1.

The following is a summary of the five responses we received.

• 100% agreed/strongly agreed that it taught descriptive, predictive, and prescriptive analytics techniques in a novel application domain.

• 80% agreed/strongly agreed that it helped them identify other application domains where analytics can be applied.

• 80% agreed/strongly agreed that it was more interesting than other cases.

• 80% agreed/strongly agreed that it was more difficult than other cases.

Some student quotes:

• “I liked the case; it taught me some skills I have never touched up upon before as well as helped build my OR solving skills when it came to solving it.”

• “It was an interesting case, and a great way to apply analytics.”

Students who had previous exposure to the case from the analytics course reported that they still learned a lot from doing the full case in this course. The laboratory in the analytics course is tight on time, and students in the case course felt they had much more time to appreciate the individual elements and how they fit together.

6.3.3. Case Course 2.

The following is a summary of the 13 responses we received.

• 61.6% agreed/strongly agreed that it taught descriptive, predictive, and prescriptive analytics techniques in a novel application domain.

• 76.9% agreed/strongly agreed that it helped them identify other application domains where analytics can be applied.

• 53.8% agreed/strongly agreed that it was more interesting than other cases.

• 100% agreed/strongly agreed that it was more difficult than other cases.

Some student quotes:

• “I honestly love analytics & sports, and I loved this case because it helped me explore more about analytics in sports. It was also pretty challenging, which made it very exciting to solve.”

• “I liked the context of the case. It was different. It was very challenging, but with guidance it helps.”

• “I liked having a big data set to work with, but I think our program lacks a bit in the courses for programming and using the functions required for this case. In other words, the case itself is super interesting, and the fault is more that we need more training in those areas in the program. All in all, I’m glad to have one more tool in my toolbelt.”

Students reported that the topic was very interesting and provided variety beyond the typical applications they had studied. Even students who struggled with the case were interested to learn more, seeking out online videos of WCR and the Moneyball movie.

There was positive feedback from the students about having a realistic-sized data set to work with (even if they struggled to use it). Another positive comment was that they liked that many elements of what they had learned across different courses (statistics, calculus, optimization) came together in this case.

7. Summary

This case teaches students descriptive, predictive, and prescriptive analytics techniques, motivated by questions from wheelchair rugby. The case includes a realistic data set, extensive teaching notes, and a Python notebook. It has been used several times already with positive feedback from both instructors and students.