Broadening Undergraduate Student Perspectives on Equity in Operations: A Summary of Materials and Experiences

1. Introduction

The increasing prevalence of analytics in students’ daily lives, from social media to sports to finances (Barnes and Bjarnadóttir 2019, Rothaermel 2019, Babier et al. 2023, Roth and Matherne 2023), has made it easier to demonstrate its importance (Bowers et al. 2018). However, this carries the risk of students assuming analytics-driven decisions are inherently correct and unbiased. Cultivating critical thinking about analytics is particularly crucial in areas like diversity, equity, and inclusion (DEI). Although analytics has been used to identify shortcomings in DEI (Johnson and Khojandi 2019) and within analytics communities (Fernandes et al. 2024), there remains a need for culturally responsive teaching in analytics education. Operations management education has given relatively little emphasis to these concerns (Johnson and Chichirau 2020), yet students need to develop the ability to question analytical results used in managerial decision making.

This paper details the delivery of a concentrated three-class, one-week module embedded within a semester-long core operations course (focused on operations management with some analytical concepts). The module is designed to foster transformative learning, specifically in equity. Transformative learning in adult education is defined as the process of critically self-examining one’s beliefs in the face of a challenge to those beliefs, potentially leading to more inclusive perspectives (Mezirow 1975; see Online Appendix).

The short course described in this paper introduces students to unexpected information about antiracism and antipoverty, aiming to create a “disorienting dilemma” (Mezirow 1975). A disorienting dilemma disrupts one’s worldview, prompting a re-evaluation of personal beliefs and biases through cognitive and affective learning, ultimately leading to expanded perspectives. The course content is structured to progress from lower-stakes scenarios to discussions of significant harms and complex decisions. The short course concludes with students critically reflecting on an inequitable business or technology situation, connecting it to course material, and considering whether the affected individual or group should remain in that situation. This final reflection serves as a primary disorienting dilemma.

Integrating critical thinking about DEI into operations management is valuable for several reasons. First, DEI is relevant to both societal well-being and organizational profitability: Evidence suggests a correlation between embedded DEI principles and improved financial performance (Hunt et al. 2018). Second, focusing on the processes underlying inequity, rather than solely on the harms, can enhance DEI progress by deepening students’ understanding of the responsible individuals or systems. Third, examining DEI through the lens of analytics and operations demonstrates that analytical tools and technology can contribute to unequal treatment across various demographic groups, highlighting that bias can affect anyone. Therefore, equipping students with a DEI-focused qualitative framework to evaluate analytical processes and outcomes can be as beneficial as dedicating classroom time to traditional operations management topics.

This paper provides instructors with a detailed description of our approach to integrating DEI content into analytics coursework, particularly emphasizing race and socioeconomic equity issues. It is intended for undergraduate operations management, analytics, and operations research instructors, especially those teaching core operations courses. Following a literature review, the course content, teaching notes, student feedback, limitations, and future opportunities are presented.

2. Literature Review

After an introduction to the transformative learning theory (TLT) of Mezirow (1975), we cite the literature that most shaped our outlook for teaching equity in this short course. For a full review of operations techniques applied to DEI problems in general, see Johnson and Chichirau (2020).

2.1. TLT Approach to Equity Education

Mezirow (1975) developed his TLT to better explain what he saw as the process of personal transformation that adult learners experience in higher education. In the 10-step model of Mezirow (1975, 1998), transformative learning in the college classroom begins when students’ existing beliefs are challenged by an acute crisis or series of provocations (e.g., novel DEI content), described by Mezirow as a disorienting dilemma, leading them to engage in critical reflection by questioning those beliefs in the face of conflicting information (Cohen et al. 2015). Nerstrom (2014) developed a simplified four-phase model depicting the stages of TLT to “provide a visual representation of how transformations are constructed and identify transformative learning as a continuous cycle of learning” (pp. 327–328). Nerstrom’s model is presented in Figure 1 and further described in the Online Appendix.

Despite rather limited published research on integrating transformative learning theory into operations management education (Brunnquell and Brunstein 2018, Brunstein et al. 2021), there is ample evidence of its utility in other disciplines (D’Amato and Krasny 2011, Pittman Brown and Brown 2015, Stone and Duffy 2015, Bernard 2019, Yun-Jeong et al. 2021, Terblanche 2022). As Terblanche (2022) described, “transformative learning…has the potential to facilitate deep, lasting changes in problematic perspectives” (p. 275). DEI topics in operations management education are especially suited to using transformative learning theory methods to influence students to expand not only their knowledge, skills, and awareness, but also their practice of equity. A more in-depth description of transformative learning is presented in the Online Appendix.

2.2. Equity Modeling for Operations Core Course Setting

Educators have noted the difficulty of teaching so-called soft skills in operations (Levasseur 1991, Bowers et al. 2018), and DEI material has similar challenges. Brunstein et al. (2021) emphasized the importance of student exposure to “problems that are considered normal or natural, helping them to identify dominant ideologies and unmasking power that maintains injustice and inequity” (p. 479). Al-Bahrani (2022) described several approaches to improving diversity, inclusion, and belonging, including the use of pop culture economics, visuals that display individuals from diverse backgrounds, and fostering a growth mindset for both students and teachers. Nock et al. (2024) included repeated social justice framing in their graduate-level engineering questions, and over time, students significantly improved their analysis. The approach of Lesser (2007) to include social justice in statistics education, and which examples best capture student attention, was perhaps the most similar to ours.

Although Sebastianelli et al. (2020) observed a change in student attitudes and perceptions after infusing social justice into operations courses, their findings highlight the difficulties in shifting student behavior and intentions. These very difficulties underscore the significant potential advantages of employing innovative transformative learning techniques in DEI education. By encouraging students to critically examine their own perspectives, make decisions about equity dilemmas, and explore their own equity dilemmas as they develop as professionals, our unique transformative learning approach offers a more effective pathway to impact student behavior and meaningfully integrate equity concepts where traditional methods have struggled.

The importance of modeling processes and outcomes within operations education necessitates explaining “institutional discrimination,” where rules and practices that appear neutral systematically benefit (or harm) specific demographics (Haney-López 2000, Bayer and Rouse 2016). Many of the equity issues involve institutional discrimination, where seemingly equitable rules in theory (e.g., hiring from the top 10 universities in a particular field) cause inequity in practice. Regarding employees, Walsh (2020) showed that workers are already trying to manipulate algorithms to achieve better outcomes and risk being trapped by algorithmic feedback. For an employee confronted with inequity, the framework of Hirschman (1970) and others applies for the choice of whether to Stay and advocate for change (Voice) or Go (Exit). Our focus on an employee perspective may help students better envision equity issues.

The Online Appendix contains an additional review of background literature to help the reader understand the context of institutional and algorithmic inequity in operations and analytics education, careers, and applications.

3. Curriculum, Learning Goals, and Teaching Notes

In this section, we describe our curriculum approach to fostering “personal growth and instrumental learning” (D’Amato and Krasny 2011, p. 237). For application of transformative learning theory and organization of the materials, we relied on the model of Nerstrom (2014), as reviewed in the Online Appendix.

3.1. Classroom Setting

Originally delivered at a private liberal arts university in Pennsylvania (four-credit business undergraduate class titled “Quantitative Methods and Operations Management”), the course is now being taught at a private liberal arts university in Virginia (three-credit business undergraduate class titled Operations Management). This paper is based on teaching this material in 2022 and 2023 to small classes (25 or fewer students) composed of about 75% general business administration and management majors, with most of the rest majoring in marketing or accounting. There were no operations major or minor options at either institution, although one (Pennsylvania) offered a data analytics minor in partnership with the math department. Students tended to be sophomore or junior level.

3.2. Learning Goals and Objectives

The three major learning goals for this short course follow the form of Bloom’s revised taxonomy (Bloom et al. 1956, Krathwohl 2002; see Online Appendix).

The first goal is for students to Understand the nature of analytics and technology and be able to explain a variety of applications. Much of this can be accomplished with analytical examples on Day 1. Rather than having students focus on inputs and outputs (intentions of individuals involved and level of harm) in order to make judgments on Day 1, this learning goal enables students to first engage in critical reflection and understanding about the process of analytical models. Then, students can be led to design or study processes to prevent the harm from occurring.

The second is for students to Analyze the assumptions and processes that lead to inequality and break them down into smaller steps and pieces. Many of these revolve around the hiring or evaluation of younger or more junior employees. Such perspective-taking classroom activities are designed to enhance student empathy and encourage the development of a more nuanced analysis of DEI issues (Junn et al. 1995, Block-Lerner et al. 2007, Rapisarda et al. 2011). By humanizing course content, students may feel empowered to interrogate systems and processes that might otherwise seem beyond their understanding, further personalizing their learning within a collaborative classroom.

The third is for students to Evaluate ongoing analytical processes to identify bias and then decide what can or should be done about it. A variety of materials can be adopted, from case studies (Miley 2015, 2016) to first-person video, to evaluation frameworks, and to individual student papers. By focusing on industries students already know and care about (e.g., entertainment, sports, and technology), students may feel more comfortable as evaluators.

The core outside materials used to help create transformative learning experiences in this course are a case study composed of a speech and first-person essay by Miley (2015, 2016) and an article on diversity and equity by Ingram (2021). Miley was a Black engineering manager facing hiring equity dilemmas at Twitter. Ingram outlines the hidden damage done by being a member of a lower socioeconomic class to future career prospects. We also created short in-class group exercises. One was a sporting goods employee hiring exercise, in which students recognize how easily suggested heuristics for hiring create a biased hiring process. In addition, we created a photography and image classification exercise that demonstrates bias in how technology records and interprets images. We recommend instructors develop their own in-class group exercises that match student interest areas.

The learning objectives after finishing this material are for students to be able to:

• Describe analytics and examples of analytics applications and processes, especially concerning hiring and employment (sporting goods in-class example).

• Be able to discuss how analytics and operations processes can contain equity pitfalls and the resulting harms, after working through the Miley (2015, 2016) material.

• Identify specific equity bias in a current process or situation, based on applying the flexible 2 × 2 framework for assessing biased processes. The students will demonstrate this in a paper response to the Ingram (2021) essay.

3.2.1. Three-Day Course Structure.

We detail the curriculum for the short course below. Daily topics, materials, and exercises are described, along with instructional scripts for student assignments and activities. Table 1 provides an overview of the three-day course structure in alignment with the transformative learning model of Nerstrom (2014) presented in Figure 1. Each day is described in detail within its own section. Teaching notes for each specific day are provided in the Online Appendix.

Table 1. Class Schedule for One-Week Course

In this cyclical model, a learner’s experience prompts construction of assumptions; their perspectives are challenged by novel/disorienting content coupled with critical examination of perspectives, leading to transformative learning.

3.3. Day 1 Curriculum

The first day of class is intended to make the point that, although algorithms and analytics (and the technology in which they are embedded) can be an incredible force for good, they can also lead to inequity; therefore, the first part of the lecture is dedicated to demonstrating a few analytical applications of statistics and forecasting. Examples include using analytics to predict which movies will be most successful at the box office or using marketing analytics to assess which demographic groups are most and least enthusiastic about an upcoming movie. These contexts allow students to use previous experience (Nerstrom Stage 1) and also investigate student-constructed assumptions (Nerstrom Stage 2) as to how algorithms are constructed and how analytical results are used. Students are encouraged to first understand the process and then break it down into parts and/or smaller decisions, allowing them to see where and how the process could become discriminatory.

Next, an exercise is introduced regarding the hiring of a sporting goods store employee; this exercise is detailed below. The sporting goods employee group exercise allows students to learn and discuss with peers and focus on a relatable situation (a retail job description and application). Students are also encouraged to think critically about solutions. Based on classroom experience, we also note that it is important that the short course includes graded material and is well integrated into the rest of the class. For example, the group exercise can count for class participation points.

3.3.1. Day 1 In-Class Sporting Goods Employee Exercise.

In this collaborative in-class activity, students are to transform a job description for a typical hourly retail job into a set of rules for hiring, with emphasis on the DEI pitfalls that could be introduced into the hiring process.

Group Exercise. Students are instructed to (1) try to write out rules that fit this job description and (2) go back and think about how these rules might exclude certain candidates. Collaborative discussions prior to writing their critiques are encouraged.

The exercise is shown below:

(script)

“Suppose that you work at [a local] Sporting Goods as an Assistant Store Manager. The Store Manager has asked you to hire someone to work in the Shoe Department.

The Store Manager tells you, ‘I want to hire someone tall and athletic. Oh and make sure they’ve had at least one year of college. We’ve hired that type of person before, and they’ve done well.’

Write out some rules or guidelines you would use to help you hire this person.”

3.4. Day 2 Curriculum

Aligning with Nerstrom’s four-stage model of transformative learning (Nerstrom 2014; see Figure 1 and the Online Appendix), a greater focus on Stages 1 and 2 (experience, assumptions) on Day 1 of the short course prepares students for Stages 3 and 4 (challenge perspectives, transformative learning) on Day 2, beginning with the Leslie Miley Case Study.

In the Miley Case Study, students watch a video and read an essay by Leslie Miley, a Black engineering manager. In the video, Miley (2015) describes his experiences in advising companies that wanted to improve their hiring equity policies in theory but often struggled in practice. In the essay (Miley 2016), he describes his dilemma as to whether to leave Twitter based on their inequitable hiring practices or remain there and advocate for change. The Miley dilemma is presented alongside a contextual framework from the book Exit, Voice, and Loyalty (Hirschman 1970).

The Miley Case Study presents students with a relatable disorienting dilemma (Mezirow 1975) designed to spark stimulating class discussions. Miley’s experience provides a clear example of how a company’s meritocratic principles, transformed into a set of rules/actions, can close the door on deserving candidates from a certain demographic. Another perspective challenge (among several) to students can occur when students attempt to reverse-engineer the process by which an applicant could meet hiring standards and thus realize the hidden process inequity traps that would prevent meeting those standards. By doing so, students are progressing from understanding bias in equity (the first learning goal, addressed in Day 1) to analyzing it in detail.

Students can then grapple with the disorienting dilemma either via an in-class discussion (time permitting) or a take-home reflective essay, as described below. At the end of Day 2, the upcoming paper assignment (Section 3.5.1) should be previewed. This material placement allows the students to draw on previous experiences (and revisit Nerstrom Stage 1) after the Day 2 disorienting dilemma and related challenges to their perspectives.

3.4.1. Day 2 In-Class Leslie Miley Exercise.

In this two-part activity, having just been presented with the Miley Case Study, students are first invited to critically examine potential examples of inequity in hiring systems by taking the perspective of students who, along with Miley, might be faced with hiring bias. Depending on the time it takes to discuss the hiring rules and Miley’s original concerns, the final discussion on whether Miley should leave Twitter can be a group discussion or a take-home essay. The group discussion exercise and the alternative take-home essay are detailed below.

Group Discussion Exercise.

Part 1:

(script)

“Take some time to look at the characteristics that HR prioritized in hiring new workers. Miley says that HR did not want to hire minority workers that he recommended for the following reasons:

1. not being fast enough on timed tests,

2. not having internships at ‘strong’ companies, or

3. took too long to finish their degree.

Provide a quick example of how a college graduate could be a good engineer or computer programmer and still be bad at one of those three things. Hint: Your own experiences may help as to why one of those three things may be difficult for a good student to achieve.” (1–3 sentences)

Part 2:

(script)

“I like this story because it’s a realistic management dilemma that I’m sure some of you will face. While the company’s words are often good (See Leslie’s respect for Jack, the CEO), the company’s actions (mainly in hiring) don’t seem to match their words. In such a situation, it’s difficult to decide whether to stay or go. I want you to put yourself in Leslie’s shoes. Pick one option and make one good point as to why Leslie should do this, in 2–5 sentences:

Option 1: Stay at Twitter and continue your efforts as described in the story, or

Option 2: Leave Twitter and work at another company.”

Take-Home Essay Option

Instead of an in-class discussion, students can reflect on the disorienting dilemma in a written essay.

Take-Home Essay Written Instructions:

“Write a take-home essay on whether Leslie Miley made the right decision to leave Twitter or if he should have stayed. You should use several facts from the article to make your case. There is not a single ‘right’ answer, but be sure to use good facts/logic in defending your answer. Please write at least two full paragraphs to receive full credit.”

Students are graded primarily on how well they justify their recommendation and are instructed that either answer, deciding to “leave the company” (exit) or “stay at the company” (voice; Hirschman 1970), is acceptable. For this assignment, analyzing the dilemma is not sufficient; they must arrive at a decision.

3.5. Day 3 Curriculum

Following Day 2’s disorienting dilemma via case study, a new cycle of transformative learning is engaged (Nerstrom 2014; see Online Appendix), preparing students for a second transformative learning opportunity (Stage 4). Building on students’ previous experience (Stage 1) and constructed assumptions (Stage 2), an instructor-created framework to assess bias in processes is introduced.

At the start of Day 3, the terms design bias and user bias (see the glossary in the Online Appendix) are presented. Design bias is described using scenarios in which the analytics or technology creators or administrators exclude or harm a particular member of a demographic group. The framing from O’Neill (2017) of algorithms as powerful but also possibly opaque and unfair helps provide perspective. Several examples of government programs that ended up intentionally or unintentionally excluding various groups are presented. Design bias should be contextually familiar to students, as many of them can recall examples where bosses, coaches, or other authority figures created rules that end up functionally discriminating against them or their friends.

User bias may be less recognizable by students, but it is a valuable inclusion in classroom discussions. Many algorithms rely on customers, users, and/or employees for correction and input/output adjustment. It is vital for students to think more critically about how an algorithm or process, originally designed to be fair, may be pushed into inequality over time via and in response to user feedback. (Note that the terms input bias and output bias may also be used interchangeably with design bias and user bias.)

Once students understand the first two types of bias, a second dimension is added: whether the bias is intentional or unintentional (Online Appendix) on the part of the designers or users. The bias assessment framework thus yields a 2 × 2 matrix, as given in Table 2. Examples for the table are given from a new product development context.

Table 2. Flexible 2 × 2 Framework for Assessing Biased Processes

The end of the last lecture offers an opportunity for the instructor to invite students to join a call for action. As Nerstrom (2014) described, “Action is the ultimate objective of transformative learning” (p. 329). Recommended actions include taking actions to protect oneself (e.g., considering how much demographic information to disclose on a resume to prevent design bias); thinking critically as a manager using analytics for decision support; and protecting those who might be targets (e.g., opposing user bias as it emerges on social media and/or from peers against specific groups). After the instructor has presented a call for action, the written paper assignment can be introduced as an opportunity for the students to share their responses to the equity material, thus demonstrating potential evidence of transformed perspectives.

3.5.1. Applying the Framework: “The Forgotten Dimension of Diversity” Paper.

For take-home work, the students were to read the article titled “The Forgotten Dimension of Diversity’’ (Ingram 2021) to introduce them to some hidden disadvantages of growing up with low socioeconomic status. In it, Ingram presented statistics and used anecdotes to show that lower socioeconomic status should have just as much (if not more) of an effect on career as membership in other traditionally disadvantaged groups. Much of this information may cause a disorienting dilemma for students, as they consider (perhaps for the first time) how their socioeconomic status will affect their career. Students are then asked to come up with their own example of bias, whether their own, a friend’s, or in the news, and use the course framework of design, user, intentional, and unintentional bias to determine whether they would stay with or leave the company. The assignment script is presented below:

(script)

“Read the article titled ‘The Forgotten Dimension of Diversity.’ It explains how social class growing up (poverty) can have a significant impact on future career options.

1. Come up with a business or technological situation (in the news or personal experience) where there is bias occurring. Your example could be poverty related and similar to the article, or it can be on another topic or group. Explain what the situation is and what bias you believe is occurring. Be detailed in explaining who is being given an unfair advantage or disadvantage (example covered in class: the average height difference between men and women, etc.). If possible, attach an article, screenshot, or other artifact to verify the situation. (Write at least half a page.)

2. Then, explain whether the problem is due to design bias, user bias, or some other issues we’ve discussed. Also explain whether you think the bias is explicit/intentional or unintentional. Connect the issue to some theme or example from class notes. (Write at least one paragraph.)

3. Finally, all things considered, what should the affected group do about it? Here are three options, or you can create one of your own. (1) The affected group should not participate in this situation: they should leave or boycott, and maybe even start their own company or version: or (2) The affected group should do nothing, because… or (3) The affected group should still stay in that situation, but they should work from the inside to try to make things better.” (Write at least one paragraph.)

3.6. Teaching Notes for Successful Course Preparation and Implementation

In this section, we outline general teaching notes to aid the instructor in implementing these materials in the classroom. Day-specific teaching notes are included in the Online Appendix.

3.6.1. General Notes.

We recognize that instructors may be teaching these and related materials in many different settings. As instructors consider how to adapt these materials to their own classrooms, we offer some general suggestions based on our experiences teaching this short course.

• Instructor Preparation:

  1. Understandably, a prospective instructor may want to try a one- or two-day version of this course at first. However, based on classroom experience (see Section 4), the material from Miley (2015, 2016) should not be introduced as a stand-alone case study or assignment. Students first need previous exposure to such dilemmas (such as the group exercise on Day 1). A prospective instructor may also choose to trim the essay so that students are not aware of whether Miley decided to leave his job.

  2. We encourage instructors to spend time in reflection and with other experienced faculty, staff, and perhaps even with a small student focus group before implementation. For example, one colleague suggested that there may be only one or two minority students enrolled in the class at smaller predominantly White institutions (PWIs), and to give said student(s) a heads-up ahead of time if sensitive equity material would be discussed.

  3. Prospective instructors teaching this material must be open to encountering new and unexpected information that may challenge their worldviews as well as those of their students. The instructor’s ability to model their initial discomfort and resultant growth encourages students to engage in transformative learning with less fear.

  4. One seminal resource we particularly recommend is Lesser (2007). Lesser focused extensively on what type of social justice examples seemed to most capture attention and motivate students. An instructor should avoid seeming amoral in teaching social justice topics but also be able to win student attention and trust in more challenging educational environments. The instructor can set the stage for transformative learning by making the covert overt. The instructor can explain the rationale behind algorithms and demonstrate how unchecked algorithms may contribute to system inequality. Additionally, the instructor can share teaching decisions (e.g., we discuss with our students our decision to not include demonstrations of artificial intelligence (AI) that claim to detect sexual preferences from photos). Such appropriate self-disclosure serves as a tool to encourage students to actively question rules and assumptions when inequalities surface.

• Student Background:

  1. The most effective integration of this short course into the existing class will be referencing previous operations material as part of this short course. For example, in the Virginia version of this course, students had already taken a business statistics course and had learned productivity material earlier in the semester. Thus, the short course included equity examples based on ratings of employees’ productivity and of forecasting sales and performance.

  2. Although it is tempting to assume that students will be familiar with many real-world applications, such as statistics in sports or social media technology, in practice, a classroom of diverse students with varied interests may not share many common experiences. Therefore, it is first necessary to establish the basic nature, application, and value of analytics and technology before critiquing their potential to create inequality.

  3. When discussing high-level mathematical theory, some students may shy away from the details of quantitative analysis and processes. Despite any such reluctance or lack of mathematical background, it is important to instill within the students the ability to ask good questions about how various conclusions are reached, or trace back into the process and understand what actor(s) or assumptions may lead to an unequal outcome. If a student at least understands the rules or assumptions themselves (if not the math), they can ask good questions as to what went wrong with the algorithm to yield an unequal outcome.

• Classroom Environment:

  1. Using group exercises is just as key to understanding as it is to the other learning goals. Group exercises asking the students to critique the process of creating forecasts for profitable films or star basketball players can also be well suited as introductory material, depending on student backgrounds. Early on, we recommend merely having the students identify tradeoffs and issues (such as the height requirement for the sporting goods exercise) rather than fully resolving those issues. However, in certain classroom environments, the students may be eager to also solve the problems they have identified or spend more time discussing in group.

  2. Instructors will often want to put data in the hands of their students to inspire students to discover equity answers for themselves (e.g., the data-driven approach in Karaali and Khadjavi (2021) and the traffic ticket data set in Vogiatzis and Kontou (2024)). However, as Lesser (2007, p. 11) explained, “Once students gain tools of statistical reasoning from any collection of examples, nothing can stop them from applying them to any topic they encounter or seek.” Students may want to address a topic that the professor specifically said would not be addressed, or an unlisted, possibly inappropriate topic. Instructors should decide whether to permit such tangents in class (or as selected topics for an assigned paper) and make that decision known to the students.

  3. Although some students may be resistant to equity material, other students may feel that the instructor is not being empathetic or not empathetic enough about equity issues. Smith et al. (2022) described various tips for culturally responsive and relevant education. Smith recommended that this education effort should include the professor critically examining their own attitudes about nondominant groups, understanding local settings, and being prepared for student resistance or opposition to culturally responsive education efforts. Marano (2019) provides guidance for student responses to sensitive topics based on her classroom experience.

In Section 4, we provide evidence supporting the utility of this short course in fostering transformative learning with an emphasis on equity issues. The data presented in Section 4 were collected after teaching this short course in 2022 and 2023 to small classes (25 or fewer students) composed of about 75% general business administration and management majors, with most of the rest majoring in marketing or accounting.

4. Student Feedback and Evaluation

After the first time teaching the week-long short course in Spring 2022, feedback was collected from students via a short pilot survey. The short course was then restructured, and we focus here on the feedback from about 35 students from the subsequent two offerings of the course in Fall 2022 and Spring 2023. The surveys focused on learning, overall satisfaction, and student ability (self-reported) to recognize bias experiences. When asked about their overall learning, 69% (24 of 35) ranked it a four or five out of five points. Overall satisfaction similarly had 66% (23 of 35) as four (“satisfied”) or five (“very satisfied”). For the most part, qualitative feedback was also positive. Evidence of transformative learning was observed in student feedback regarding two disorienting dilemmas (Miley 2015, Ingram 2021). Specifically, 71% of students (25 of 35) ranked their learning experience at a minimum of four out of a possible five. Table 3 contains the exact figures. Finally, students praised the teaching approach with comments such as “I think you hit a vast range of examples that cover a good deal!” and “I think you came from a very neutral POV and (I) appreciated your lectures and assignments on bias.”

Table 3. Breakdown of Student Feedback over the 2022–2023 School Year

Another indicator that student learning improved was in Fall 2022 via a survey given before the equity material in this short course was taught. In this prior survey, 81% ranked their ability to recognize bias in using numbers to hire and fire employees as at least a three out of five. When students were again surveyed after the equity material, 100% of the students ranked their bias recognition at least a three out of five. Students gave feedback such as “[in] my HR role in [Student Club]: I have started to not look at the names on resumes for any unconscious bias (even though I don’t think I have any it never hurts)” and “[I appreciated] being able to see my own bias as a worker.” Students were also connecting the dots with material in other business classes, such as the comment that “I liked learning about it because some forms of bias can be easily overlooked and it’s important to remember it. It was a good refresher from my HR class.”

Finally, a sample of student topics for their written papers (described in Section 3.5.1) is shown in Figure 2. For design bias, students came up with some of the following examples: financial barriers in equestrian sports and U.S. Women’s National Team; cameras, automated soap dispensers, and makeup all not fitted to darker skin tones; and numerous examples of managers showing hiring bias at the students’ jobs. For user bias, students tended to write more personal examples. These included the following: bias at restaurants where customers preferred physically beautiful waitstaff or training bias where women employees were not equally trained or were pushed toward inferior work. Students also wrote about how fellow employees, not just managers, could impede the progress of younger or Asian-American workers to managerial positions in hospitality settings. Note that 18.3% of papers had significant content toward two labeled themes and were thus double-listed. Some of these were fully intersectional (e.g., effects of race compounded by poverty). Others of the two-category papers simply had several examples (e.g., inequity at their previous workplace suffered by one older employee and another lower socioeconomic status employee).

5. Limitations

The DEI material in this short course is condensed to fit smoothly into one week of class material. The materials focus mostly on marketing and human resources applications of operations. Topics such as redlining and systemic bias by governmental institutions (e.g., policing, voting, college admissions) were either not originally covered or have been trimmed to support the focus on process. As reported earlier, the focus on process occasionally means less focus on the mindset of those designing or using the process or less focus on the exact outcomes and harms.

Also, because of time constraints, it is impossible to examine the full spectrum of diversity. The approach of giving students case examples addressing more visible demographic characteristics does mean leaving out other demographics who are suffering inequality. For example, in this short course, there is no clear linkage given to students of the intersectional effects of poverty and race overlap or other common demographic intersectionality. In addition, most inequitable situations that are based on behavior or opinion (e.g., the speech, beliefs, identities, or personal expressions held by a group of people) are excluded, and thus, equity examples dealing with so-called invisible minorities (see the Online Appendix) are not closely analyzed.

Having taught this short course three times, in reporting student feedback and evaluation (see Section 4), the composition and small size of the classes and teaching at a PWI may have influenced the reception of the material and student feedback. In such a small class at a PWI (as small as 12 students), students with strongly felt opinions may have also been concerned about being identified or otherwise nervous about recording their full opinions.

5.1. Future Work

Although it would be impossible to cover all possible aspects of inequity in analytics coursework, the authors contend that by being flexible and responsive, instructors can more closely approach a comprehensive framework of course design for students to better understand the multiple aspects of DEI in operations and analytics. Course development is inherently an iterative process. Given cultural shifts, accommodating student disabilities and learning challenges, and changing language and practices regarding DEI in the workplace and in academia, the authors acknowledge that, ultimately, neither this short course nor any course is ever fully developed.

In addition to increased coverage of intersectionality factors and/or invisible minorities (as described above), the course material could be strengthened by including more coverage of unique voices discussing emerging equity concerns. For example, Shen et al. (2021) detailed how the sudden shift toward anti-Chinese rhetoric at many workplaces was not well responded to by companies, even those that had robust DEI programs and investments in place. An instructor could use this example to illustrate how companies with strong existing DEI programs also need to be prepared to pivot when new biases or targeted groups emerge, and how analytics can help detect such biases (or be misused to target such groups). If done correctly, this could be tied into students recognizing the need for DEI efforts to adopt operations principles of continuous improvement and environmental alertness.

6. Conclusion

Using a transformative learning approach, this short course provides undergraduate students in an operations core course an opportunity to evaluate both outcomes and processes related to antiracism, antipoverty, and general DEI practices in operations and analytics in a way that supports critical reflection, perspective taking, and meaning making. In addition to emphasizing equity in analytics, this approach empowers students to become cocreators of their own knowledge as they are invited to question assumptions, especially their own. Such critical reflection and engagement with course material can be encouraged within a collaborative classroom culture of safety and support. In addition, the material includes perspectives from a variety of first-person sources, thus allowing students to put themselves in the shoes of the people discussing the harms and thereby engaging in cognitive and affective learning. Finally, using the stay or leave framework (see Section 3.2) and the novel emphasis on user bias (see Section 3.5) can help personalize the material to a student level. Students can thus better experience the equity dilemmas, enhance their perspectives, and be equipped to recognize and remedy inequity in systems. Evidence of meaning making and transformed perspectives in student feedback (see Section 4) supports the utility and future expansion of this approach to teaching in operations management classrooms and beyond.