Sense of Belonging in the Industrial Engineering Classroom

1. Introduction

Women and minorities continue to be underrepresented in science and engineering. Women account for 20% of bachelor’s engineering degrees awarded and underrepresented minorities do as well (National Science Foundation 2017). Gains in representation have been slow, as women and people who are Black or of Hispanic origin comprised 10.8%, 4.5%, and 4%, respectively, of the engineering workforce more than two decades ago, in 2002 (U.S. Bureau of Labor Statistics 2002). Today, those proportions have risen only to 15.1%, 7.1%, and 6.1%, respectively (Martinez and Christnacht 2021, U.S. Bureau of Labor Statistics 2022). In industrial engineering, women are more represented than in engineering in general, comprising 24% of the workforce in 2022 (U.S. Census Bureau 2022). People who are Black or of Hispanic origin, however, are less represented in industrial engineering than in engineering in general, making up 4.4% and 5.3%, respectively, of the industrial engineering workforce (U.S. Bureau of Labor Statistics 2022).

Organizations have worked to increase the number of women and minorities entering STEM fields for years, so the reasons why the representation gaps persist remain open questions (Meiksins et al. 2021). The choice to pursue a particular field of study, and persist in it, can be dependent on environmental and psychological factors (Good et al. 2012, Meiksins et al. 2021). For example, an analysis of secondary data from the 2002 Educational Longitudinal Study found that having a growth mindset about math—the belief you have the ability to get better at math—predicts college STEM achievement and STEM career attainment (Seo et al. 2019). Another factor that may be important to persisting in a field is sense of belonging. This refers to “one’s personal belief that one is an accepted member of an academic community whose presence and contributions are valued” (Good et al. 2012, p. 701). Studies have established a relationship between sense of belonging and achievement and persistence among university students (Walton and Cohen 2007, Pittman and Richmond 2008).

For college students generally, demonstrated predictors of sense of belonging include race, income, and parent education, with white, high-income, and not-first-generation college students reporting a greater sense of belonging to their educational programs than their counterparts (Meeuwisse et al. 2010, Ingram 2012). Some of these predictors have been identified specifically for engineering and related fields as well. For example, females show a lower sense of belonging in STEM fields than males (National Academies of Sciences, Engineering, and Medicine 2016). Underrepresented minorities in engineering have a lower sense of belonging to engineering as a discipline than their majority counterparts (Jordan and Sorby 2014). Race and gender are predictors of belonging in math and computing fields (Good et al. 2012, Sax et al. 2018).

Because sense of belonging can increase achievement and persistence, a number of studies have tested interventions targeted at improving sense of belonging in engineering (e.g., Jordan and Sorby 2014, Al-Qudah et al. 2018, Rhee and Johnson 2019). Values affirmation exercises encourage students to reflect on their personal values and why those values matter (Verschelden 2017). These exercises have been shown to improve performance and retention and protect students from threats to their sense of belonging (Cook et al. 2012, Verschelden 2017). Exposure to in-group (e.g., same race, same gender) role models through one-on-one contact or media exposure generates greater feelings of belonging in engineering and STEM fields (Dasgupta 2011, 2016). Helping women find greater belonging in their engineering programs relates more to their intentions to persist in engineering rather than fostering belonging outside of their engineering programs, for example, through extracurricular activities and other pursuits (Glisson 2023).

This research is motivated by the desire to improve industrial engineering persistence and retention, especially among women and other underrepresented minorities in engineering. It examines how selected course components implemented during an academic semester in applied industrial engineering/operations research courses impact students’ sense of belonging to the discipline. The course components include (i) assigning values affirmation exercises, (ii) inviting guest speakers with diverse backgrounds, and (iii) giving reading assignments for journal articles with diverse authorship. The inclusion of values affirmation exercises is selected based on the findings of Verschelden (2017) and Cook et al. (2012). The latter two course components are aimed at providing exposure to in-group role models within the industrial engineering curriculum rather than outside it (Dasgupta 2011, 2016; Glisson 2023).

To assess how the selected course components impact sense of belonging, we administer surveys to control groups (courses in which the components are not delivered) and intervention groups (courses in which the components are delivered) at the beginning and end of the semester. The surveys measure sense of belonging with an abbreviated three-item scale adapted from the Math Sense of Belonging Scale (Good et al. 2012). Similar three-item instruments have been used to measure sense of belonging in the computing field (Sax et al. 2018, Lewis et al. 2019). Further details and justification for the adapted three-item scale are provided in Section 3.2.

The hypotheses we test using the collected responses are as follows:

These two primary hypotheses are examined on the entire sample and on subgroups who self-identify as belonging to historically marginalized populations. Because baseline differences in sense of belonging may impact observed changes during the term, we propose two supplementary hypotheses to identify these differences between groups:

These supplementary hypotheses are examined on the entire sample.

The remainder of this paper is organized as follows. Section 2 describes the intervention group courses and provides details of each component used in the intervention group courses. Section 3 documents the instrument used to measure sense of belonging and describes the experiment design and statistical methods used to test whether course components are effective in increasing sense of belonging. Section 4 presents results. Section 5 highlights key findings and provides instructor takeaways. Finally, conclusions, limitations, and suggestions for future research are provided in Section 6.

2. Course Components

This section describes the courses in which the components were administered and provides details of each component and its implementation in the courses.

2.1. Courses Selected for Component Delivery

The course components are implemented in select industrial engineering (IE) and operations research (OR) courses at two universities in the United States, denoted U1 and U2. Both universities are R1 Research and land-grant institutions, and both offer undergraduate and graduate degree programs in IE. At U1, the selected course is required for IE undergraduates, is delivered in-person, and is intended for junior- and senior-level students. Its focus is the application of IE and OR to transportation logistics. The course components were implemented in the Spring and Fall 2023 offerings of this course, and the total enrollment across both was 72 IE students. At U2, two courses are selected for component delivery. The first is the Fall 2022 offering of a blended enrollment and blended delivery course in OR applications in transportation logistics. The 29 students enrolled in this course included 3 undergraduate and 22 graduate students in IE, OR, or engineering management, plus 4 students in non-IE-related graduate degree programs (civil engineering, grain science, and electrical/computer engineering). This course included 11 in-person students and 18 fully asynchronous online students. The second course selected for component delivery at U2 is the Fall 2023 offering of an in-person OR methods course intended for junior-level IE students. Enrollment in this course was 30 students.

2.2. Values Reflection Course Components

Two values reflection exercises are included in the course interventions. Both are modified only slightly from those in Verschelden (2017). In the first exercise, students are presented with a list of values and asked to identify the 10 values from the list most important to them, and from those, their top three values overall. Next, students are asked to write a short letter explaining to another student why those top three values are important to them and what difference they have made in their life. The students are asked to focus on their thoughts and feelings rather than how well written the letter is. The letter is not actually shared with another student. A copy of this exercise is provided in Appendix A. In the second values reflection exercise, students are asked to write a letter reflecting on the key points they will take from the course and the connections to their prior knowledge, life experiences, and top three values. A copy of this exercise is provided in Appendix B.

At both universities, the first exercise was assigned midterm and the second exercise was assigned at the end of the term. At most 10 minutes of class time was used to introduce the activities, with the students completing the reflection letters outside of class. Together, the two assignments comprised approximately 5% of the overall course grade. Sample rubrics for evaluating the values reflection exercises are in Appendix C.

2.3. Exposure to In-Group Role Models Course Components

Two course components aim to expose enrolled students to in-group role models. First, guest speakers from industry are invited to visit the course. The guest speakers are allotted 50 to 75 minutes (one class meeting) and are asked to connect course topics to real-world experiences in their profession. For example, at U1, the guest speakers work for transportation logistics companies and describe projects in which they have applied industrial engineering and operations research techniques to solve transportation problems for their employer. Attention is paid to recruiting guest speakers who provide multiple groups of students with exposure to in-group role models. Guest speakers from two companies visited each course at U1 in this study. The two companies visited the course on separate days and sent between two and five representatives each. At U2, guest speakers from two companies visited the Fall 2022 course; the two companies visited on separate days and each sent one or two speakers. One speaker visited the U2 Fall 2023 course. At U1, the students were asked to reflect on what they learned from the guest speakers and submit a one-half- to one-page essay worth approximately 1% of the overall course grade. A sample guest speaker reflection assignment is provided in Appendix D. No graded follow-up exercises were associated with guest speakers at U2.

The second set of course components in this category is reading assignments and discussions of journal articles. Journal articles are selected on the basis of their connection to class topics and on having authors who provide multiple groups of students with exposure to in-group role models. In the courses at both universities, two to three journal articles are assigned for discussion. Each journal article discussion is allotted 30 to 45 minutes and begins with the article authorship, with author names and their current affiliations and titles introduced. Professional headshots for authors are displayed on the classroom screen for this portion of the discussion. Then, the technical content of each article and its connection to the course are presented. At U1, students were asked to submit a reading assignment writeup worth approximately 2% of the overall course grade prior to the class meeting in which the reading would be discussed; students in the U2 Fall 2023 course were also given this assignment. A sample reading assignment is provided in Appendix E. No graded exercises were associated with journal article reading assignments at U2 in Fall 2022.

3. Evaluation of Sense of Belonging

In this section, we describe the experiment design used to test whether the course components are effective in increasing sense of belonging among enrolled students. We outline how control and intervention groups are determined, document the instrument used to measure sense of belonging, and discuss response collection procedures. Finally, we describe statistical methods used to evaluate our hypotheses.

3.1. Experiment Design

Control and intervention groups were designated at both universities. Details of the universities, academic terms, course types, student levels, and numbers of students enrolled in the courses selected for control and intervention groups are summarized in Table 1. Course design is identical for all students enrolled in a course, regardless of whether individual students choose to opt into the research study according to the procedures outlined in Section 3.3.

Table 1. Details Including Term, Course Type, Primary Student Level, and Student Enrollment for Courses Selected as Control and Intervention Groups in the Study Described in This Paper

3.2. Measurement of Sense of Belonging

Sense of belonging is measured with a three-item scale based on established measures. The Math Sense of Belonging Scale is a 30-item instrument with a mix of positively and negatively worded items, such as “I feel that I belong to the math community” and “I feel like an outsider” (Good et al. 2012). The Sense of Belonging component of the Perceived Cohesion Scale contains three items, such as “I feel that I am a member of _____ community” (Bollen and Hoyle 1990). We adopt the brevity of the measure from Bollen and Hoyle (1990) and update items from Good et al. (2012) with domain-specific wording to obtain

• I feel like I belong in industrial engineering (SB1);

• I see myself as an industrial engineering person (SB2);

• I feel like an outsider in the industrial engineering community (SB3).

Students are asked to rate their level of agreement with each statement on a Likert scale (1 = strongly disagree; 5 = strongly agree), in keeping with studies such as Lewis et al. (2017) and Sax et al. (2018). Responses to these items are recorded and aggregated into a single aggregate sense of belonging score, SBA. Note that item SB3 is reverse coded; therefore, we introduce SB3′ = 6 − SB3 to account for this difference in item direction, and compute SBA as SB1 + SB2 + SB3′. This results in a single measure (SBA) with range 3 to 15, as in Sax et al. (2018).

3.3. Recruitment of Survey Participants

To determine how changes in sense of belonging are associated with the study intervention, the three-item instrument was administered twice in each course designated as a control or intervention group. The first administration began approximately in week 2 of the academic term, and the last administration began in week 15 (just before the end of the term). In all cases, the response collector remained open for approximately one week. We note that university closures because of inclement weather delayed the timing of the first administration of the survey in the Spring 2023 intervention group at U1.

For each survey administration in a designated Fall 2022 or Spring 2023 course (control or intervention), every enrolled student received a recruitment email inviting them to complete an online survey. The survey consisted of the three sense of belonging items plus demographic questions. A copy of the survey is provided in Appendix F. Before taking the survey, students read an electronic debriefing document describing the research study, why they were being asked to participate, and their rights as a participant, including the right to choose not to participate with no penalty. Students received reminder emails at scheduled intervals until either they completed the survey or response collection ended. For each survey administration in a designated Fall 2023 course, the debriefing document was circulated in hardcopy and the survey was administered in hardcopy during a course meeting. The change to hardcopy administration was implemented after observing low response rates to online surveys in Fall 2022 and Spring 2023. This study was conducted with Institutional Review Board approval.

3.4. Statistical Tests Used to Evaluate Hypotheses

Our primary hypotheses concern the impact on sense of belonging associated with the selected interventions. We expect that the interventions will result in greater sense of belonging at the end of the semester than what was measured at the beginning of the semester (Hypothesis 1) and that sense of belonging will be unchanged in classes where no interventions are implemented (Hypothesis 2). We use the Wilcoxon signed rank test to make inference on the change in sense of belonging for each group. This nonparametric test is selected because we do not expect that the data will satisfy assumptions for standard parametric tests (Hollander and Wolfe 1999, Wackerly et al. 2002). This test is used with paired observations from a single population to detect shifts in location (median) for the distribution of the variable of interest arising from an intervention.

Because baseline differences in sense of belonging may impact observed changes during the term, we propose two supplementary hypotheses to identify these differences between groups. We expect that intervention groups will exhibit greater changes in sense of belonging from the beginning to the end of the term than will control groups (Hypothesis 3). Further, we expect that sense of belonging at the beginning of the term will be no different in intervention and control groups (Hypothesis 4). We use the Wilcoxon rank sum test to make inference on these potential differences between intervention and control groups (Hollander and Wolfe 1999, Wackerly et al. 2002). This test is used with observations from two separate populations and is based on a two-sample shift model. It detects whether the distribution of one population is shifted with respect to the second population, or in other words, whether one cumulative distribution function is stochastically greater than the other.

In all statistical analyses, we use significance levels less than or equal to $0.1$. We report p-values for the signed rank tests. Because the conclusions for the Wilcoxon rank sum test are drawn by comparing the test statistic to entries in a values table for upper tail probabilities, exact p-values are not available for this test. Instead, in cases where the null hypothesis for the Wilcoxon rank sum test is rejected at the $0.1$ level, we report the smallest significance level that supports this conclusion. When reporting about intervention groups, we provide results that include all courses and results that exclude the U2 Fall 2022 course because of its mixed-modality, mixed-population delivery.

4. Results

This section summarizes the survey responses received, followed by descriptive statistics and detailed analyses of matched responses.

4.1. Responses and Internal Consistency

A total of 78 pre- and 68 postsurvey responses are received from intervention groups combined, whereas 55 pre- and 56 postsurvey responses are received from control groups combined. Our analyses are conducted using only those respondents who completed both the pre- and postsurvey in their respective courses. Prior to analyzing the data, we removed two anomalous responses involving the sense of belonging scale in which a student selected either “strongly agree” for all items or “strongly disagree” for all items. Because the third item is negatively worded, these response combinations may indicate lack of attention or misunderstanding of the question. This yielded 51 matched responses from the intervention group and 32 from the control group.

Before analyzing the results, we assessed internal consistency for the three-item scale that was used to measure sense of belonging in industrial engineering by computing Cronbach’s coefficient alpha ($\alpha$) (Cronbach 1951). For all matched responses to the presurvey, $\alpha =0.70$; for all matched responses to the postsurvey, the value is $0.79$. The corresponding values when the U2 Fall 2022 intervention course is excluded are $0.65$ and $0.72$, respectively. These values indicate acceptable internal consistency, supporting the conclusion that the three items in the scale are measuring the same underlying phenomenon. In the following subsections, we present descriptive statistics for the aggregate sense of belonging score (SBA), as well as the inferential statistical test results for our two main and two supplementary hypotheses.

4.2. Descriptive Statistics

In total, there were 51 matched responses from intervention groups and 32 from control groups. Table 2 summarizes the associated group sizes and the median values for the aggregate sense of belonging (SBA) at the beginning and at the end of the semester. Figures 1 and 2 illustrate the frequency distributions for the respective groups’ pre- and postsurvey responses. The intervention groups exhibit observable gains in SBA between the beginning and end of the semester overall and for three out of the four courses. The median SBA exhibits a slight decrease for the control group overall. Although the median SBA for both intervention and control groups on the postsurvey is 13, the presurvey value for the intervention groups overall is lower than that for the control groups.

Table 2. Number of Matched Pre- and Postsurveys for Each Intervention and Control Group, Along with Median Values for Aggregate Sense of Belonging (SBA) for Each Group

Table 3 summarizes students’ self-reported identities based on five survey items. The first of these items asks, “Are you a member of any group that the National Science Foundation considers to be underrepresented in science and engineering? These groups include women, persons with disabilities, or persons who belong to one of these racial/ethnic groups: Hispanic or Latino, black or African American, and American Indian or Alaska Native. Answer ‘yes’ if you belong to any of these groups.” The remaining items gather information about students’ gender, membership in the LGBTQ+ community, race and ethnicity, and first-generation student status. Results are reported for the aggregate intervention and control groups, as well as for each course section. In the intervention group, 45% identify as belonging to a group that the National Science Foundation (NSF) considers underrepresented. Gender minorities constitute 41% of the intervention group, whereas almost 20% identify as LGBTQ+. Underrepresented racial and ethnic identities (American Indian or Alaska Native, Black or African American, Middle Eastern or North African, and Hispanic, Latino, or Spanish origin) are held by about 18% of intervention group respondents. First-generation students represent 14% of the intervention group. In the control group, 41% report an identity that NSF considers underrepresented. Gender minorities and LGBTQ+ individuals make up 44% and 16% of the control group, respectively. Underrepresented racial and ethnic identities constitute about 13% of the control group, whereas 22% are first-generation students. Overall, the identities reported by respondents in the intervention and control groups are similar, although the intervention group has slightly greater racial and ethnic diversity and a smaller proportion of first-generation students than the control group.

Table 3. Respondents’ Self-Reported Identitiesa

4.3. Detecting Changes Within Intervention and Control Groups

The Wilcoxon signed rank test (Hollander and Wolfe 1999, Wackerly et al. 2002) is used to make inference about changes in aggregate sense of belonging measured at the beginning and at the end of the semester pursuant to our two primary hypotheses. We examine these changes within the combined intervention and combined control groups, respectively, as well as for subgroups defined by students’ self-reported identities. Analyses are supported by statistical functions in the SciPy application programming interface (API) (Virtanen et al. 2020).

Hypothesis 1 posits that intervention groups will exhibit a greater sense of belonging at the end of the term than at the beginning. We thus use a one-sided test, wherein the null hypothesis is that the median treatment effect is zero, or in other words, the change in aggregate sense of belonging from presurvey to postsurvey is symmetrically distributed about zero. The alternative hypothesis is that the median change is greater than zero.

Hypothesis 2 concerns control groups, in which we hypothesize that sense of belonging will be no different at the end of the term than at the beginning. In this case, we employ a two-sided test. The null hypothesis is that the median change from presurvey to postsurvey is zero, and the alternative hypothesis is that the median change is not zero.

4.3.1. Full Sample Analyses.

The results for all students in the intervention and control groups are summarized in the first three rows of Table 4. The rows correspond to the group being analyzed (control, intervention including the U2 Fall 2022 course, and intervention excluding the U2 Fall 2022 course) and indicate the associated group size. Pre- and postsurvey medians are displayed, alongside the fractions of the group for whom aggregate sense of belonging increased and decreased, respectively. The final column reports the p-value for the Wilcoxon signed rank test (two-sided test for control groups, one-sided test for intervention groups).

Table 4. Pre- and Postsurvey Median Values for Aggregate Sense of Belonging (SBA) for Each Group, Along with Fraction of Respondents Whose Median SBA Increased/Decreased and p-Value of Wilcoxon Signed Rank Test

For the combined intervention groups, there is a statistically significant positive change between the pre- and postsurvey measures of aggregate sense of belonging ($p=0.00044$ for all intervention groups; $p=0.00068$ when the U2 Fall 2022 course is excluded). We find no statistically significant change in aggregate sense of belonging from the beginning to the end of the term for the combined control groups ($p=0.10458$).

4.3.2. Subgroup Analyses.

Hypotheses 1 and 2 are next examined for subgroups based on students’ identities. Results for each identity category are summarized in Table 4.

NSF underrepresented groups include all respondents who answered “yes” to the associated question on the survey, where NSF considers underrepresented groups to include women, persons with disabilities, and persons who belong to one of the following racial/ethnic groups: Hispanic or Latino, black or African American, and American Indian or Alaska Native. We find a statistically significant positive change in sense of belonging for this subgroup in intervention courses ($p=0.00336$ for all intervention groups; $p=0.01074$ when the U2 Fall 2022 course is excluded). In the control courses, we fail to reject the null hypothesis that there is no change between the pre- and postsurvey ($p=1.0).$

The underrepresented gender identity category includes students who identified as women, nonbinary, or gender diverse. Among intervention groups, there is a statistically significant positive shift ($p=0.00122$ for all intervention groups; $p=0.00488$ when the U2 Fall 2022 course is excluded). In the control courses, no shift is detected ($p=1.0).$

Among students who answered “yes” to the question about LGBTQ+ community membership, we find a positive change in sense of belonging when all intervention courses are considered ($p=0.01953$). When the U2 Fall 2022 course is excluded from the analysis, we fail to reject the null hypothesis ($p=0.10938$). Although this sample does not demonstrate statistically significant increases, 71% of the students in this group had greater sense of belonging scores on the postsurvey than on the presurvey compared with 14% with lower scores. In the control courses, we find no change in sense of belonging ($p=1.0)$.

The underrepresented racial/ethnic identity category includes all students who selected a response other than White or Asian on the corresponding question. In the intervention group, we find no statistically significant change ($p=0.15625$ both with and without the U2 Fall 2022 course). In this group, sense of belonging increased from pre- to postsurvey for 56% of students and decreased for 11% of students. For the control groups, we fail to reject the null hypothesis that there is no change between the pre- and postsurvey measures ($p=0.5)$.

Finally, we find no statistically significant increase in sense of belonging among students who identify as first-generation college students ($p=0.65625$ for all intervention groups; $p=0.40625$ when the U2 Fall 2022 course is excluded). When all intervention groups are included, 43% of respondents reported an increased sense of belonging at the postsurvey compared with the presurvey and 43% reported a decrease. These numbers are 50% and 33%, respectively, when the U2 Fall 2022 is excluded. Among the control groups, there is likewise no statistically significant change in sense of belonging ($p=0.31250)$.

4.4. Detecting Differences Between Intervention and Control Groups

To better characterize the differences that we observe between our intervention and control groups in Section 4.2, we explore two supplementary hypotheses using the large-sample approximation for the Wilcoxon rank sum test to compare distributions (Hollander and Wolfe 1999).

First, we examine whether the distribution of the changes in aggregate sense of belonging (SBA) between the pre- and postsurvey is the same for intervention and control groups (Hypothesis 3). The null hypothesis is that the distributions of changes are identical for intervention and control groups. The alternative hypothesis is that the intervention group distribution is shifted to the right of that of the control group, that is, that the cumulative distribution function (CDF) of changes is stochastically greater for the intervention group than for the control group. At significance level $0.0003$ for this one-sided test, we reject the null hypothesis and conclude that the intervention group exhibits a greater change in sense of belonging between the pre- and postsurveys; the same conclusion holds at significance level $0.0005$ when U2 Fall 2022 responses are excluded. This is consistent with the results for Hypothesis 1 presented in Section 4.3.1 above.

Based on observations about the median values (Table 2) and frequency distributions (Figures 1 and 2) for the respective groups, it appears that the control and intervention groups may not have the same distributions for aggregate sense of belonging at the beginning of the semester. To test this, we use Hypothesis 4, where the null hypothesis is that the distributions of aggregate sense of belonging on the presurvey are identical for intervention and control groups; the alternative hypothesis is that the distribution for the intervention group is shifted in comparison with that of the control group. This is a two-sided test. We reject the null hypothesis at the $0.0018$ significance level ($0.0108$ when U2 Fall 2022 responses are excluded). To follow up, we repeat the process with a one-sided test and the alternative hypothesis that the control group CDF is stochastically greater than that of the intervention group. At the $0.0009$ significance level ($0.0054$ when U2 Fall 2022 responses are excluded), we reject the null hypothesis and conclude that the control group exhibits greater sense of belonging on the presurvey.

5. Key Results and Instructor Takeaways

In this section, we highlight key results for our primary and secondary hypotheses and summarize what instructors need to know in order to use intervention materials and prompts in their courses.

5.1. Summary of Key Results for Hypotheses

Analyses of matched pairs of pre- and postsurvey responses suggest mixed results for our primary (Hypotheses 1 and 2) and secondary (Hypotheses 3 and 4) hypotheses. A summary of results is provided in Table 5. Gains in sense of belonging over the course of a term are statistically significant for the combined intervention group, confirming Hypothesis 1 for this case. The gains are statistically significant for NSF underrepresented groups, underrepresented gender identities, and also LGBTQ+ identity, when U2 Fall 2022 intervention group data are included for the latter. However, the intervention effect is not significant for the LGBTQ+ identity when U2 Fall 2022 intervention group data are excluded, nor is it significant for underrepresented racial/ethnic identity and first-generation student identity. No statistically significant changes in sense of belonging over the course of a term were detected for the combined control group nor any control subgroup, confirming our second primary hypothesis, Hypothesis 2.

Table 5. Summary of Whether Each Primary and Secondary Hypothesis Is Confirmed via Inferential Statistical Testing of the Response Set

The distribution of gains in aggregate sense of belonging over the course of a term are stochastically greater for the combined intervention group than for the combined control group, confirming Hypothesis 3. However, the distribution of aggregate sense of belonging at the beginning of the term is stochastically greater for the combined control group than for the combined intervention group, not supporting Hypothesis 4. This result should temper the conclusions drawn from Hypothesis 3. Although no gains were detected in the distribution of aggregate sense of belonging scores for the combined control group (in Hypothesis 2), it may be that relatively higher presurvey scores (Hypothesis 4) left little room to detect such changes (Hypothesis 3). This outcome raises questions about factors outside this study’s scope that may impact presurvey scores. Nonetheless, there is still good evidence to support that our intervention is effective in impacting sense of belonging for the intervention group as a whole and for some subgroups (Hypothesis 1).

5.2. Instructor Takeaways

A summary of what is required in order for instructors to implement each intervention component in their own courses is below.

5.2.1. Guest Speakers.

Form connections with industry professionals working in fields related to the course who also provide multiple groups of students with exposure to in-group role models. We relied on our own former students plus our departmental alumni organizations to identify potential guest speakers. If desired, use the Guest Speaker Reflection Assignment in Appendix D as a graded follow-up exercise. Reserve approximately 50–75 minutes of class time per guest speaker visit.

5.2.2. Journal Article Reading Assignments.

Select journal articles that are related to course topics and have authorship that provides multiple groups of students with exposure to in-group role models. Prepare a discussion guide for each selected article. In our case, the discussion guide consisted of a slide deck highlighting the authors and the technical content of the articles. If desired, use the Journal Article Reading Assignment in Appendix E as an accompanying graded exercise. Reserve approximately 30 minutes of class time per article discussed.

5.2.3. Values Reflection Assignments.

Use Values Reflection Exercises 1 and 2 in Appendices A and B and, optionally, the grading rubrics provided in Appendix C. If desired, use 5 to 10 minutes of class time to introduce the first of two exercises.

The values reflection exercises required some instructor effort to determine how to describe them to students in a way that would not bias their experiences with the exercise or bias our study results. A values reflection exercise is an uncommon assignment in an engineering course that focuses on quantitative modeling. Thus, students may wonder what their values have to do with their technical skills, and why the assignment is related to the course. This is especially true for the first values reflection exercise, which asks students only to identify their top values and the difference these values have made in their lives. It is not until the second values reflection exercise that the students are asked to connect their values to the course and to their careers and professional fields. Our advice is for instructors to acknowledge, at the first values reflection exercise, that this assignment will be different from their usual experience with assignments in engineering courses, and that a second related assignment later in the semester will make the connection between the assignment and the course clearer. Many students, after completing the reflection exercises, expressed how much they enjoyed them.

One potential challenge associated with delivering these course components is the class time requirement. If the time available for these components is limited in a semester, it may be possible to limit the paper discussions to two papers and the guest speakers to a single class period, though it is unclear how this might impact the effectiveness of the intervention on increasing sense of belonging. We expect that class time devoted to paper discussions and guest speakers could be reduced and gains in sense of belonging still be achieved, so long as both the guest speakers and paper authors are chosen such that multiple groups of students will have exposure to in-group role models.

Overall, the two instructors enjoyed the experience of incorporating the intervention components in their courses. Both observed that doing a study on sense of belonging and using values affirmation exercises opened doors to conversations with students that those students may not have felt comfortable initiating otherwise. Anecdotally, the study helped facilitate instructor-student connections, particularly among students from underrepresented groups.

6. Conclusions

Results of this study provide preliminary support for the value of the described course components in achieving gains in sense of belonging for some but not all subgroups. Because of the small sample sizes, we suggest continuing to study this phenomenon.

This study has some limitations. First, the number of matched pre- and postsurvey responses is small compared with total course enrollments. The analyses described in Section 4 are based on 51 matched intervention group responses and 32 matched control group responses, compared with total class enrollments of 131 and 97 in courses selected for intervention and control groups, respectively. Those students motivated to complete both the pre- and postsurveys may not be representative of all students in a course.

A second limitation is that 7 of 51 matched intervention group responses are from students who took two consecutive courses with an instructor at U1. Further examination of those seven matched responses would be required to better understand whether and how this influences aggregate results. We have refrained from conducting that examination at this point in time because of the difficulty of making inferences from the small sample size.

A third limitation of the study is that one of our designated intervention groups, U2 Fall 2022, is from a mixed undergraduate and graduate enrollment course. It was selected for inclusion based on the constraints of what courses each author was offering when the study was designed and initiated. Because this group is unlike other control and intervention groups in our study, statistical analyses for Hypotheses 1 and 2 were conducted with and without data from the mixed-enrollment course included. This limitation is tempered by the observation that excluding these responses changes only the Hypothesis 1 conclusion for the LGBTQ+ identity subgroup.

Possible areas for future research include exploring why the intervention demonstrated an effect for some but not all subgroups, or exploring how to make the intervention more effective for all subgroups. Another possibility is to measure gains from individual components included in the suite of interventions described in this study to identify whether any are more impactful than others. In this case, we do not recommend surveying more often during a single course, because of time limitations, the risk of additional survey burden reducing response, and the risk of revealing that course components are linked to an intervention. Rather, a new experiment to test individual components could be designed; for example, select a single component to include in a semester course (or courses) and measure before and after.

Finally, because our measurement window was limited to a single course, we implemented all course components in a single term for each intervention group. An alternative recommendation is to work across curricula to embed exposure to in-group role models and relate values to coursework and the profession throughout several courses. This could provide the foundation for longer-term studies that measure the relationship between the selected interventions, sense of belonging, persistence, and graduation.

Appendix A. Values Reflection Exercise 1

Why Reflect?

What you learn in college builds each semester, new knowledge builds on prior knowledge, and reflection and reflexive writing help you to develop and clarify the connections between what you already know and what you are learning, between theory and practice, and between what you are doing and why you are doing it.

In this assignment, you will reflect on values that are important in your life. In a follow-up assignment later this semester, you will reflect on the connections between your values and your experiences in this course.

Tasks

1. Consider the list of values in the table below. Circle the 10 values that you consider to be the most important in your life. If there is something you value that’s not on the list, you may add it.

2. Think for a bit about each of those 10 values. Put a star next to the three that are the most important of all of them.

3. Write a short letter (about 500 words) explaining to another student why these top three values are important to you and what difference they have made in your life. Give some examples of things you have done or choices you have made in your life based on these three values. Focus on your thoughts and feelings, and don’t worry about spelling, grammar, or how well written it is.

   Table

   Wisdom	Reliability	Integrity	Enthusiasm
   Winning	Productivity	Inspiration	Efficiency
   Well-being	Power	Initiative	Dignity
   Wealth	Personal growth	Independence	Dependence
   Volunteering	Perseverance	Humor	Curiosity
   Truth	Peace	Humility	Creativity
   Trust	Patriotism	Hope	Courtesy
   Tradition	Patience	Honesty	Courage
   Teamwork	Orderliness	Heritage	Cooperation
   Success	Optimism	Health	Conflict resolution
   Spirituality	Openness	Harmony	Confidence
   Simplicity	Open communication	Generosity	Competitiveness
   Service	Nature	Fun	Competence
   Self-reliance	Mercy	Friendship	Compassion
   Self-esteem	Making a difference	Freedom	Community
   Self-discipline	Loyalty	Forgiveness	Commitment
   Safety	Love	Flexibility	Collaboration
   Sacrifice	Listening	Fitness	Civility
   Romance	Learning	Financial stability	Caring
   Risk-taking	Leadership	Family	Boldness
   Responsibility	Kindness	Fame	Beauty
   Respect	Justice	Faith	Ambition
   Resilience	Joy	Fairness	Adaptability
   Reputation	Job security	Excellence	Achievement
   Religion	Intuition	Ethical behavior	Accountability

Appendix B. Values Reflection Exercise 2

Why Reflect?

What you learn in college builds each semester, new knowledge builds on prior knowledge, and reflection and reflexive writing help you to develop and clarify the connections between what you already know and what you are learning, between theory and practice, and between what you are doing and why you are doing it.

In a reflection assignment earlier this semester, you identified the top values that are important in your life. In this new assignment, you will reflect on the connections between your experiences in this course and your most important values, prior knowledge, and life experiences.

Task

Write a short letter (about 750 words) explaining to another student the key points you will take from this class and their connections to your prior knowledge, life experiences, and most important values. Focus on your thoughts and feelings, and don’t worry about spelling, grammar, or how well written it is.

Use the following guide to assist in developing your letter:

1. Take a moment to revisit the values you identified as important to your life in your prior reflection assignment.

2. Consider the topics and skills that you have studied in this class. Reflect on the key takeaway points that you will carry forward in your education and career, not on lists of facts or model details that were presented in the class lectures.

3. As you reflect, consider the following questions:

   1. What did you learn this semester?

   2. What did you learn about yourself this semester?

   3. How did what you learned this semester fit with your prior knowledge, what you have learned in other courses, your life experiences, and your most important values?

   4. Can you think of any examples of how you might do your (present or future) job differently because of what you learned in this course?

Appendix C. Values Reflection Exercises Rubrics

Table C.1 lists the rubric items used to evaluate the values reflection exercises. Numbers in the second column of the table indicate the approximate percentage of assignment points allocated to each rubric item.

Table C.1. Grading Rubrics for Values Reflection Exercises 1 and 2

Table C.1. Grading Rubrics for Values Reflection Exercises 1 and 2

Exercise	Wt.	Rubric item description
1	30%	Identifies 10 most important values to you, and of those, 3 most important of all
1	35%	Explains why each top value is important and what difference it has made in your life
1	35%	Provides examples of things you have done or choices you have made based on top values
2	25%	Focuses on big ideas and concepts, not facts presented in class (reflective)
2	25%	Indicates how what you learned relates to your top values (relationship to values)
2	25%	Indicates how you what you learned relates to life experiences and/or other classes (relationship to prior knowledge)
2	25%	Indicates how what you learned relates to your field of study and your profession (application to profession)

Appendix D. Guest Speaker Reflection Assignment

Guest speakers are important and invaluable experiences for connecting what you are learning in your classes to how you might apply this knowledge in your future careers. In this assignment, you will take some time to reflect on what you learned from the guest speaker presentation. How did it relate to knowledge you already have? How might it inform your future professional choices, including, for example, your career plans and/or additional courses you may want to take?

Task

Prepare a 350–500-word reflection addressing what was significant about this guest speaker experience. Some writing prompts that you may find helpful are below. You are not expected to address all of these in your reflection; they are provided more as a starting point to help you reflect. The key is to be reflective and insightful. You will be graded on how well you connect specific details of the guest speaker experience with your overall processing of the experience, as well as on the quality of your writing in general.

Reflexive Writing Prompts to Consider

• How did the information shared inspire or motivate you?

• Would you consider a career in the guest speaker’s field? Why or why not?

• What was the most memorable aspect of the guest speaker experience? Why?

• Can you relate some of what the guest speakers discussed with what we have learned in class? How?

• What did you learn about the guest speaker’s company and its employees?

• What knowledge and skills are you learning in college (this class and others) that will be useful at the guest speaker’s workplace? Please explain.

• What knowledge or skills do you need to strengthen in order to be successful at the guest speaker’s workplace? What will you do to strengthen these skills? Please explain.

• What did the guest speaker presentation add to your understanding of the field of logistics?

• The part of this experience that….

  • ○ stuck with me was….

  • ○ excited me the most was…

  • ○ surprised me was…

  • ○ bored me was…

  • ○ (Make sure to address the “why” in all of the above.)

• This made me imagine that in the future I could…

• This made me remember…

Appendix E. Journal Article Reading Assignment

Research the authors of the assigned journal article. For each author, prepare a table or bulleted list that identifies elements (a) to (c) below. Put this table or bulleted list in your Reading Article Writeup.

1. Their full name and any degrees/acronyms listed with their name

2. A thumbnail image/headshot

3. Current position title and affiliation (where the authors are today, not where they were at the time of publication for the journal article)

Fully read the Abstract, Introduction, and Conclusions of the assigned journal article. In addition, skim the remainder of the paper, without diving too deeply into technical details. Include the following elements (a) to (e) in your Reading Article Writeup:

1. The full citation of the article

2. The outline of the paper, including all headers and subheaders

3. A bulleted list of three to five main points of the paper

4. A bulleted list of two to three new items you would know that you didn’t know before reading the paper, if you read the entire paper (including all technical content) in detail (i.e., new knowledge)

5. A bulleted list of two to three new skills you would have that you didn’t have before reading the paper, if you were to read the entire paper (including all technical content) in detail (i.e., new skills)

Appendix F. Survey Questions

(Italicized text enclosed in angle brackets (e.g., <explanatory text>) indicates information that was not on the survey instrument and that has been added here for clarity.)

===

We would like to ask you some questions about your experience with industrial engineering courses and in the industrial engineering community. When we mention the industrial engineering community, we are referring to the broad group of people involved in the field, including students in an industrial engineering course and people working in related careers.

We would like you to think about your membership in the industrial engineering community. Because you are taking one or more industrial engineering courses, you could consider yourself a member of the industrial engineering community. Given this broad definition of belonging to the industrial engineering community, please respond to the following statements based on how you feel about that community and your membership in it.

There are no right or wrong answers to these statements. We are interested in your honest reactions.

Question 1

Please read each statement and select the number that reflects your degree of agreement with the statement (1 = strongly disagree; 5 = strongly agree).

<responses for each item presented in a grid, with Likert scale descriptors strongly disagree, somewhat disagree, neither agree nor disagree, somewhat agree, strongly agree>

<SB1> I feel like I belong in industrial engineering

<SB2> I see myself as an industrial engineering person

<SB3> I feel like an outsider in the industrial engineering community

The remaining questions ask about demographic characteristics. Students’ experiences may differ based on these characteristics. These questions are optional. Your answers will help us better understand students’ experiences in the industrial engineering community.

Question 2

Please indicate the degree program(s) you are currently enrolled in:

• BS in Industrial Engineering

• BS in another discipline

• MS in Industrial Engineering, Operations Research, or Engineering Management

• MS in another discipline

• PhD in Industrial Engineering

• PhD in another discipline

Question 3 <if selected an undergraduate degree above>

What is your current academic standing? Please base the classification on how many hours you have already completed.

• Sophomore (30 to 59 hrs)

• Junior (60 to 89 hrs)

• Senior (90+ hrs)

• I’m not sure

Question 4

Are you a first-generation college student? (First-generation students are defined as those who do not have a parent/guardian who earned a four-year degree.)

• Yes

• No

Question 5

What is your current cumulative GPA?

• 3.5 or above

• 3.0–3.49

• 2.5–2.99

• 2.0–2.49

• Under 2.0

• Unsure or prefer not to answer

Question 6

Are you a member of any group that the National Science Foundation considers to be underrepresented in science and engineering? These groups include women, persons with disabilities, or persons who belong to one of these racial/ethnic groups: Hispanic or Latino, black or African American, and American Indian or Alaska Native. Answer “yes” if you belong to any of these groups.

• Yes

• No

• Prefer not to answer

Question 7

What is your gender identity?

• Woman

• Man

• Nonbinary or gender diverse

• Prefer to self-describe

• Prefer not to answer

Question 8

Do you identify as a member of the LGBTQ+ community?

• Yes

• No

• Prefer not to answer

Question 9

What is your race/ethnicity? Please select all that apply.

• American Indian or Alaska Native

• Asian

• Black or African American

• Hispanic, Latino, or Spanish origin

• Middle Eastern or North African

• Native Hawaiian/Pacific Islander

• White

• Some other race (please specify)

• Prefer not to answer