Investigating How Social Justice Framing for Assessments Impacts Technical Learning

1. Background

Traditionally, engineering curricula have primarily used deductive instruction in which a professor delivers lectures, heavy in theoretical concepts, with limited application of the principles to real-life engineering (Narayanan and Adithan 2015, Karabiyik et al. 2022). This misses the opportunity to train students to recognize, consider, and evaluate how engineering products will intersect with and impact society. It can be argued that the purpose of engineered systems is to address society’s needs (Apelian 2011). Yet, traditional theory-based engineering education promotes disengagement with social issues (Cech 2014, Nasser and Romanowski 2016, Morgan et al. 2020) leaving students ill-prepared to engage with the broader society and to understand the role that their work plays in creating or upholding inequities. Incorporating social justice into engineering courses and curricula can reverse this and extend students’ understanding of how to include and assess the role of technology in society.

Social justice is defined as the state in which (a) benefits and burdens in society are dispersed equitably in accordance with some allocation principle (or set of principles), (b) the procedures and rules that govern decision making processes are inclusive, and (c) all persons involved are treated with dignity and respect (Jost and Kay 2010). Radical changes in engineered systems, for example, the federal highway system (Nall 2018) and broadband Internet access (LaRose et al. 2007), have often been drivers of massive social inequities. Over the past four decades, engineering colleges have put tremendous effort into improving engineering education by developing student-centered pedagogies (Delyser et al. 2003), developing active-learning techniques (Prince 2004, Christie and De Graaff 2017, Nock 2020), and improving the recruitment and retention of women and minority groups through culturally inclusive and responsible teaching (Villegas and Lucas 2002, McIntosh 2020). There have also been efforts to teach students to become more innovative thinkers through comparisons of analysis or on experiential learning (Seidel et al. 2020). These efforts in engineering education innovations, along with the traditional technical focus, have led to high levels of technical competence within the graduating engineers but often misses creating a deep comprehension of how engineered systems impact the broader socioeconomic and environmental spheres (King 2008, Apelian 2011, Cumming-Potvin and Currie 2013) that encompass social justice.

Culturally relevant and social justice education aims to shift this paradigm by integrating the human dimension into engineering curricula, increasing the number of diverse perspectives in the classroom (Dover 2013) and integrating social justice into the course content. Culturally relevant education is a theoretical model for instructing students that focuses on multiple aspects of achievement and advancement in the classroom, while encouraging students to uphold their cultural identities. A key foundation of culturally relevant education is using it as a framework for integrating social justice into education and using the classroom as the starting place for social change. Much of the work has addressed challenges and opportunities for integrating social justice into the education systems and has focused on many topics ranging from curriculum and students (Snyder et al. 2008, Dimick 2012) to teachers, for example, a teacher’s ability to teach to the whole student and incorporate multiple backgrounds and perspectives into the classroom (Aronson and Laughter 2016). One potential benefit of incorporating culturally relevant education into classrooms is that this will translate into socially relevant projects when students progress to engineering firms.

Engineering has enormous implications for all members of society, thus requiring engineers to be able to incorporate many different perspectives and potential stakeholders into their decision-making processes (Mazzurco and Daniel 2020). There is a complex relationship between the individual cultural identities that each person brings to the table and the continuum of privilege and harm that each of these identities impart on the individual and other members of society (Wijeyesinghe et al. 1997, Snyder et al. 2008). A key step in building a socially responsible engineer is training the student to recognize that privilege and power are inequitably distributed throughout society, and how technological inventions play a role in these power dynamics. To accomplish this, some programs have started including diversity, equity, and inclusion in their curriculum (Armanios et al. 2021).

The challenge with overcoming this first step is that students may feel high levels of discomfort when their viewpoints are challenged, or they are reminded of a social injustice that has historically caused their group harm or been caused by members of a group they identify with (e.g., racial profiling, mass incarceration, housing discrimination) (Wijeyesinghe et al. 1997, Candelario and Huber 2002). Integrating these social topics into a university’s engineering curriculum adds an emotionally taxing dimension to classroom learning, which may impede learning of core technical concepts. As the students are coping with anger, denial, guilt, ignorance, naiveté, or a combination of these factors, they may fall into the trap of defaulting to refocusing on purely technical aspects of engineering problems. Yet, an instructor-led social justice analysis may show them that relatively simple frameworks can transform awareness (Wijeyesinghe et al. 1997), increase student commitment for socially responsible engineering, and provide an avenue for understanding how technological advances and engineered systems could exacerbate or alleviate social inequities.

This paper presents a framework and analysis for evaluating how integrating social justice considerations into engineering investment-decision homework problems and classroom discussions impact student learning. Specifically, we examine the following research questions:

1. Does framing engineering concepts within a social justice context aid in the learning of engineering concepts?

2. At the conclusion of a course on social justice and engineering, how well can students integrate social justice into engineering decision making when prompted and unprompted?

A key factor driving the incorporation of social justice into the course content is the instructor’s passion and commitment to integrating the human dimension into the course curriculum. One study found that engineering faculty perceive a strong relationship between engineering and social justice (i.e., poverty, care for the environment, gender equality, and public safety) (Jiménez et al. 2019). However, we recognize that traditional engineering faculty have given lower value to the social side of engineering and far greater attention to the technical concepts in their courses. Although we do not debate the importance of rigorous technical training, it can be argued that quality engineering must take into account social justice (Leydens and Lucena 2017). In particular, the depoliticization of engineering (i.e., the view that social and political issues are tangential to technical decisions) (Cech 2013) can exacerbate social inequalities and injustices by removing the sense of responsibility from those that design engineered systems. Our aim is to highlight how small changes to classroom examples, as well as homework and exam problem framing, can enhance critical thinking skills and promote social responsibility within student groups. This is crucial to developing engineering graduates who seek and promote social justice through critical thinking, detailed impact assessments of their technologies, and participate in reflecting on the potential impacts of their work (Cumming-Potvin and Currie 2013).

2. Literature Review

Our paper is related to the following literature: social justice engineering education and instructor training. Here we touch on both topics. One approach for integrating social justice into the engineering courses, from the humanities curriculum, is to include critical and reflective thought in coursework. This encourages engineering graduates to consider the diverse needs of the communities that will adopt their technologies (Riley 2008). Reflective and critical thought discussions can engage engineering students in considering how their technologies will be adopted and used in society before they are deployed. This could lead engineers to consider who will use their technologies and who will be excluded from them. Some scholars argue that higher education needs to teach students to work on local and global scales and teach students how to evaluate the social, economic, and political implications of their work (Bourn and Neal 2008). Another approach is to incorporate social justice frameworks into project-based learning in ways that empower students to bring their unique perspectives for holistic problem solving (Chiki and Sallar 2021). In project-based classes, the emphasis is often on student voice and enhancing the opportunities to engage with the community (Schneider and Munakata-Marr 2013).

Despite a strong need in the engineering curriculum, these human dimensions get lost in current engineering curricula at all educational levels, leaving a gap between the course content and students’ lives. For engineering to live up to its potential to create a more just society, it must challenge inequalities around the world through “justice engineering” (Tharakan 2020).

There are multiple approaches to social justice engineering education. Hackman (2005) identifies five components for social justice education (i.e., content mastery, tools for critical analysis, tools for social change, tools for personal reflection, and awareness of multicultural group dynamics). Gates and Jorgensen (2009) define a spectrum of “forms” of social justice education ranging from “moderate forms” that focus on equal access to engineering education to “radical forms” that focus on dismantling social inequities through engineering. Additionally, Calabrese et al. (2020) emphasize a student-centered approach where the students choose the problem they want to solve and solution methods with the teacher supporting and enabling the students’ goals. These approaches go beyond active-learning (Lasker et al. 2017), ethics-training (Waugaman et al. 2018, Morgan et al. 2020), and service-learning courses (Morgan et al. 2020) by moving engineering curriculum beyond an equity perspective and into a social justice lens. There have also been efforts to integrate social justice into classrooms through teacher education and discussions regarding how diversity affects the way students learn (Leydens and Lucena 2017). Yet, there is a gap in analysis and methods for quantifying the degree to which integrating social justice into the engineering classroom assignments aids or detracts from student learning of technical concepts, and the degree to which it promotes critical thinking skills. Our paper aims to fill this gap.

Incorporating social justice into STEM education requires teachers themselves to grapple with the topics. Often, there is a tension between “covering” the technical course content and educating students on social justice topics (Winberg and Winberg 2017, Nicol et al. 2019, Xenofontos et al. 2020, Mattheis et al. 2023). However, technical and social justice topics do not have to be in conflict or considered as separate domains, even conceptually, if social justice is infused into the core material in a meaningful way. For example, an optimization course may start with the first objective to minimize costs in an energy system (Cheng et al. 2023) but then may progress to maximizing social benefits such as increasing the number of people connected to the power system (Nock et al. 2020, Sackey et al. 2022, Van-Hein Sackey and Nock 2022), reducing local air pollution emissions (Goforth and Nock 2022, Mayfield 2022), or electrifying rural areas (Akbas et al. 2022, Montañés et al. 2023). Tough social justice issues (e.g., mass incarceration and its link to technology development) can challenge the students to grapple with the ethical and complex nature of decision making. This has led some students to resist the inclusion of social justice and claim that it is outside the boundaries of the course content (Riley 2015).

Along with previously noted innovations in the engineering curriculum, we find that one popular approach to closing the void between the technical and societal impacts of engineered systems has been through integrating stand-alone subjects into dense course content (King 2008, Leydens and Lucena 2017). A benefit of this method is that it provides students with a broad understanding of the intended and sometimes unintended consequences of engineered systems. However, this understanding is often shallow and takes a retroactive view, which does not provide the tools necessary to identify possible consequences in future engineering endeavors.

Much of the literature focuses on training K-12 teachers (Finkel 2018, Wolfmeyer 2018, Mattheis et al. 2023) and assessing student perceptions of including social justice topics into the literature (Liu et al. 2020). Within the limited amount of literature focusing on students’ experiences and understanding of social justice in engineering education, most report qualitative data from reflective activities (Riley 2015, Dodson et al. 2017, Badenhorst et al. 2020). One study proposed using project-based learning to help students link technical issues with social responsibility challenges (Rulifson et al. 2018). Although project-based learning can aid student understanding of social justice ties, this approach is very time consuming and often class-specific. Thus, there is a need for flexible methods to incorporate social justice considerations into course content.

Despite the aforementioned studies, there is a gap in the literature regarding analysis and methods that evaluate how reframing engineering homework and exam questions to include social justice may impact students’ learning of the technical content and their critical thinking about the social implications of engineered systems.

3. Methods

3.1. Course Context

This study involved a graduate level engineering economic decision making and infrastructure investment course offered at a private university in the eastern United States. This course is part of a civil and environmental engineering program and covers operations research topics ranging from uncertainty analysis, decision making, investment planning and siting, engineering economics, and estimation methods. Traditionally, course instructors have used lecture-based instruction to frame investment decisions solely in terms of economic parameters. This approach often fails to include additional human-centered considerations such as social justice impacts (e.g., the number of people impacted by decisions and how these decisions may impact individuals differently).

This research was first deployed during the Fall 2020 semester, in a completely virtual environment, during the COVID-19 pandemic. In Fall 2021, the research was conducted in a completely in-person environment. Many graduate students in the course are civil and environmental engineering majors with aspirations of working in consulting, project management, or sustainability fields. In a class survey, many students indicated that they did not know much about mass incarceration or social justice prior to taking the instructor’s course. Prior to taking this investment planning course, the students in Fall 2020 indicated that many students had taken zero (27%) or only one (31%) course which incorporated social justice concepts into the instruction. The Fall 2021 results were similar with 55% reporting having taken one or no classes that incorporate social justice concepts alongside technical instruction. Given that all students in our sample were MS or PhD students, we feel that this highlights the huge gap in engineering education.

We acknowledge that the demographic makeup of the class could influence our results. The educational background of the graduate students for 2020 was as follows: 23 civil and environmental engineering, 1 engineering and public policy, 2 business, and 2 other majors (architecture and law). Of the students in the class, approximately 35% identified as male and 65% as female. Within the class, 29% of the students self-identified as an underrepresented ethnic or racial minority. In Fall 2021, there were 49 students that remained through the class for the entire semester (started with 55), all of which were civil and environmental engineers. Of the students in the Fall 2021 class, approximately 58% identified as male and 40% as female. Within the class, 79% of the students self-identified as an underrepresented ethnic or racial minority, and 60% were non-U.S. citizens (e.g., international students).

At the beginning of the course, an outside consultant informed the students that some of the materials from class would be used to evaluate student learning. Students were given the chance to prevent their materials from being used in the analysis. The students were not told which assignments would be used in the analysis. This minimized the chance that students would remember the experiment was taking place by Homeworks 3 and 4. Additionally, if a student dropped the course, they were excluded from the analysis of Homeworks 3 and 4. It is noted that some students did not complete all parts of all assignments.

3.2. Methods for Incorporating Social Justice

To infuse social justice into the course, the instructor made two key changes to the traditional lecture-based course. First, the instructor adapted the homework to include social justice concepts. Second, the instructor adapted the class examples to include references to how engineering concepts have impacted social inequalities. We describe these core changes in the subsections below.

3.2.1. Course Materials.

Within the course, the instructor introduced diversity issues and social justice connections to technical decision making by reframing question descriptions in the homework problem sets and quiz problems to be situated in a social justice context.

Homework assignments included social justice components that students had to independently research such as diversity in STEM fields, mass incarceration, impacts of climate change on low-income communities, immigration, and discrimination facing people with disabilities (full homework questions in Online Appendices B–E). For all homework assignments, except Homework 1, the students were asked to complete the assignment individually. In the first week of class, students learned estimation techniques. Traditionally, the first homework assignment asked students to estimate how much of some technology (e.g., televisions or cars) have been deployed in the United States for a given year. The first homework assignment was adapted to include discrimination and diversity challenges by asking the students to estimate the number of minority professionals working in the coal or other parts of the energy sector. Then, students were asked to comment about the uncertainty of their estimates and prompted to discuss social justice. Specifically, they were asked to reflect on how the numbers of minority workers compared with the demographics of the U.S. population and why they thought the numbers differed.

In the second homework assignment, students created data visuals to help build their data analysis and processing skills. In the previous iterations of the class, prior to 2020, the instructor’s teaching materials asked the students to make visuals about transportation travel patterns (Hanig et al. 2023, Lezcano et al. 2023). The professor adapted the data visualization assignment to one where the students had to highlight the problem of mass incarceration in the United States using data from the Prison Policy Initiative (Sawyer and Wagner 2020). The third homework built on the mass incarceration data analysis by asking the students to evaluate whether a company should use prison labor to increase profits through reducing their labor costs. In the technical and social justice version of the homework assignment, the students needed to use economic concepts and equations to evaluate supply and demand, consumer surplus, and implications of changing prices. The question in the third homework set was originally devoid of social justice as follows:

Original question: Toll roads are often used to cover the maintenance costs of different infrastructure projects. The demand of trips per hour on a toll road is given by: p = 30 − 0.2*q.

• (a) If the price is $10, how many trips will be taken per hour?

• (b) At a price of $10, what is the price elasticity of demand?

• (c) At a price of $10, what are total benefits, user costs, and net user benefits (a.k.a., consumer surplus)?

• (d) If price rises to $12, what are changes in: total benefits, user costs, and net user benefits?

• (e) Do you think the tolls should be risen? Why?

• (f) When making this decision, what individuals or groups of individuals should be considered or asked to provide input?

Half of the students in the class received the following revised version of the question to incorporate a social justice lens, which needed to be answered individually and required the same quantitative analysis as the original version of the question. Only the context and numbers given differ between the two versions.

Revised question: A car company builds vehicles for disabled populations. These vans allow people who have lost limbs to drive and experience the independence that comes with increased mobility. One challenge is that the cars are very expensive, and they want to increase the number of customers they are able to serve. This company is deciding whether or not to outsource its car manufacturing operations to a local prison. The demand of cars per year for the handicapped accessible vehicles is given by: p = 80,000 − 0.2*q.

• (a) If the company outsources the manufacturing to the local prison population, then the price will be $25,000. This reduced price stems from the low and nonexistent wages the company will have to pay the prisoners. At this price, how many cars can the company expect to sell per year?

• (b) At a price of $25,000, what is the price elasticity of demand?

• (c) At a price of $25,000, what are total benefits, user costs, and net user benefits (a.k.a., consumer surplus)?

• (d) If the car company decides to keep their operations in house, then they expect the price to rise to $50,000, what are changes in: total benefits, user costs, and net user benefits?

• (e) What do you think the company should do? Why?

• (f) When making this decision, what individuals or groups of individuals should be considered or asked to provide input?

The homework assignments were graded by teaching assistants, and parts (e) and (f) were graded for completion. Parts (e) and (f) are excluded from our statistical analysis. We note that a challenge of integrating social justice into homework problems is the extra wording required to provide context. In the original question, there were 35 words, whereas the social justice version of the problem included 88 words. By more than doubling the word count, the students have more information to cipher through. This may impact students in a time-sensitive setting like a quiz or exam, but the word count may have less of an effect in an untimed homework setting.

3.2.2. Instruction in Fall 2020 and Fall 2021.

Although the technical content covered in the class lectures and the assignments were the same across both semesters, there were a few differences in the instructional strategies. In Fall 2020, the students were in a completely remote environment, and the instructor required preclass quizzes that covered the reading material. The students were also able to watch the recorded class lectures multiple times. In the Fall 2021 semester, the students were in a completely in-person environment, and the instructor did the reading quizzes in class. The instructor noticed lower scores on the reading quizzes and suspects that many of the students were no longer prereading the class material. The final reading quiz was only given in the Fall 2020 semester.

Additionally, in the Fall 2021 lecture, the class sessions were no longer recorded, and the instructor only posted recordings of half of the lectures online. Students in the Fall 2021 class mentioned a desire to have more lectures online.

3.3. Research Design and Data Sources

This study focuses on (1) evaluating the impact of social justice framing on student learning and (2) investigating how well students can integrate social justice into engineering decision making when prompted and unprompted. To address the first point, this study used a within-subject design in which students were exposed to both conditions (questions with social justice context and questions without social justice context) to evaluate how social justice impacts overall student learning. Table 1 displays the outline for how the two groups of students shifted between the treatment and control conditions for a given homework assignment. To address the second point, students were asked two questions in their final exam related to decision making and investments. The first was an engineering decision, and the second question asked the students to trace out the social implications of a technical decision.

Table 1. Within-Subjects Research Design for Homework 3

3.3.1. Homework Assignments.

Throughout the course, each student had the same level of access and instruction to course material. Homework assignments served as the main data source to measure student learning. On a given homework assignment, students alternated between having problems framed in a purely technical context (technical condition) or with a social justice component (social justice condition) (Table 1). For both types of questions, students needed to use the same technical equations from the lecture to solve the homework problem.

In Homework 3, the students were randomly sorted into individual conditions A or B and asked to answer questions individually. Treatment A received the technical version of the economic supply and demand question and the social justice version of the economics supplier equilibrium question. Treatment B received the social justice version of the economic supply and demand question and the technical version of the economics supplier equilibrium question. The same social justice context (mass incarceration) was used for both social justice treatment questions in Homework 3. For each, the students were asked to decide if the company should use prison labor to increase their profits and the number of at-risk LGBTQ youth (supply and demand question) or disabled populations (equilibrium/producer surplus question) they could serve. In a class survey, many students indicated that they did not know much about mass incarceration or social justice prior to taking the instructor’s course. To ensure the students were knowledgeable about the mass incarceration problem in the United States, the students were provided with additional information about these topics (Sawyer and Wagner 2020).

Homework 4 followed a similar within-subjects design, with the students randomly placed into treatment groups A or B. Treatment A received the technical version of the risk analysis question (i.e., teenagers needing to purchase car insurance) and the social justice version of the Monte Carlo uncertainty analysis question (i.e., reducing air pollution from the electricity sector). Treatment B was given the social justice version of the risk analysis question (i.e., a teenage driver with a disability is being charged a higher insurance premium), whereas the Monte Carlo uncertainty analysis question was purely technical (i.e., investment decisions for which power plants would make the most profit). See Online Appendices B–E for the full question descriptions.

3.3.2. Exams.

Exams were used to test (1) student learning (similar to the homework assignments) and (2) students’ ability to tease out how engineering decisions impact social inequalities. To test student learning, the instructor incorporated a previously discussed social justice topic (homeless youth) into a question about how stakeholders would make decisions. The topics tested included utility theory, decision trees, and uncertainty analysis concepts. Half the class received the question framed in a social justice component (homeless youth). The decision tree used in the question can be seen in Figure 1, and the two variants for the exam question were as follows:

1. Technical: Based on the following decision tree, which of the following statements is true?

   • ○ Anyone who is risk seeking will prefer Porch over Indoors.

   • ○ Anyone who is risk averse will prefer Indoors over Outdoors.

   • ○ An expected value decision maker does not clearly prefer an alternative.

   • ○ “None of the above” or “More than one of the above” or “Insufficient information”

2. Social Justice: Company A in Pittsburgh regularly holds fundraisers for homeless youth searching for housing. They are deciding where to host their annual benefit event. Use the following decision tree to determine which of the following statements is true.

   • ○ Anyone who is risk seeking will prefer Porch over Indoors.

   • ○ Anyone who is risk averse will prefer Indoors over Outdoors.

   • ○ An expected value decision maker does not clearly prefer an alternative.

   • ○ “None of the above” or “More than one of the above” or “Insufficient information”

The technical version of the question is short (14 words), whereas the social justice version of the question is more than twice as long (38 words). The increased word count reflects the challenge of social justice framing requiring more outside information than purely technical questions. However, this additional word count may impact students more on timed assessments (e.g., exams) than untimed assessments (e.g., homework). This multiple-choice question was graded by an online software.

Students’ ability to think critically about the implications of engineering decisions and tease out social justice factors were evaluated using two questions in the final exam. At the conclusion of the semester, in a final exam, the instructor asked students to (1) trace out the social implications of a technical investment decision (prompted social justice) and (2) evaluate the suitability of the economic and decision models for informing a technical decision (unprompted social justice).

4. Data Analysis

4.1. Statistical Analysis of Student Technical Problem Solving

To address research question 1, we used a combination of descriptive and inferential statistics (nonparametric Mann-Whitney U tests) to evaluate whether reframing homework assignment language to illustrate how social justice linked to engineering investment decision aided the knowledge retention or distracted from the technical course content. Alternating students between the technical and social justice conditions ensured that all students are exposed to social justice concepts and provided the opportunity to evaluate the impact of social justice framing in a multitude of technical domains (uncertainty analysis, time value of money, net present value, estimation, economic equilibrium calculations).

The nonparametric Mann-Whitney U tests were performed due to the nonnormally distributed dependent variable. As such, the tests examined the similarity of the distribution of scores between each treatment group. For example, in Homework 3, one of the null hypotheses is that distributions of scores for the two treatment groups (those with the technical economics supply and demand question versus those with the social justice economics supply and demand question) are equal.

With each Mann-Whitney U test performed, the following assumptions were tested and accounted for: (1) continuous dependent variables, (2) one categorical independent variable, (3) independence of observations, and (4) examination of the distributions.

4.2. Inclusion of Social Justice in Technical Decision Making

When investigating research question 2, one of the researchers anonymized the student responses to the final exam and read through them holistically, coding for level on the Bloom’s Taxonomy Scale (Bloom et al. 1956). Classifying on the Bloom’s Taxonomy Scale allows us to understand the depth and quality of student responses regarding tying social justice with technical analysis. The traditional Bloom’s Taxonomy contains six categories used to rank cognitive skills ranging from lower-order skills that require less cognitive processing (i.e., knowledge, comprehension, application) to higher-order skills that require greater levels of cognitive processing (i.e., analysis, synthesis, evaluation) (Adams 2015, Narayanan and Adithan 2015). We modified the Bloom’s Taxonomy Scale to focus on students’ cognitive ability to evaluate the social justice impacts of technical decisions.

We classify the degree to which students incorporate social justice into their exam responses using a modified Bloom’s Taxonomy Scale (Bloom et al. 1956) as seen in Table 2. Bloom’s Taxonomy level zero indicates the students lacked any mention of social justice. Level 1 indicates that students were able to recall social justice factors we mentioned in class and provide a brief reference in their exam answer. Students who achieved level 4 were able to mention social justice factors not previously mentioned in class and provided explanations for how engineering technology deployment and investment decisions will impact social factors.

Table 2. Modified Bloom’s Taxonomy Scale for Evaluating Inclusion of Social Justice Factors in Technical Decision Making

Social justice coding for the student responses included the following: impact of investment decisions on racial or income groups, discussion of the external factors that may worsen equality for lack of investment, discussion regarding how cost-driven decisions influence local populations, mentioning and detailing the need to include multiple stakeholders from diverse backgrounds into the decision-making process, and other distributional and procedural justice considerations. Coding for analyzing the specific responses looked for phrases that specified an impact on people or local communities. Some phrases included in the coding are as follows: “people impacted,” “community impacted,” “local perspectives,” “social justice factors,” “vulnerable groups,” “disadvantaged groups,” and “local stakeholders.” If students mentioned specific underrepresented or vulnerable groups, then this was included as a social justice consideration.

5. Results

Here we present the results of our classroom analysis for the graduate level course that covers engineering investment decisions, uncertainty analysis, and economics concepts. We first discuss how reframing homework questions to include social justice topics impacted student learning. We conclude this section by investigating the degree to which students incorporate social justice into their technical analyses when prompted and unprompted at the conclusion of the semester.

5.1. Research Question 1: Does Framing Engineering Concepts Within a Social Justice Context Aid in the Learning of Engineering Concepts?

In the first homework assignment, all students received the social justice questions related to discrimination and diversity challenges in the energy sector. Here the students had to estimate the number of minorities working in the energy sector. Then the students were asked to reflect on, and answer, the following questions. (1) How might the level of diversity in this sector affect the technologies people are building? (2) How do these numbers compare with the racial breakdown of the U.S. population? (3) What could have caused the level of diversity in the energy sector to diverge from the national demographics?

The instructor expected the students to discuss how their estimated numbers compared with the level of minorities in the United States. However, the majority of the class (Fall 2020, 83%; Fall 2021, 65%) submitted responses that were completely devoid of social justice considerations despite being prompted to do so. Instead, students focused solely on the validity of their estimation techniques and commented on the uncertainty inherent in the data. This highlights the students’ limited ability at the start of the class to tie engineering technical decisions with social justice topics (i.e., lack of diversity in a technology sector).

When examining student responses in Homework 3 for the economics supply and demand item, there was no statistically significant difference (p = 0.86) in scores between the technical group (mean = 0.95) and social justice group (mean = 0.94), indicating that the social justice component did not impact student performance on that question (Figure 2). However, when students were asked to complete a problem on economics, students’ performance decreased when presented in a social justice context (mean = 0.80) compared with the question solely focused on the technical content (mean = 0.89; Figure 2). Furthermore, a Mann-Whitney U test indicated that the difference between the two groups on the economics problem was a significant difference (U = 458, z = −2.51, p = 0.012).

The difference in performance may be explained by the students’ comfort with the technical topics. The students received five lectures about net present value and economics supply and demand, while just two lectures about microeconomics which were mostly in a review context. Additionally, during the lecture portion of the class, the instructor led a brief discussion about challenges faced by those with disabilities (social justice lens for economics supply and demand) but not LGBTQ challenges (social justice lens for economics producer surplus). In the beginning of the course, we found that layering social justice on top of a more difficult question or concept (economics) without prior discussion in the course content decreased student learning (p < 0.05; Figure 2).

The performance on technical topics deteriorating when layering on the social justice component is absent by the end of the semester, as seen in Figure 3. One group of students were given engineering problems (on risk and uncertainty analysis) in the context of social justice performed while the second group did not receive the version of the homework problem with social justice framing. In terms of individual homework scores for 2020 and 2021, the economics and uncertainty analysis questions had a wider distribution of scores in the social justice question earlier in the semester (Homework 3) and a narrower distribution of the scores later in the semester (Homework 4) (see Online Appendix F). In the risk analysis problem for Homework 4, students in the social justice condition (mean = 0.91) score similarly to their peers in the technical condition (mean = 0.88). Similarly, the two groups (social justice condition, mean = 0.96; technical condition, mean = 0.92) demonstrate similar scores in the uncertainty analysis problem for Homework 4.

We see indications of the value of social justice in increasing student learning outcomes as the semester progresses. In Fall 2020, the final exam (i.e., Knowledge Check) at the end of the semester presented a question asking students to evaluate how stakeholders would make decisions. This question was designed to test utility theory, decision trees, and uncertainty analysis concepts. Half the class received the question framed in a social justice component (homeless youth) (Figure 4). We found that even on a timed final exam, the mean total exam score of students who had the social justice component integrated into their exam question was higher than that of those who did not, although not at a statistically significant level (p > 0.05; Figure 4).

5.2. Research Question 2

At the conclusion of a course on social justice and engineering, how well can students integrate social justice into engineering decision making when prompted and unprompted?

Over the course of the semester in both years of our study, the students’ level of critical thinking skills in regard to social justice increased. At the start of the semester, in the first homework assignment, just 17% in the 2020 class, and 35% in the 2021 class mentioned social justice considerations regarding challenges related to employment of minorities in the energy sector. In Figure 5, we see that after just one semester, 51% of all students in our sample mentioned social justice in their technical analysis when unprompted. When the students were prompted to include social justice in their engineering investment analysis, 88% of students traced out the social implications of an engineering investment strategy. Across both classes, 23% of the students showed the highest level of critical thinking skills (Bloom’s Level 4) with their answers including some amount of information not previously covered in class. We see lower levels of critical thinking about the relationship between social justice and engineering on the Bloom’s Taxonomy Scale when the students were unprompted. Specific student responses under each of the Bloom’s levels can be found in Online Appendix A.

Here, we highlight a few examples from student responses when prompted. We see that Student G has mentioned key concerns in water quality distribution in low-income and minority communities and how engineering investment decisions impact the health of the community. On the other hand, Student H discussed methods for reducing the burden of additional spending required by households.

Student G:

“I’m making an investment decision in a company where I’m updating sewage/water pipeline infrastructure. The social justice topic that impacts this investment decision is environmental justice - making sure black/brown and low-income communities are receiving the same quality of water then their whiter and wealthier counterparts…A notable example is Flint, Michigan, where the low-income Flint residents had high levels of lead in their water after the town’s source water changed, and the pipes didn’t have the proper corrosion control in place to deal with the shifted water chemistry of the source water.”

Student H:

“I am deciding to invest in a new water metering system that measures exactly how much water each household is using. In order to fund this decision, I will need to increase the household water bill by a set amount each month. This will disproportionately impact poorer families, as it is a larger percentage of their income. To distribute this metering fee, I could instead increase each household’s water bill by a percentage, so that the costs are distributed more equitably.”

Although both students scored high on the Bloom’s level (level 4, outside connections) we find that they incorporated social justice into the decision making in different ways. Student G focused on the system level implications, whereas Student H focused on the way that system level changes would impact an individual household.

5.3. Student Perceptions of Incorporating Social Justice into Their Learning and into Engineering Overall

In the Fall 2020 class, the instructor gave a pre- (beginning of course) and posttest (end of course) asking students how important social justice was (a) to include in engineering classes and (b) in terms of being helpful to their learning. Student perceptions of the value of incorporating social justice factors into the engineering curriculum increased over the course of the semester, as seen in Figure 6. The results for the overall average increase in both the importance of these topics and the value to student learning is significant (p < 0.05; Table 3). In general, students rank social justice topics as highly important and helpful to their learning and remain consistently high (Rank > 5) throughout the semester.

Table 3. Pre- and Post-Student Perception Data for Fall 2020 (Wilcoxon Sign Rank Test, n = 23)

6. Discussion

Here we present the discussion and interpretation of our results under each of our research questions. We first discussed how social justice framing of technical problems impacted student learning. We conclude with a dive into how students’ cognitive ability to evaluate the social justice implications of technology investment and deployment increased by the conclusion of the course.

6.1. Research Question 1: Does Framing Engineering Concepts Within a Social Justice Context Aid in the Learning of Engineering Concepts?

In general, when social justice framing was included in the homework problems, we found decreases in student performance at the beginning of the semester and no discrepancies in student learning of technical concepts by the end of the semester. One possibility for this decrease is the related psychological component to social justice evaluation and analysis. When the social justice context is new and unfamiliar (i.e., the first homework with social justice tradeoffs), the students have to divert their available cognitive load from technical analysis to get a handle on the social context. This is in line with the literature that suggests traditional theory-based engineering education promotes disengagement with social issues (Cech 2014, Morgan et al. 2020), thus leaving students without tools to consider the impact their work may have on societal issues.

By the final homework assignment, the students had been exposed to multiple social justice topics in class, meaning that reframing the homework problem to include social justice could have been less overwhelming, and the students may have needed to divert less cognitive load to contextualizing the problem. Additionally, multiple students expressed interest in environmental sustainability, which was present in both the technical and social justice versions of uncertainty analysis question. Changing question framing on the surface without additional instruction may promote confusion. The social justice topics may need to be explained in greater depth in class to help students understand how to evaluate social issues in the context of technical investment decisions and gain insights into connecting technical topics to broader societal injustices. Varying degrees of familiarity and comfort influence the students’ ability to think critically about technology’s impact on social injustices. Students may benefit from social justice topics being incorporated into technical topics they are comfortable with.

The difficulty in processing technical information in a social justice context was validated by student responses to a midsemester survey in which the instructor asked if there was anything the students liked or disliked about the class. Multiple students commented about the difficulty of covering so many topics (social justice and technical) in a short period of time in the class. Student A remarked that “sometimes the cutting away to activities is a little overwhelming, maybe fewer activities and longer amount of time to complete.” However, in a pre- and posttest where students were asked if they thought social justice was important to include in engineering decisions and important to their learning, students ranked these topics very highly.

In the students’ final exam, there was an overall increase in the scores of students who received social justice framing in their questions. Although this increase was not statistically significant, this could indicate that longer question prompts, which include social justice framing, do not detract from student learning when the concept has previously been covered in class or homework assignments. Overall, our results suggest that by the end of the course, students could consider social justice impacts in a technical context without detracting from their performance, even in a pressured situation such as an exam.

6.2. Research Question 2: At the Conclusion of a Course on Social Justice and Engineering, How Well Can Students Integrate Social Justice into Engineering Decision Making When Prompted and Unprompted?

During the beginning of the course, the instructor found only 17% of students in Fall 2020 and 35% in Fall 2021 discussed social justice considerations in a homework assignment where they were prompted with explicit instructions to discuss the inequities related to employment of minorities in the energy sector. The void of social justice discussions, when explicitly prompted, highlights a fundamental gap in graduate students’ initial ability to reflect on the implications of their engineering estimates in a broader social justice and equality context. This is in line with prior literature that suggests engineers believe the myth that they can be objective in their decision making (Riley 2008) and a recent survey highlighting the lack of teaching about social justice and a heavy bias among faculty toward technical instruction (Nasser and Romanowski 2016).

At the conclusion of the semester, it was found that a majority of students were able to identify at least one aspect of how engineering impacted social justice efforts when prompted (2020, 81%; 2021, 90%) and unprompted (2020, 72%; 2021, 41%). When the class was prompted, 45% in 2020 and 32% in 2021 reached the modified Bloom’s Taxonomy level 3 or 4, indicating that they were able to apply the concepts learned in class to new social justice topics and engineering decisions. Although we saw high levels of critical thinking on the instructor-guided social justice question, these higher level critical thinking indicators did not always appear when the students were unprompted.

When unprompted, less than 10% of students reached Bloom’s Taxonomy levels 3 or 4. This reinforces the notion that students need scaffolding and repetition to shift the paradigm of viewing engineering decisions through a purely technical lens. Additionally, we find that social justice considerations are more salient for students when they have to make decisions at the individual, household, or community level. We acknowledge that social justice considerations may also be more or less salient depending on the stakeholder group: those whose impacts are usually accounted for (customers, company) and those whose impacts are sometimes not accounted for (residents affected by externalities; taxpayers). Reflective assignments and critical thought discussions can promote engineering students to stop and consider how their technologies will be adopted and used in society before they are deployed, while helping them develop skills in problem solving, complexity, and contextual understanding of large challenges (Hadgraft and Kolmos 2020).

Instructor-led classroom discussions and homework exercises that link technical engineering concepts with social justice help students increase their ability to recall facts and suggest avenues for integrating these concepts into decision making. When the instructor mentions or reminds students that there are social justice implications related to the decision, students are able to draw connections between social and technical aspects of a decision problem. Thus, if instructors want to explicitly emphasize social justice in their courses, then students should be prompted.

One takeaway for instructors integrating social justice into writing assignments is a need to structure these assignments to reach both social justice and analytical course goals (Getchell and Pachamanova 2022), which can be a challenge. Another challenge, as highlighted in previous literature, is that social justice considerations are very context specific, and there are many different definitions of justice (Baillie and Levine 2013). Given that a large portion of students in our sample were international students, some of the U.S. examples may not have resonated due to differences in life experiences.

7. Limitations and Considerations for Other Classes

Here we discuss some considerations for our study and future classes.

7.1. Limitations

There are a few limitations of our analysis. First, some of the students (Fall 2020) were taking this course in a completely virtual environment during the COVID-19 pandemic, which may have disrupted overall learning. There is no evidence that students who took the course completely online fared worse than students in previous years (Ferdous et al. 2021). The social justice conversations are difficult to gauge in a virtual environment due to many students having their video feeds turned off. In person, many of the students were wearing masks, making it difficult for the instructor to gauge facial expressions, body language and students’ level of engagement.

Second, the sample size of the class is small at 28 students in Fall 2020 and 49 students in Fall 2021. To overcome this challenge, each student completed an individual technical and social condition, thus acting as both the treatment and control group (on different individual questions) for a single homework assignment.

Given that the sample size of the graduate class is small, and only completed at one institution, there would also need to be expansion to consider other learner types. Future work should expand this study to consider undergraduate versus graduate learning, wider diversity of topic areas (applications, analytic methods), and wider diversity of disciplines (civil engineering, electrical engineering, business).

Last, not all students completed all assignments. This led to some variation in the reported sample size. In the future, it would be good to ensure all students completed all assignments and control for students dropping the class.

7.2. Considerations for Other Classes

When designing homework assignments to include social justice framing, there are a few things instructors will need to consider. First, professors will need to give more than token attention to equity issues, which has historically been the case (Marshall and Ward 2004). Second, each of the reframing of technical decisions will need to be plausible and integrated into the decision. This class was an investment-planning and decision-making class, and students’ decision-making tasks were based on actual companies that work toward addressing some social justice issue. Some classes that can benefit from this reframing include optimization classes (e.g., plant siting and product decisions), product design classes (e.g., asking students to consider how the products sourced in their designs will impact vulnerable communities), and simulation or analytics classes that consider labor and workforce decisions (e.g., using prison or overseas labor, layoffs). The locational context (e.g., in developed or developing countries) will also make a difference for the framing the decision problem and decision maker preferences (Van-Hein Sackey et al. 2023).

Regardless of the class, a simple way to include social justice framing into homework questions in investment and planning classes is to ask students which population is made worse off by their design or investment strategy. This will train students to think about the implications for vulnerable groups.

One consideration for instructors is how to diversify the social justice topics, such that undue emotional burden is not placed solely on marginalized students. In an informal survey, the instructor asked for feedback from the class on social justice topics covered. One student stated, “I like that we talk about these issues, but it’d be nice if it wasn’t about race all the time. I know these issues real well, and I feel like people from outside [the U.S.] look to me to explain what’s going on all the time.” Following this feedback, the instructor integrated more examples of social justice topics that affected immigrants. There needs to be a diverse number of vulnerable groups incorporated into the homework questions (e.g., racial minorities, immigrants, disabilities, age).

Another consideration is that instructors will need to educate themselves on social justice topics and may not feel comfortable or knowledgeable enough to lead these student discussions around social justice (Carroll et al. 2023). We recognize that some instructors who may feel that they don’t understand the minority view or be hesitant to address any issue related to race, especially, for fear of offending someone by sounding racist unintentionally. Having an opportunity to discuss these issues openly is beneficial to all, especially if there is a “safe” environment introduction, so everyone is free to speak as a learning opportunity for sensitivity. To increase instructor comfort and efficacy in leading discussions, there are a number of resources that would help an instructor get started with integrating social justice concepts into the course (Enns and Sinacore 2005; Riley 2008, 2015; Lucena 2013) and train in theories and concepts of intercultural communication and instructional communication. For example, this paper (Chen and Lawless 2020) discusses (re)thinking “uncomfortable” and “difficult” classroom conversations about power dynamics, privilege, and intersecting cultural identities (e.g., sexism, racism). Professors will also need to take care to increase the sense of belonging for vulnerable students (Sax et al. 2018), teach to the whole student and incorporate multiple backgrounds into the classroom (Aronson and Laughter 2016), and minimize the discomfort experienced by students from historically marginalized groups during tough topic discussions. To minimize the burden on students from historically marginalized groups, the types of vulnerable groups should be varied. For example, each of the homework questions could cover issues related to a different vulnerable group (e.g., racial minorities, immigrant populations).

Given that the participants were graduate students in an engineering economy course in a civil engineering program, the course may lend itself more easily to human-physical systems through the lens of applied economics. It is possible that this course is more amenable to social justice-based pedagogy than other courses in which human judgement and policy may play a less direct role. To adapt this course for undergraduate students, instructors should provide more in-class instruction on how to integrate these topics into decision-making paradigms.

8. Conclusions

To ensure engineered systems do not create, perpetuate, or exacerbate social inequities, we must train engineers to consider the social justice implications of their work. This requires training them to recognize and evaluate how their designs and investment decisions impact the broader society. Our study investigated the degree to which incorporating social justice framing into homework, exams, and classroom discussions (1) aided in student learning and (2) promoted critical thinking skills regarding how engineering technologies tie to social justice. Statistical assessments suggest that incorporating social justice concepts in assigned work may detract from student learning of technical concepts absent adequate discussion of social justice in class. However, as the course progressed and students became more familiar with social justice considerations, their learning of technical concepts became comparable to that of students who did not have the social justice components in their assignment. A potential cause of the decrease in the earlier homework is the required paradigm shift combined with a difficult technical topic.

Many of the students struggled with the economic portion of the class, and most of the students had taken one or fewer classes that incorporated social justice concepts. All the students in our sample were MS or PhD students, which highlights the lack of formal training in social justice decision making within an engineering curriculum. Additionally, social justice may mean different things to different people, making it difficult for some students to see how they play a role in promoting social justice. For example, some may view social justice as having political implications and requiring governmental interventions, while others may hear social justice and focus on equality of opportunity (not equality of result) (Baillie and Levine 2013).

Despite students having different views of what social justice means, we found students’ ability to address social justice considerations in their technical responses and the students’ critical thinking skills, in relation to analyzing how social justice is tied to engineering, increased substantially. At the beginning of the semester, few students (29%) integrated social justice considerations into their technical analysis. Yet at the end of the semester, the majority of students across both semesters (51%) included social justice in their exam responses even when unprompted. Although critical thinking increased, many students still needed some prompting from the instructor to move beyond simply recalling facts discussed in class. Thus, a singular graduate-level class can bring students’ attention to potential inequities that will be exacerbated by engineering infrastructure investments and technologies but may not be enough to push the students to consider social justice in all their technical decision making.

Although we argue that social justice and engineering are linked, instructors will need to take care to frame social justice issues in ways that students can identify clear links between social justice issues and the engineering profession. A key challenge for instructors is how to integrate social justice concepts into technical courses in a way that promotes student learning but also does not place undue emotional burden on the minority and marginalized students (Calabrese Barton and Tan 2020). Thus, courses which aim to teach students how to evaluate the social justice implications of technology investment and deployment decisions should take care to adhere to culturally responsible and inclusive teaching practices.

Social justice has many facets. There are stakeholder and community characteristics to be accounted for in anal analysis involving stakeholders (e.g., race, ethnicity, gender, disability status). In addition, there are the application domains for which social justice is salient, but which may not receive appropriate attention in the modeling process or technical analysis (e.g., social determinants of health, impacts on education outcomes, environmental and community impacts, community perceptions; Cranmer et al. 2022) (Lucena 2015). To create socially responsible engineers, we need multiple courses, covering a wide range of social justice topics (e.g., racial discrimination, immigration policies, disabilities), to be incorporated into engineering curriculum and train students to think critically about how their creations promote or erode social equity.