An Operations Research–Based Teaching Unit for Grade 10: The ROAR Experience, Part I

1. Introduction

Operations research (OR) is a branch of applied mathematics usually taught at university level. In recent years, various initiatives have been developed to introduce OR to younger students: promotion activities, national and international projects, competitions, training courses for teachers, workshops for students, and teaching units or lectures tested in classrooms (Raffaele and Gobbi 2021). Taking a cue from these, we designed Ricerca Operativa Applicazioni Reali (ROAR; in English, Real Applications of Operations Research), a three-year project for higher secondary schools. The main aim of ROAR is to improve students’ interest, motivation, and skills related to Science, Technology, Engineering, and Mathematics (STEM) disciplines by integrating mathematics and computer science through OR. Indeed, ROAR is composed of three teaching units, all containing examples and problems closely connected with students’ everyday life or with the industrial reality, balancing mathematical modeling and algorithmics. The first teaching unit, addressed to grade 10, focuses on introducing OR, in particular linear, integer, and mixed integer linear programming, in terms of mathematical models and techniques. It also involves the use of an automatic solver and other digital technologies. The second teaching unit, addressed to grade 11, concerns common graph theory problems and algorithms. Finally, the last teaching unit, addressed to grade 12, is about the implementation of OR methods and algorithms in a programming language students are already familiar with and also about the learning of an algebraic modeling programming language. ROAR is designed to be attended, throughout all the three years, by the same students.

In early 2021, ROAR has been implemented into a project-work that fit a Percorso per le Competenze Trasversali e l’Orientamento (PCTO; in English, Path for Transversal Skills and Orientation), which is an Italian innovative teaching method that aims to strengthen students’ knowledge acquired at school through practical experience (Ministero dell’Istruzione, Università e della Ricerca 2018). In Italian higher secondary schools, a PCTO is a mandatory experience that can be realized in collaboration with local companies or organizations. The PCTO of ROAR was activated through a formal agreement between the Department of Mechanical and Industrial Engineering of University of Brescia and the scientific high-school IIS Antonietti in Iseo (Brescia, Italy). The experimentation started with the first teaching unit. From March to May 2021, six lectures were held in a grade 10 class composed of 25 students. Because of the COVID-19 pandemic, the lectures were mostly conducted in distance mode through the Microsoft Teams platform. In each lecture, we alternated frontal teaching with group works in collaborative learning. Also, during the first lecture, we divided the students into five groups, and we gave each group a project to develop and present in the final lecture in the form of an authentic problem, that is, a realistic problem coming from a particular field and recognized by workers in that field as a possible problem they might face in their daily work (Niss 1992).

This paper, focused on the first teaching unit of ROAR, is structured as follows. In Section 2, we recall some pedagogical research about teaching units, collaborative learning, and authentic problems. Then, we illustrate the state of the art of OR-based teaching units addressed to grades 10–12. In Section 3, we explain the design of the first teaching unit of ROAR in terms of objectives, students’ prerequisites, and instructors’ roles. Section 4 provides a description of the implementation in a grade 10 class at IIS Antonietti. First, we focus on the teaching methods adopted and the digital technologies used. Then, we explain the organization of the lectures held, and the OR topics and methods presented. In Section 5, we report the main results and feedback received from both students and teachers and some considerations on distance learning. In Section 6, we draw our conclusions on this first part of the ROAR experimentation. Finally, we include all the problems proposed during the lectures in Online Appendix A, whereas Online Appendix B contains the texts of the authentic problems assigned to students as final project.

2. Background

In this section, first we describe the main results from pedagogical research about teaching units, collaborative learning, and authentic problems. Then, we review the state of the art about OR-based teaching units.

2.1. Pedagogical Research

Teaching units constitute the main tool for the practical structuring of the training offer to some learners (Roberi 2012). Starting from them and their training needs, teaching units make explicit the training objectives, the activities, the organizational methods, the times, and the methods needed to promote the development of competences, as well as the possible procedures to evaluate the competences acquired (Trinchero 2012). Petracca (2015) says that teaching units make it possible to work based on competences, that is, to make effective and active use of the knowledge gradually assimilated by integrating it, when appropriate, with knowledge from other fields (i.e., transdisciplinary) or with technical competences (i.e., use of technological technologies). We notice that a teaching unit is not yet an elaborated plan for a series of lectures, although each one contains essential points of such a plan. There is no rigid algorithm to follow when structuring a teaching unit. To successfully design one, it is necessary to ensure that the school environment is collaborative/cooperative, that an active students’ participation is fostered by laboratory and interactive activities, and that self-evaluation of learned competences is encouraged. Wittmann (2021, p. 29) notes that “Appropriate teaching units provide opportunities for doing mathematics, for studying one’s own learning processes and those of students, for evaluating different forms of social organization, and for planning, performing and analysing practical teaching. Therefore teaching units are a unique means for penetrating all components of teacher training and relating them to one another.” According to Wittmann (2021), a teaching unit is an idea or a suggestion for a teaching approach that intentionally leaves various options of realization open. Some elements are indispensable. In the case of a mathematics course, besides objectives and materials, a teaching unit must have mathematical problems arising from the context of the unit and the mathematical (sometimes psychological) background of the unit in terms of prerequisites.

Collaborative learning is an important element of active-learning theory and practice. Research suggests that students learn best when they are actively involved in the process (Davis 2009). Indeed, working in small groups has the power to foster cognitive development and thus to empower learning (Garvin 2001). The group is a place to belong to, and as such it provides support and motivation; it promotes cooperation and the activation of latent cognitive potentials through the sharing of different competences and working/thinking styles; and it supports coconstruction of knowledge by socialization and reciprocal negotiation of meanings (Getzel et al. 1987, Kurland and Salmon 1992). Through the socio-cognitive conflict that emerges in groups, learners have the opportunity and the need to explain, confute, and defend their beliefs; new aspects and prospects can be seen; and personal experiences and point of views can be downsized and put in perspective (Feichtner and Davis 1984, Beebe et al. 1986). A collaborative learning environment, as opposed to a passive learning environment helps students learn more actively and effectively (Murphy et al. 2005). This approach is especially fruitful to develop competences in problem solving, which are usually difficult to acquire by means of explanations and examples only (Hmelo-Silver 2004). On the contrary, the learning process can be activated when facing a problem for which one’s own repertoire of known procedures is insufficient: With exploration and discovery activities in small working groups, learners can learn together what they need to know to solve the problem (Kolb et al. 2014).

Small working groups can be adopted to solve authentic problems. Here we refer to authentic problems in the sense of Niss (1992). An authentic problem comes from a particular field and is recognized by people working in this field as a possible problem they might face in their daily work. According to Kaiser and Schwarz (2010), authentic problems are very advantageous to make students experience the power of mathematical modeling for the understanding and the resolution of real questions meaningful to many people. Moreover, by tackling these problems, students can be convinced of the usefulness of mathematics and mathematical modeling for their real life.

2.2. State of the Art

In the literature, there are not many papers about OR-based educational initiatives addressed to higher secondary school students, as reported in the review by Raffaele and Gobbi (2021). In terms of teaching units, Raffaele and Gobbi (2021) mention the projects developed by Schuster (2004), addressed to grades 9–12 in some German high schools. Through those experimentations, the author wanted to investigate the role of combinatorial optimization in teaching mathematics and computer science at high school level. Another early effort to bring OR to preuniversity mathematics classrooms was done by Edwards and Chelst (2004), through the development and the experimentation of the HSOR (High School Operations Research) project in the United States. Schettino and Bonetto (2013) presented a pathway that included statistics and OR, addressed to two grade 10 classes in an Italian technical school. They also developed an extracurricular project to introduce OR and strengthen English listening skills. Lonati et al. (2017) proposed a teaching unit, addressed to grades 10–12, to present greedy strategies as a natural way to cope with optimization problems. Students were first organized in pairs and then in groups to face the proposed activities about cash management and scheduling problems. They were also provided with an ad hoc software tool to support their tests and strategies. Colaço et al. (2018) showed how to adapt three optimization problems to create mathematical tasks of increasing difficulty addressed to grades 1–12. Fornasiero and Malucelli (2020) reported on a small experience carried out in an Italian high school to introduce games, puzzles, and challenges based on optimization problems to stimulate students’ intuition and creativity. Based on these activities, Fornasiero et al. (2021) described an outdoor orienteering competition. To consolidate problem-solving skills, they adopted puzzle-based learning as the main teaching methodology.

3. Design of the First Teaching Unit of ROAR

In this section, we describe the main objectives pursued, the prerequisites needed, and the roles to be played by the instructors in the first teaching unit of ROAR, addressed to grade 10.

3.1. Objectives

The first teaching unit of ROAR has the following seven objectives.

The first objective is to introduce OR, by discussing its origin and development and by showing its practical applications in several fields. In this way, we want to make students aware of the discipline and its relevance and try to increase their motivation toward STEM discipline.

The second objective is related to presenting mathematical modeling: basic notions and definitions are introduced, such as the concepts of variables, constraints, objective function, model, feasible region, and optimization problem. Every problem presented is inspired either by a situation closely connected to students’ everyday life or by the industrial reality. Students have to understand the key features of the problem by reading a textual description.

In the third objective, the teaching unit aims to present linear, integer, and mixed integer linear programming as possible paradigms to represent problems. After the comprehension of the problem and the relevant data, students are required to translate these into a mathematical language to apply the right paradigm and formulate a model.

The fourth objective is related to solving optimization problems by means of some OR methods. During this first year of ROAR, we discuss brute-force methods and greedy algorithms. We also introduce the graphical method to solve two-variable linear programming problems: We present both the method of comparing all the values of the objective function at the vertices of the feasible region and the method of using the family of straight lines parallel to the line of the objective function to compute an optimal solution.

The fifth objective involves developing students’ information technology (IT) skills. In particular, we want to teach them to use a solver, to tackle problems with more than two variables, and to use software that helps them during the problem-solving process, such as graphing calculators and spreadsheets.

The last two objectives are about the strengthening of some soft skills. Through homework assignments, done individually, and exercises during the lectures tackled by groups of students, we want students to acquire problem-solving and teamwork skills in a collaborative environment (sixth objective). Moreover, each students’ group is assigned with an authentic problem as a project. All groups are required to report the results of their work through a live presentation, hence enhancing students’ public speaking skills (seventh objective).

3.2. Students’ Prerequisites

Before attending the first teaching unit of ROAR, students must know linear equations, linear inequalities, and some notions of analytic geometry, such as lines and families of straight lines, or plotting a function in a Cartesian coordinate system. Regarding digital technologies, they should be at least familiar with Microsoft Excel and GeoGebra.

3.3. Instructors’ Roles

We distinguish two different roles. We use the term experimenter to indicate an OR expert who presents the topics of the teaching unit. Instead, we denote by observer someone who attends the lectures and takes notes of the activities, what and how the experimenter explains, what students do, and how they react. To implement the teaching unit, there must be at least one experimenter, who is in charge of leading the lectures by introducing new topics, guiding, and involving students in the activities. On the contrary, the role of observer is not strictly necessary, but it becomes fundamental in case of collecting data for research purposes. The experimenter role can be covered either by a classroom teacher of the grade 10 class or by some experts external to the class (or also to the institute). The role of observer can be covered by any person, including experimenters in some parts of the lectures (e.g., during group works). Neither the observer nor the classroom teacher needs to be an OR expert. Nevertheless, this would allow them to provide ongoing support to the students and the experimenters and to autonomously elaborate the data collected.

4. Implementation of the First Teaching Unit of ROAR

The first teaching unit of ROAR, composed of six lectures, was implemented from March to May 2021. Because of the COVID-19 pandemic, we had to hold some lectures through distance learning by means of the Microsoft Teams platform.

4.1. Grade 10 Class

The Grade 10 class involved in the first year of ROAR was composed of 16 males and 9 females. According to their mathematics teacher, M. Picchi, their mathematical skills were just above the average level, both in comparison with other grade 10 classes in the same school and with the national level. In general, the students’ interest toward mathematics and engagement greatly decreased during the distance-learning mode imposed after the COVID-19 spread. Students moved from collaborating with peers in the classroom to relying on Microsoft Teams. Before attending the first teaching unit of ROAR, they had already participated in problem-solving activities. Indeed, during ordinary lectures in previous years, they had solved problems found in mathematics and physics books related to real-life situations. Also, some of the students in the class had previously participated in mathematical challenges, such as the Mathematical Olympiad. Regarding digital technologies, the students were already familiar with GeoGebra and Microsoft Excel (the former is commonly used in mathematics and physics lectures by teachers and students, whereas the latter is presented in Information and Communications Technology lectures during the first year in high school). The students did not know Solver, a Microsoft Excel add-in to solve optimization problems (see Section 4.5), or, in general, OR.

4.2. Instructors

In our implementation of the first teaching unit of ROAR, we usually held the lectures during the hours of the mathematics teacher, M. Picchi, who did not cover the experimenter role. There were two experimenters (A. Gobbi and A. Raffaele, both researchers in OR) and two observers (G. Colajanni and E. Taranto, researchers in OR and in mathematics education, respectively). They were all external to the class and had never met them before Lecture 1.

During work-group activities, both the experimenters and the mathematics teacher assumed the role of observers. Thus, the total number of observers was five, and the students were divided into five groups (see Section 4.4.2).

4.3. Positioning in the Mathematics Program

The first teaching unit of ROAR was inserted during the regular school schedule of the grade 10 class. Before its implementation, in terms of previous mathematical knowledge, all students were already able to solve simple equations and inequalities, as well as problems involving systems of linear equations in two variables (graphically and algebraically) and problems involving quadratic inequalities in one variable. Moreover, they were able to plot graphs and analyze absolute-value linear and quadratic functions. They recognized the characteristics of graphs of linear relations, including intercepts, slope, domain, and range. They knew how to determine the equation of a linear relation given a graph, a point and the slope, or two points. Also, given a line, they could compute the equation of a parallel or perpendicular line, they could find the intersection points of two lines, and they were already familiar with families of straight lines.

4.4. Teaching Methods

In this part, we provide details about the main teaching methods adopted during the teaching unit.

4.4.1. Frontal Teaching.

Each topic was presented using frontal teaching and by starting with an example problem. To solve it, students were enabled to use knowledge or skills that they already possessed or that had been transferred to them during the explanations before the example problem. In general, during this part, students were passively listening but could freely ask questions or make hypotheses on how to proceed. Every phase of frontal teaching was always followed by a collaborative phase, in which students were more actively involved.

4.4.2. Collaborative Learning.

We implemented collaborative learning by dividing the students into five groups at the end of Lecture 1. The groups were fixed for all the lectures composing the teaching unit and for the final project as well. Except for Lecture 1, during each lecture, students had to work in groups by discussing, confronting each other, and deciding what to do to tackle the problems we assigned them. During work group activities, each group was observed by either an experimenter, an observer, or the classroom teacher. It was established that these instructors should have only observed and not helped the students achieve their goals.

4.4.3. Homework Assignments.

At the end of a lecture, we assigned some exercises on the topics just covered to be done for the following lecture. Students could do the homework individually or in groups. Each one of them had to send the classroom teacher his or her own resolution. At the beginning of the following lecture, the experimenters checked the work with the help of some students, who were called on to explain the method they had followed. Thus, the homework assignments helped the students strengthen their knowledge on the OR topics and methods seen during the lectures, as well as improve their mathematical and technical skills.

4.4.4. Authentic Problems.

Besides the work group during the lectures, students had to collaborate to carry out a final project that they had to present in the last lecture. In particular, we decided to give each group an authentic problem (as defined in Section 2.1). Students were encouraged to think critically to apply their knowledge and skills to real-world challenges, by taking cues from what was taught during the lectures. Moreover, they developed both technical and transversal skills, such as public speaking.

All the authentic problems had the setting in common; that is, they all asked to optimize a different aspect of the SuperAmazingMarket company, an imaginary chain of supermarkets. They were introduced as follows.

“Assume you are all members of the operations research area of SuperAmazingMarket, a supermarket chain in the province of Brescia. In particular, you are divided into five groups of five persons each. Every group will focus on solving a different problem related to the company activities. These problems are currently not being managed optimally, still being solved manually. In order to reach your goals, you are allowed to use the Internet and all the digital technologies you prefer. The results will have to be presented to the executive directors in the form of a presentation.

Every presentation will have to include:

• an analysis of the problem description;

• a manual resolution to compute a feasible solution;

• the model formulation of the problem by applying Linear, Integer, or Mixed Integer Linear Programming;

• an implementation and resolution of the problem performed with Solver;

• the textual description of an analogous problem that arises in a completely different context from the SuperAmazingMarket company.”

All the texts of the assigned authentic problems are reported in Online Appendix B.

4.5. Digital Technologies

We describe the main digital technologies we used to carry out the teaching unit.

4.5.1. GeoGebra.

GeoGebra is open-source mathematics software that acts as a graphing calculator, allowing the representation on the Cartesian plane of curves and geometric shapes, as well as studying them using specific commands. It is designed for all levels of education and supports STEM education and innovations in teaching and learning worldwide (GeoGebra 2021). For instance, an earlier use of GeoGebra to teach OR in higher secondary schools was described by Edwards and Chelst (2015).

Since the beginning of the teaching unit, we mainly used GeoGebra to represent constraints of two-variable problems to apply the graphical method. During the group activities, the students were also able to use this technology.

4.5.2. Solver.

Among its several features, the Microsoft Excel software (Microsoft Corporation 2019) offers an add-in program called Solver to solve mathematical optimization models. The add-in program comes with a popup window, which allows the user to select all the model data, by choosing among the cells previously filled in an Excel sheet. Indeed, given an optimization problem, the parameters and variables must first be inserted in groups of cells and then bound together through formulas to define the constraints and the objective function. After that, the graphical user interface of Solver allows to declare what are the cells of interest to establish whether to maximize, minimize, or fix the value of the objective-function cell, to enter the constraints, and to set the variables types needed. Then, to find an optimal solution, Solver asks to select a solving method among LP Simplex (the only one we used, for linear programming models), Generalized Reduced Gradient Nonlinear (for smooth-nonlinear problems), and Evolutionary (for nonsmooth problems).

Given its ease of use, Solver is designed to also be used for didactical purposes (Fylstra et al. 1998). In particular, Edwards and Chelst (2014) demonstrated the use of Solver to solve linear programming problems with two and four variables.

We introduced Solver in Lecture 3, only after the students had completely understood the bases of mathematical modeling. The add-in program was mainly exploited to tackle problems having more than two variables. Because of the limited amount of time we had to carry out the experimentation, we did not utilize Solver to perform sensitivity analysis.

4.5.3. Mentimeter.

Almost every lecture was characterized by the adoption of interactive polls developed through Mentimeter (2021), to actively involve students in participating. Through live polls such as open-ended questions (Figure 1(a)), multiple choice (Figure 1(b)), ranking questions (Figure 1(c)), word clouds (Figure 1(d)), 2-by-2 grids (Figure 1(e)), and scales questions (Figure 1(f)), we investigated students’ understanding and feedback related to the contents presented, as well as their feelings and impressions in that moment.

The interactive polls had to be prepared in advance. At the chosen moment, one of the experimenters communicated to the students a Mentimeter code corresponding to the poll. Then, each student had to go to www.menti.com and insert the given token. A poll could be presented in two modes: Audience Pace and Presenter Pace. In the former, each student could directly answer all questions in the poll independently. In the latter, it was the experimenter that decided the pace of the poll, showing the students one question at a time. We decided to adopt this last modality to control that each student answered each question.

4.6. Questionnaires

One week before meeting the grade 10 class for the first time, we asked the students to fill in an anonymous questionnaire about their feelings toward mathematics and their expectations on the teaching unit. Another similar questionnaire was given at the end of the lectures, asking the same questions together with some other feedback. Indeed, in this last questionnaire, students could express their opinion about several aspects, such as the interest and understanding about the topics taught, the pace of the teaching unit, the temporal organization of the lectures, and the impact of distance learning on the conduct of the lectures. Through the responses, the experimenters, together with the observers and the classroom teacher, could realize, for instance, what difficulties the students had encountered, whether some topics should have been deepened more, or if time was well managed. Both questionnaires were created with Google Forms, sharing the related links with the students. The main results that arose are reported in Section 5.2.

4.7. Assessment

At the end of the teaching unit, to become aware of what students had understood and learned, we did not choose to perform a summative individual assessment, but we focused again on collaborative learning and development of soft skills, such as teamwork. Indeed, the students’ acquisition of competencies was assessed by a final presentation on the authentic problems assigned to the five groups, thus resulting in a formative and summative assessment. We evaluated each group’s presentation in terms of how the students participated in the development of the project and collaborated with each other, how they analyzed the available information on the problem, how they understood what to do to tackle it, and finally, how they presented the results of their work. In particular, each experimenter and each observer graded each student by expressing a score from 1 (i.e., very poor) to 10 (i.e., excellent) for each one of the following aspects: “Work group” (in terms of collaboration and harmony), “Analysis and deepening of the contents” (observation and reasoning), “Exposition” (clarity, completeness, and lexicon), and “Knowledge and understanding” (abstraction and generalization). The classroom teacher only graded the “Work group” aspect. The final scores, obtained by computing the average of the individual ones, are reported in Section 5.1.

4.8. Organization of the Lectures

Hereafter, follow the details of the six lectures held in the grade 10 class at IIS Antonietti, from March 15 to May 13, 2021. For each lecture, we indicate the date, an overall indicative duration, the structure (i.e., all the activities performed), the homework assigned, the teaching methods adopted, and the digital technologies used. Also, we indicate whether the lecture was held in distance learning or not.

Each lecture had the same macro-structure, as shown in Figure 2.

The texts of all problems are reported in Online Appendix A.

4.8.1. Lecture 1.

The main aim of the first lecture was to present the whole teaching unit. We introduced the topics and the teaching methods we adopted, which actively involved the students. Indeed, immediately from the beginning of the lecture, we made students participate by making them answer a live interactive poll, developed using Mentimeter. In particular, we asked them the following questions: “What do you think Operations Research is?” (Figure 1(a)) and “What does ‘optimization’ mean to you?” The main features of Lecture 1 are summarized in Table 1. By using the Netflix problem, a simple application of the Knapsack problem inspired by a situation very close to students’ reality, we first introduced brute-force methods and the binary-tree method. Then, we presented the mathematical model formulation of the problem, by also talking about linear, integer, and mixed integer linear programming. We distinguished the kinds of variables a problem could require to obtain its mathematical formulation by analyzing the texts of some problems to be modeled. At the end of the lecture, we divided the students into five groups of five students each, and we discussed the final project. Each group was assigned an authentic problem to tackle and present in the last lecture. Finally, we assigned some homework to do for Lecture 2.

Table 1. Details of Lecture 1

4.8.2. Lecture 2.

At the beginning of the lecture, we used Mentimeter to review the meaning of “variables,” “constraints,” “objective function,” and “linear expressions” (Figure 1(b)). Then, we focused on strengthening the students’ modeling skills by analyzing textual descriptions of several problems and formulating the corresponding mathematical models. In the second part of the lecture, we presented the graphical method by solving, together with the students, a two-variable linear programming problem. Both we and the students relied on GeoGebra. In particular, we explained how to compute an optimal solution by comparing all the values of the objective function at the vertices of the feasible region of the model. Then, we divided the students’ groups into the breakout rooms and assigned them their first group work. After about 30 minutes, we discussed the approach used by the groups. Then, we assigned homework for the successive lecture. Finally, through a live poll, we asked them to rank the tasks needed to solve a two-variable problem (Figure 1(c)). All details are shown in Table 2.

Table 2. Details of Lecture 2

4.8.3. Lecture 3.

At the beginning of the lecture, through Mentimeter, we asked the students the following questions, to recap the previous topics: “What kind of Linear Programming problems can we solve by using the graphical method?,” “What is the ‘feasible region’?,” “What is a ‘vertex’?,” and “What is the first task to do to solve a Linear Programming problem?” We also investigated whether the students had already used Microsoft Excel (Figure 1(d)). Then, in the first part of the lecture, described in Table 3, we finished discussing the graphical method for two-variable linear programming problems. In particular, we explained how to exploit a family of straight lines parallel to the objective function line to compute an optimal solution. Then, we split the students into the breakout rooms to carry out a group work on the graphical method. In the second part of the lecture, we underlined the applicability of the graphical method to two-variable problems only. Thus, we introduced Solver by solving together with the students another example problem with more than two variables. Then, we assigned the students another group work and we gave them some homework to do for the successive lecture. Finally, through Mentimeter, we asked them to express how much simple and relevant they had found a given list of tasks needed to solve a problem (Figure 1(e)).

Table 3. Details of Lecture 3

4.8.4. Lecture 4.

In this lecture, summarized in Table 4, we did not introduce any new OR methods. We wanted the students to focus on improving their modeling and solving skills by using Solver. Through Mentimeter, we recapped some notions about Microsoft Excel and Solver. In particular, we asked the following questions: “What is the Excel function that allows to sum and multiply the elements of two matrices?,” “In the Solver popup window, how can you set that the variables can assume only integer values?,” and “Given a Linear Programming problem with three variables, what methods can be used to solve it?” Then, we split the students into the breakout rooms, and we assigned them problems of increasing difficulty. Indeed, some of these involved two-index variables or using two different kinds of variables that need to be bound together. Both these two features were not previously discussed in any example; thus, the students were free to apply their problem-solving skills. The homework assigned for the successive lecture, the Mediterranean diet problem, contained all these features. In the last part of the lecture, through Mentimeter, we asked the students to grade the difficulty they had perceived when tackling the different tasks of the two problems (Figure 1(f)). Finally, every group had to provide an update about the authentic problem they were assigned to.

Table 4. Details of Lecture 4

4.8.5. Lecture 5.

In this lecture, described in Table 5, we corrected the Mediterranean diet problem, one of the most complex problems we assigned as homework. Then, we helped the groups understand how to overcome some difficulties encountered while tackling their authentic problems before their final presentation.

Table 5. Details of Lecture 5

4.8.6. Lecture 6.

In the last lecture of the teaching unit, detailed in Table 6, every group had about 20 minutes to present how they tackled the authentic problem they were assigned to. They used Microsoft PowerPoint and Prezi to produce and show their presentations. After a phase of questions and answers, we asked the students to fill in the final questionnaire.

Table 6. Details of Lecture 6

All groups were able to describe and solve the assigned problems, as well as discuss comments and questions coming from the audience.

4.9. Linking Implementation to Objectives

In Table 7, we summarize how the objectives of the teaching unit listed in Section 3.1 were achieved and in which lectures, through the implementation just described in this section.

Table 7. Linking the Objectives of the Teaching Unit to the Implementation Described in Section 4

4.10. Interdisciplinary Connections

The theme and setting of the authentic problems assigned as final project of the teaching unit allowed other teachers of the grade 10 class to make several interdisciplinary connections.

Because the authentic problems had a supermarket chain as a setting, the art teacher deepened with the students the theory of colors for the arrangement of items on the shelves of a store and the history of architecture of markets and supermarkets. In addition, the students actively participated in two speeches given by some economics and law teachers of the same high school, but external to the class. Some topics, such as the laws of supply and demand and the models of organization of stocks in a warehouse, were presented to help students better understand the setting of the authentic problems. Moreover, the English teacher suggested delivering the last part of the final presentation (i.e., the one regarding the textual description of an analogous problem in a different context) in English. Furthermore, in the week following the conclusion of the teaching unit, the Italian teacher asked the students to write a report that recounted their experience and collected opinions about it. Some of these are reported in Section 5.2.2.

5. Results and Feedback

In this section, first we report the main results obtained by students in terms of the formative and summative assessment described in Section 4.7. Then, we analyze the main feedback received from both the students (through the three questionnaires we gave them) and the classroom teachers involved in the ROAR project. Moreover, we discuss advantages and disadvantages of distance learning because of the COVID-19 pandemic restrictions for the grade 10 class and for the experimenters and observers.

5.1. Results of the Group Work

The grade 10 class had a very positive reaction to the first teaching unit of ROAR. On average, the students were able to acquire good problem-solving and modeling skills, reaching a discrete level of autonomy in tackling the problems proposed in the last lectures and the authentic problems. As stated in Section 4.7, we evaluated the students’ acquisition of competences through a final presentation by expressing scores on four different aspects related to the authentic problems assigned. In Figure 3, we report a boxplot diagram for each assessed aspect.

5.2. Feedback from Students

Twenty-four students filled in both questionnaires, whereas one student filled in only the first one. Because we needed to compare the answers of each student in the first and in the second questionnaires, we discarded this incomplete response.

5.2.1. Quantitative.

First, we looked at the responses obtained to two questions proposed both in the first and second questionnaires. In these two questions, we used a Likert scale that ranged from 1 (i.e., absolutely disagree) to 4 (i.e., absolutely agree). In particular, Figure 4(a) shows the comparison of the responses to the statement “In real life, math is useless.” At the beginning of the teaching unit, 58.4% of the students disagreed with the statement (i.e., 1 and 2 in the Likert scale used). In the final questionnaire, this percentage rose to 83.3%. Although the percentage of those who absolutely agree remained constant, the percentage of those who partially agreed decreased by 25%.

Figure 4(b) is about the statement “In my life, I will never use most of the topics studied in mathematics.” The percentage of the students who absolutely disagreed or disagreed with the statement (i.e., 1 and 2 in the Likert scale used) increased from 54.1% to 58.4%. In particular, the percentage of those who agreed increased by 9.4%. From this analysis, we can understand how this first teaching unit of ROAR helped students strengthen their interest and motivation toward mathematics.

Then, we analyzed some responses coming only from the second questionnaire submitted to the students. We asked them to sort the topics covered during the teaching unit according to their preference, by placing their preferred topic in first place and their least-liked topic in the last place. A summary of the results obtained is shown in Figure 5(a).

Fifty percent of students put the use of Solver in the first position. Thirty-eight percent of them chose mathematical modeling as their second favorite topic. By calculating the average between the places in which these two arguments have been placed by students, they obtain an average place of 2.3 and 2.6, respectively. Right after, we find the arguments related to the graphical solution, with an average position of 2.75. Instead, the binary-tree topic gets an average of 4.5: in fact, 75% of the students put it in place 5. The disaffection for this topic is probably because it was used only as an introduction to mathematical programming and was not particularly thorough. In Figure 5, (b)–(d), we find the results about some statements the students expressed in accordance with a Likert scale, from 1 (i.e., absolutely no) to 4 (i.e., absolutely yes). In particular, in Figure 5(b), we find the responses to the questions “Do you think you have understood the topic […] and know how to apply it correctly?” We note that 100% of the students believe they have fully or fairly well understood the graphical solution (by comparing the values of the objective function at the vertices of the feasible region) and mathematical modeling. The first of these topics did not require the acquisition of particular new knowledge. Regarding modeling, the many examples studied and developed have probably given students a certain degree of confidence in the topic. Finally, we point out that 38% believe they have not understood in part or at all the topic on the binary tree.

In Figure 5(c), we find the responses related to the question “Did you appreciate the teaching methodology […]?” The working-group sessions organized during the lectures were the most appreciated moments, with an average score of 3.7. Right behind, with a score of 3.5, we find the moments of interactivity obtained with the Mentimeter live polls that thus turns out to be a good tool to stimulate the students’ interest. Among the last positions, we find homework and group works that students had to do in extra school hours to tackle the authentic problems, with an average score of 2.3 and 2.8, respectively. The commitment required outside class, in addition to the ordinary homework of other subjects, was perhaps in some cases considered excessive. In Figure 5(d), there is a radar chart that shows the average scores to the question “Do you think you have acquired competence […]?” Each competence referred to one of the objectives (2–7) described in Section 3.1. We can see that, on average, students believe they have achieved all the six considered objectives. In particular, 70% of the students answered 4 regarding problem-solving and teamwork skills developed; 50% responded 4 referring to IT skills and the use of a solver.

Finally, we report some general considerations: 92% of the students considered the duration of the lectures to be adequate; 88% of the students asserted that what they learned in this first teaching unit will be useful in the future, whether at school or at work; and 92% of the students asserted that they would gladly repeat a similar experience where the world of university and school come together. Indeed, ROAR can be also seen as an orientation activity, as well as providing new knowledge and skills useful to those who want to enter the world of work after higher secondary school. Furthermore, it can involve and entice students to pursue a university career in a STEM discipline.

5.2.2. Qualitative.

In the written reports required by the Italian teacher of the class, many students wrote very positive sentences, such as “ROAR gave me the chance to realize that mathematics and modeling are part of daily reality way more than we thought” or “We are now able to tackle some real situations that we could not solve with just simple calculations.” They appreciated the fact to “ascertain and experience that the theory learnt can be concretized to face the real problems and needs of the working world.” In particular, one of them stated the following: “We moved from just knowing to knowing what to do.” Moreover, they reported that the activities helped them “think out of the box.” Furthermore, they enjoyed the opportunity to always work in teams, debating, and improving their relationships. Indeed, a student said “Each of us could individually contribute to the resolution.”

5.3. Feedback from Teachers

The mathematics teacher declared that “ROAR has helped decrease the misbeliefs that math is boring and useless because it consists of repetitive and convoluted computations. Problem-solving activities show the students how math can be creative.” She also pointed out that the lectures were “not an open project but adapted to the needs and skills of the students.” ROAR has given the opportunity to establish a formal collaboration between university and high school to improve the learning of mathematics and to experiment new teaching methods.

Also, other teachers gave positive feedback about the project. In particular, the science teacher said that the lectures were well balanced between theory and practice. The art teacher appreciated the setting of the problems near to students’ everyday life. Finally, the philosophy teacher underlined that this teaching unit offered students the opportunity to discuss the role of error: “As Popper said, we do not have to be afraid of making mistakes. These are precious because they allow us to improve, in order to make science closer to the truth and the reality. Moreover, I think that the collaborative part of ROAR is very relevant, being a work of consciousness and knowledge together.”

5.4. On Distance Learning

As mentioned previously, because of the COVID-19 pandemic, we held most of our lectures at a distance through the Microsoft Teams platform. Here we illustrate the advantages and disadvantages related to distance learning that we and the students faced during the activities.

About the advantages, the Microsoft Teams platform allowed all instructors, including observers, to always be present in every lecture. This also allowed having a number of observers equal to the number of groups into which the students were divided (this is not taken for granted in usual teaching experiments). Moreover, the observers were able to play their role more easily. In fact, they could record the activities of the groups in a more straightforward way than it would have been in the classroom, without some audiovisual equipment. With Microsoft Teams, we could also record each lecture and share it with the students who did not attend in a particular day or those who wanted to watch it again. Microsoft Teams also offered us the functionality of the breakout rooms, thus allowing us to do plenty of group work, starting it almost immediately once the groups have been formed. Conducting group work in an environment as small as the classroom might have been too confusing. Finally, distance learning simplified the sharing of screens and the performing of interactive live polls with Mentimeter, even if these would have been possible through the smart board available in the classroom.

Regarding disadvantages encountered, distance learning was quite challenging for the students in terms of concentration and attention and did not allow us to have direct feedback from them. Indeed, since Lecture 2, we had to force them to always keep the video on to ensure they were following the explanations and not doing anything else. It was more difficult to establish a relationship with the students, who felt blocked by the screen from asking questions or even just asking for clarification. It was only in the last three lectures that greater confidence with the experimenters was perceived.

In Figure 6, we can see the responses to two questions addressed to students about distance learning in the second questionnaire, asking, respectively, “How much did distance learning positively affect the understanding of the topics?” (left) and “To what extent has distance learning positively influenced the way the lectures are used?” (right). Students could respond with a score from 1 (not at all) to 4 (a lot). Most of the students (79% and 67%, respectively) found a few or no positive aspects to distance learning. In particular, we can read among the comments, left to some open questions, that it was not easy to maintain concentration and that often the Internet connection was not good, with the consequent loss of some snippets of explanation. Among the positive aspects, we find the possibility of rewatching a lecture (maybe because some students were absent or because they did not understand well a certain topic) or greater clarity in the explanation of some topics, such as Solver.

6. Conclusion and Further Work

We introduced ROAR, a three-year project composed of three teaching units addressed to grade 10, grade 11, and grade 12, respectively. The main aim of ROAR is to improve students’ interest, motivation, and skills related to STEM disciplines through collaborative learning, authentic problems, and digital technologies. The project was activated in early 2021, through a formal agreement between the Department of Mechanical and Industrial Engineering of University of Brescia and the scientific high-school IIS Antonietti in Iseo (Brescia, Italy).

In particular, we focused on the first teaching unit of ROAR by providing details about its design and implementation, from Spring 2021 in a grade 10 class composed of 25 students. With this unit, we wanted to introduce OR and mathematical modeling, together with linear, integer, and mixed integer linear programming. Moreover, we aimed to develop some IT skills, which was done by exploiting GeoGebra and Solver to tackle proposed problems, and some soft skills such as problem solving, teamwork, and public speaking. We held six lectures by alternating frontal teaching, collaborative learning, and interactive live polls performed through Mentimeter. To assess the students’ acquired knowledge, we split them into five groups and assigned each group an authentic problem to tackle and present during the last lecture. Every group was able to solve its assigned problem. Moreover, the theme of the authentic problems (i.e., optimizing different aspects of a chain of supermarkets) allowed other teachers to introduce several topics in their courses, thus making the project-work interdisciplinary. Most of the lectures were held in distance-learning mode by means of the Microsoft Teams platform, with 50% or 100% of the students remotely connected, as well as experimenters and observers. Only the last lecture, when students presented their works, was held on site with all of them. The feedback received was positive from both students and teachers, even considering distance-learning disadvantages.

All the slides used during the lectures and the material are available on a public repository.¹ In particular, the texts of all problems are both in Italian and English. We hope that they will be used by other researchers and/or higher secondary school teachers to develop other teaching units or similar experimentations. Indeed, although the three units have been conceived as consequential in the perspective of a PCTO project-work, the three individual modules could also be made independent of each other. They could easily be adapted to other activities as well, such as extracurricular workshops or seminars of shorter duration.

As further research, in a paper more focused on mathematics education (Taranto et al. 2022), we deeply analyzed the work done by three chosen groups of students. In particular, we discussed whether it is appropriate to include OR in ordinary mathematics lectures. Also, we investigated the role of collaborative group work and the use of digital technologies in fostering the students’ development of modeling competences and in the problem-solving process of authentic problems. In particular, we compared how the three groups tackled four significant problems assigned throughout the whole unit. Finally, we also investigated whether these activities had increased students’ understanding and appreciation of OR. Thus, this other work complements the activities and the results of the questionnaires described here by focusing on the impact of the first teaching unit of ROAR on the students.

In Winter 2022, we will implement the second teaching unit of ROAR in the same class by focusing on graph theory and network applications, such as road transportation. At the end of experimenting with the second teaching unit, we are going to provide a full description of it as we did here. We will do the same for the third teaching unit as well. Similar to what done in the first teaching unit, also at the beginning and at the end of the second and third teaching units, we are going to ask the students to fill in questionnaires analogous to the ones reported here. Thus, at the end of the ROAR experimentation, we will be able to perform a longitudinal study by analyzing the overall impact on the students.