Predicting Instructor Performance in Online Education: An Interpretable Hierarchical Transformer with Contextual Attention

References

• Adamopoulos P (2013) What makes a great MOOC? An interdisciplinary analysis of student retention in online courses. 34th Internat. Conf. Inform. Systems ICIS 2013 (Association for Information Systems, Atlanta), 13.Google Scholar
• Algozzine B, Gretes J, Flowers C, Howley L, Beattie J, Spooner F, Mohanty G, Bray M (2004) Student evaluation of college teaching: A practice in search of principles. College Teaching 52(4):134–141.Crossref, Google Scholar
• American Marketing Association (2020) Higher education marketing statistics before and during the covid-19 pandemic. Retrieved August 24, 2020, https://www.ama.org/marketing-news/higher-education-marketing-statistics-before-and-during-the-covid-19-pandemic/.Google Scholar
• Asr FT, Zinkov R, Jones M (2018) Querying word embeddings for similarity and relatedness. Walker M, Ji H, Stent A, eds. Proc. 2018 Conf. North Amer. Chapter Assoc. Comput. Linguistics Human Language Tech., vol. 1 (Long Papers) (Association for Computational Linguistics, Kerrville, TX), 675–684.Google Scholar
• Ba JL, Kiros JR, Hinton GE (2016) Layer normalization. Preprint, submitted July 21, https://arxiv.org/abs/1607.06450.Google Scholar
• Barbieri F, Camacho-Collados J, Neves L, Espinosa-Anke L (2020) TweetEval: Unified benchmark and comparative evaluation for tweet classification. Preprint, submitted October 23, https://arxiv.org/abs/2010.12421.Google Scholar
• Benton SL, Cashin WE (2014) Student ratings of instruction in college and university courses. Paulsen M, ed. Higher Education: Handbook of Theory and Research (Springer, Dordrecht, Netherlands), 279–326.Crossref, Google Scholar
• Breden A, Moore L (2020) Detecting adverse drug reactions from Twitter through domain-specific preprocessing and BERT ensembling. Preprint, submitted May 11, https://arxiv.org/abs/2005.06634.Google Scholar
• Chatterjee A, Ghosh C, Bandyopadhyay S (2009) Assessing students’ rating in higher education: A SERVQUAL approach. Total Quality Management Bus. Excellence 20(10):1095–1109.Crossref, Google Scholar
• Clough MP (2007) What is so important about asking questions? Iowa Sci. Teachers J. 34(1):2–4.Google Scholar
• Costin F, Greenough WT, Menges RJ (1971) Student ratings of college teaching: Reliability, validity, and usefulness. Rev. Ed. Res. 41(5):511–535.Crossref, Google Scholar
• Coursera (2020) Coursera 2020 impact report. Retrieved August 24, 2020, https://about.coursera.org/press/wp-content/uploads/2020/09/Coursera-Impact-Report-2020.pdf.Google Scholar
• Deci EL, Spiegel NH, Ryan RM, Koestner R, Kauffman M (1982) Effects of performance standards on teaching styles: Behavior of controlling teachers. J. Ed. Psych. 74(6):852–859.Crossref, Google Scholar
• Dellarocas C, Van Alstyne MW (2013) Money models for MOOCs. Comm. ACM 56(8):25–28.Crossref, Google Scholar
• Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. Burstein J, Doran C, Solorio T, eds. Proc. 2019 Conf. North Amer. Chapter Assoc. Comput. Linguistics Human Language Tech. (Association for Computational Linguistics, Kerrville, TX), 4171–4186.Google Scholar
• Di Vesta FJ, Di Cintio MJ (1997) Interactive effects of working memory span and text context on reading comprehension and retrieval. Learn. Individual Differences 9(3):215–231.Crossref, Google Scholar
• Feldman KA (1977) Consistency and variability among college students in rating their teachers and courses: A review and analysis. Res. Higher Ed. 6(3):223–274.Crossref, Google Scholar
• Feng W, Tang J, Liu TX (2019) Understanding dropouts in MOOCs. Proc. AAAI Conf. Artificial Intelligence 33(01):517–524.Crossref, Google Scholar
• Gordon RJ, Kane TJ, Staiger D (2006) Identifying Effective Teachers Using Performance on the Job (Brookings Institution, Washington, DC).Google Scholar
• Gupta V, Saw A, Nokhiz P, Netrapalli P, Rai P, Talukdar P (2020) P-SIF: Document embeddings using partition averaging. Proc. AAAI Conf. Artificial Intelligence 34(05):7863–7870.Crossref, Google Scholar
• Huang N, Zhang J, Burtch G, Li X, Chen P (2021) Combating procrastination on massive online open courses via optimal calls to action. Inform. Systems Res. 32(2):301–317.Link, Google Scholar
• IBISWorld (2020) Educational services in the US market size 2005–2026. Retrieved August 24, 2020, https://www.ibisworld.com/industry-statistics/market-size/educational-services-united-states/.Google Scholar
• Jones M, Sieck WR (2003) Learning myopia: An adaptive recency effect in category learning. J. Experiment. Psych. Learn. Memory Cognition 29(4):626–640.Crossref, Google Scholar
• Keith KA, Stent A (2019) Modeling financial analysts’ decision making via the pragmatics and semantics of earnings calls. Preprint, submitted June 7, https://arxiv.org/abs/1906.02868.Google Scholar
• Kokkodis M, Ipeirotis PG (2021) Demand-aware career path recommendations: A reinforcement learning approach. Management Sci. 67(7):4362–4383.Link, Google Scholar
• Kumar A, Mehra A (2018) Remedying education with personalized homework: Evidence from a randomized field experiment in India. Working paper, University of Florida, Gainesville.Google Scholar
• Le Q, Mikolov T (2014) Distributed representations of sentences and documents. Xing EP, Jebara T, eds. Proc. 31st Internat. Conf. Machine Learn., Proceedings of Machine Learning Research, vol. 32, no. 2 (PMLR, Bejing, China), 1188–1196.Google Scholar
• Lee D, Hosanagar K, Nair HS (2018) Advertising content and consumer engagement on social media: Evidence from Facebook. Management Sci. 64(11):5105–5131.Link, Google Scholar
• Lehman S, Schraw G (2002) Effects of coherence and relevance on shallow and deep text processing. J. Ed. Psych. 94(4):738–750.Crossref, Google Scholar
• Leung ACM, Santhanam R, Kwok RCW, Yue WT (2022) Could gamification designs enhance online learning through personalization? Lessons from a field experiment. Inform. Systems Res. 34(1):27–49.Link, Google Scholar
• Liu X, Zhang B, Susarla A, Padman R (2018) YouTube for patient education: A deep learning approach for understanding medical knowledge from user-generated videos. Preprint, submitted July 6, https://arxiv.org/abs/1807.03179.Google Scholar
• Liu X, Zhang B, Susarla A, Padman R (2020) Go to You Tube and call me in the morning: Use of social media for chronic conditions. MIS Quart. 44(1b):257–283.Crossref, Google Scholar
• Liu C, Wu X, Yu M, Li G, Jiang J, Huang W, Lu X (2019) A two-stage model based on BERT for short fake news detection. Douligeris C, Karagiannis D, Apostolou D, eds. Knowledge Sci. Engrg. Management 12th Internat. Conf. KSEM 2019 Proc. Part II, Lecture Notes in Computer Science, vol. 11776 (Springer, Berlin), 172–183.Google Scholar
• Loughran T, McDonald B (2011) When is a liability not a liability? Textual analysis, dictionaries, and 10-ks. J. Finance 66(1):35–65.Crossref, Google Scholar
• Mao J, Liu W (2019) A BERT-based approach for automatic humor detection and scoring. Proc. IberLEF 2019 Iberian Languages Evaluation Forum 2019 35th Conf. Spanish Soc. Natl. Language Processing (SEPLN 2019) (Bilbao, Spain), 197–202.Google Scholar
• Mohammad SM, Kiritchenko S, Zhu X (2013) NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. Preprint, submitted August 28, https://arxiv.org/abs/1308.6242.Google Scholar
• Mozafari M, Farahbakhsh R, Crespi N (2019) A BERT-based transfer learning approach for hate speech detection in online social media. Cherifi H, Gaito S, Mendes JF, Moro E, Rocha LM, eds. Complex Networks Appl. VIII Complex Networks 2019, Studies in Computational Intelligence, vol. 881 (Springer, Cham, Switzerland), 928–940.Google Scholar
• Narvaez D, Van Den Broek P, Ruiz AB (1999) The influence of reading purpose on inference generation and comprehension in reading. J. Ed. Psych. 91(3):488–496.Crossref, Google Scholar
• Olsher G, Kantor ID (2012) Asking questions as a key strategy in guiding a novice teacher: A self-study. Studying Teacher Ed. 8(2):157–168.Crossref, Google Scholar
• Pan L, Wang X, Li C, Li J, Tang J (2017) Course concept extraction in MOOCs via embedding-based graph propagation. Kondrak G, Watanabe T, eds. Proc. Eighth Internat. Joint Conf. Natl. Language Processing, vol. 1 (Long Papers) (Asian Federation of Natural Language Processing, Kerrville, TX), 875–884.Google Scholar
• Pappagari R, Zelasko P, Villalba J, Carmiel Y, Dehak N (2019) Hierarchical transformers for long document classification. 2019 IEEE Automatic Speech Recognition Understanding Workshop (ASRU) (IEEE, Piscataway, NJ), 838–844.Google Scholar
• Platz F, Kopiez R (2013) When the first impression counts: Music performers, audience and the evaluation of stage entrance behaviour. Musicae Sci. 17(2):167–197.Crossref, Google Scholar
• Qin Y, Yang Y (2019) What you say and how you say it matters: Predicting stock volatility using verbal and vocal cues. Korhonen A, Traum D, Màrquez L, eds. Proc. 57th Annual Meeting Assoc. Comput. Linguistics (Association for Computational Linguistics, Kerrville, TX), 390–401.Google Scholar
• Rehurek R, Sojka P (2010) Software framework for topic modelling with large corpora. Proc. LREC 2010 Workshop New Challenges NLP Frameworks (University of Malta, Valletta), 45–50.Google Scholar
• Research and Markets (2020) Global online education market worth $319+ billion by 2025 - North America anticipated to provide the highest revenue generating opportunities. Retrieved August 24, https://www.globenewswire.com/news-release/2020/04/16/2017102/0/en/Global-Online-Education-Market-Worth-319-Billion-by-2025-North-America-Anticipated-to-Provide-the-Highest-Revenue-Generating-Opportunities.html.Google Scholar
• Roitero K, Bozzato C, Della Mea V, Mizzaro S, Serra G (2020) Twitter goes to the doctor: Detecting medical tweets using machine learning and BERT. Couto FM, Krallinger M, eds. Proc. Internat. Workshop Semantic Indexing Inform. Retrieval Health Heterogeneous Content Types Languages (SIIRH 2020).Google Scholar
• Rowan B, Chiang FS, Miller RJ (1997) Using research on employees’ performance to study the effects of teachers on students’ achievement. Sociol. Ed. 70(4):256–284.Crossref, Google Scholar
• Schraw G, Wade SE, Kardash CA (1993) Interactive effects of text-based and task-based importance on learning from text. J. Ed. Psych. 85(4):652–661.Crossref, Google Scholar
• Shavelson RJ (1973) What is the basic teaching skill? J. Teacher Ed. 24(2):144–151.Crossref, Google Scholar
• Sun C, Adamopoulos P, Ghose A, Luo X (2022) Predicting stages in omnichannel path to purchase: A deep learning model. Inform. Systems Res. 33(2):429–445.Link, Google Scholar
• Suzuki D, Fujiwara T (2017) The multifunctionality of ‘possible’ modal adverbs: A comparative look. Language 93(4):827–841.Crossref, Google Scholar
• Tost LP, Gino F, Larrick RP (2013) When power makes others speechless: The negative impact of leader power on team performance. Acad. Management J. 56(5):1465–1486.Crossref, Google Scholar
• Trakunphutthirak R, Cheung Y, Lee VC (2019) A study of educational data mining: Evidence from a Thai university. Proc. AAAI Conf. Artificial Intelligence 33(01):734–741.Crossref, Google Scholar
• United Nations (2020) Policy brief: Education during COVID-19 and beyond. Retrieved August 24, https://www.un.org/development/desa/dspd/wp-content/uploads/sites/22/2020/08/sg_policy_brief_covid-19_and_education_august_2020.pdf.Google Scholar
• van de Grift W, Helms-Lorenz M, Maulana R (2014) Teaching skills of student teachers: Calibration of an evaluation instrument and its value in predicting student academic engagement. Stud. Ed. Evaluation 43:150–159.Crossref, Google Scholar
• Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Guyon I, Von Luxburg U, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, eds. Adv. Neural Inform. Processing Systems 30 (NIPS 2017) (Long Beach, CA), 5998–6008.Google Scholar
• Wang W, Li B (2021) Empower audience creativity using multi-modal video analytics: Evidence from TED talks. ICIS 2021 Proc. 12 (Association for Information Systems, Atlanta).Google Scholar
• Wang Y, Currim F, Ram S (2022) Deep learning of spatiotemporal patterns for urban mobility prediction using big data. Inform. Systems Res. 33(2):579–598.Link, Google Scholar
• Wang T, He C, Jin F, Hu YJ (2021) Evaluating the effectiveness of marketing campaigns for malls using a novel interpretable machine learning model. Inform. Systems Res. 33(2):659–677.Link, Google Scholar
• Wang X, Shi W, Kim R, Oh Y, Yang S, Zhang J, Yu Z (2019) Persuasion for good: Towards a personalized persuasive dialogue system for social good. Preprint, submitted June 16, https://arxiv.org/abs/1906.06725.Google Scholar
• Williamon A, ed. (2004) Musical Excellence: Strategies and Techniques to Enhance Performance (Oxford University Press, New York).Crossref, Google Scholar
• Wilson WR (1932) Students rating teachers. J. Higher Ed. 3(2):75–82.Crossref, Google Scholar
• Yang K, Lau RY, Abbasi A (2022) Getting personal: A deep learning artifact for text-based measurement of personality. Inform. Systems Res. 34(1):194–222.Link, Google Scholar
• Yu J, Wang C, Luo G, Hou L, Li J, Liu Z, Tang J (2019) Course concept expansion in MOOCs with external knowledge and interactive game. Korhonen A, Traum D, Màrquez L, eds. Proc. 57th Annual Meeting Assoc. Comput. Linguistics (Association for Computational Linguistics, Kerrville, TX), 4292–4302.Google Scholar
• Zhang DJ, Allon G, Van Mieghem JA (2017) Does social interaction improve learning outcomes? Evidence from field experiments on massive open online courses. Manufacturing Service Oper. Management 19(3):347–367.Link, Google Scholar
• Zhang J, Hao B, Chen B, Li C, Chen H, Sun J (2019) Hierarchical reinforcement learning for course recommendation in MOOCs. Proc. AAAI Conf. Artificial Intelligence 33(01):435–442.Crossref, Google Scholar
• Zhou M, Chen GH, Ferreira P, Smith MD (2021) Consumer behavior in the online classroom: Using video analytics and machine learning to understand the consumption of video courseware. J. Marketing Res. 58(6):1079–1100.Crossref, Google Scholar