Predicting Instructor Performance in Online Education: An Interpretable Hierarchical Transformer with Contextual Attention

Online education is a vital consumer industry that is undergoing rapid technological change. This paper develops a deep learning model to predict instructor performance on online education platforms from a content-based perspective. Specifically, we design an interpretable hierarchical transformer with contextual attention to predict instructor rating and course rating using textual data. Our model captures the inherent hierarchical structure of online courses and the sequential dependency of lectures within a course. Moreover, it goes beyond prediction and enables interpretability analysis, which provides additional insights into potential reasons behind the predictions made by the model. Extensive experiments demonstrate that our model outperforms classic machine learning models as well as state-of-the-art deep learning models. Furthermore, we conduct in-depth interpretability analysis to explore what factors might predict the success of an online course at the lecture, sentence, and word level. We also showcase the value and applicability of our model through a randomized experiment. Our findings and methods provide managerial implications for instructors and online education platforms to improve course creation and delivery in this vitally important emerging market.

History: Ravi Bapna, Senior Editor; Tianshu Sun, Associate Editor.

Funding: This work was supported by the Social Sciences and Humanities Research Council of Canada [Grant 430-2022-00297].

Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2021.0310.