Practice Prize Report: The 2024 Gary Lilien ISMS Practice Prize Competition

What Is the Practice Prize?

The INFORMS Society for Marketing Science (ISMS) runs the Gary Lilien Practice Prize Competition to celebrate excellence in applied marketing science. The year 2024 represented the 12th round in which the Practice Prize has been awarded. Since the first competition in 2003, the Prize has attracted hundreds of submissions, resulting in 43 finalists. These finalists demonstrate marketing science applied to a wide range of managerial problems, using an immense diversity of research techniques. Lilien et al. (2013) provide an evaluation and summary of this portfolio, and many finalists’ work has also been published in Marketing Science. The Practice Prize Competition is an important communications vehicle for marketing scientists concerned with the practice side of our profession. The Practice Prize provides an avenue to celebrate impactful marketing science applied to solving real managerial/public policy problems and the way in which it pursues that objective.

The Practice Prize is awarded for outstanding implementation of marketing science concepts and methods. The methods used must be innovative, sound, and appropriate to the problem and organization, and the work should have had a significant, verifiable, and preferably quantitative impact on the performance of the client organization. In assessing the rigor of the prize entries, along with their focus on relevance and organizational impact, the judging panel takes a number of criteria into account. Specifically, the panel considers the following:

• Implementation: Who uses the research, for what, and how?

• Impact: How did the research impact decisions, and what was the effect of those changed decisions?

• Methodological quality: Is the methodology appropriate to the problem and, preferably, innovative?

• Technical originality: Do the problem, approach, and implementation demonstrate uniqueness and flair?

• Difficulty: Does the application solve hard problem(s), whether that difficulty arises from problem formulation, methodological solution, or implementation complexity?

• Transportability: Is the work applicable to other contexts, whether potential or demonstrated, that are valuable to drive organizational adoption of marketing science forward?

• Charm: Does the application demonstrate the work of marketing scientists to engage with organizational stakeholders, including its impact on society and newsworthiness?

The 2024 Competition

The 2024 prize committee comprised Lan Luo (Professor of Marketing at the University of Southern California; ISMS Vice President (VP) of Practice) as the Prize Committee Chair, Nicholas Chu (CEO and Founder, Sinorbis; and Professor of Practice at the University of New South Wales), Steve Cohen (Cofounder of In4mation Insights, LLC), Tamara Howe (CMO at SunRice Group), Wendy Mak (CMO at MUFG Pension & Market Services), Koen Pauwels (Distinguished Professor of Marketing at Northeastern University), and Olivier Toubia (Glaubinger Professor of Business at Columbia University). Gary Lilien, Koen Pauwels, and John Roberts provided feedback during the finalists’ rehearsal presentations. The committee received several excellent entries, each of which described both the work itself and the impact that the work has had on the client organization. From that set of entries, the judges selected four finalists and then a winner, following presentations at the ISMS Marketing Science Conference organized by the University of New South Wales in Sydney, Australia, on June 27, 2024.

The winning paper, “Sales Automation,” was presented by Prof. Zhu on behalf of the author team (Hu et al. 2024). This paper reported a deep-learning-based recommender system that the author team has developed for new salespeople at a major insurance company in China that managed over 100,000 sales professionals. Low productivity among new sales agents is a common problem in sales force management. Prof. Zhu and her coauthors developed a model that learns from experienced salespeople’s transaction records to help new salespeople identify suitable customers with high conversion potential. The model is designed to actively learn from “missing by choice” data, such as experienced salespeople’s own failures, which are prevalent, but often not recorded in the company database. The authors validated their method using sales force transaction data from their client company. The author team showed that their method outperforms common benchmarks, while being easy to implement. Their recommender system has started to be rolled out at the client company via a mobile app, with a one-third organic adoption rate among sales professionals upon launch.

The winner showcases a fruitful large-scale and sophisticated collaboration between a major corporation and a team of highly competent scholars. Home to nearly 10% of the labor force in the United States, sales is one of the private sectors with the highest labor participation rate. More importantly, the sales job industry has historically been considered as a “low-tech” sector, where past experiences, clientele base, and interpersonal skills such as likeability, persuasion, and effective communications tend to dominate business practice in this industry. Meanwhile, sales jobs are notoriously known as one of the most stressful jobs, especially for new sales professionals. As such, the sales sector is also known for its high attrition rate. In this paper, the author team was able to address several pain points in the sales sector by leveraging the existing transactional records from the experienced salesforce to develop a neural recommendation system so that each new salesperson is offered a personalized recommender of customer types of high conversion potential. Their recommender system was well-adopted by new sales reps in a major insurance company in China. Overall, this winner is an exemplar of a rigorous and relevant research that has a long-reaching impact on both the client organization and the sales sector in general.

The other finalists in the competition (in alphabetical order of first author in the submission) were as follows. Prof. Kolsarici presented their work on motivating organizations’ sustainable energy consumption (Amaral et al. 2024). Despite a breadth of research showing how to reduce individuals’ energy consumption with techniques such as behavioral insights and dynamic energy pricing, research on their effectiveness in organizations and over time has been scarce. Amaral and colleagues partnered with an energy consulting company in Canada to develop and test a multidisciplinary approach to reduce organizations’ energy consumption in a demand pricing program. Using a multiphase longitudinal randomized field experiment, they tested the effectiveness of improved demand forecast models using artificial intelligence and behaviorally informed emails (i.e., leveraging planning prompts) in a range of large and diverse organizations in Canada. They discovered that enhancing the pricing program with both improved models and behaviorally informed emails each significantly contributed to organizations’ reduced energy consumption. In addition, the authors demonstrated that their interventions remained effective, even after repeated exposures. This paper provides evidence of the applicability of a multidisciplinary solution in which both artificial intelligence and behavioral nudges are jointly leveraged to effectively reduce organizations’ energy consumption.

In another finalist paper, Cowan et al. (2024) presented the development and adoption of a novel customer lifetime value (CLV) at a large UK lender. Understanding customer lifetime value is key to nurturing long-term customer relationships; however, estimating it is far from straightforward. In the banking industry, commonly used approaches rely on simple heuristics and often do not take advantage of the high predictive ability of modern machine learning techniques. In this paper, Cowan and colleagues developed a novel CLV framework in which machine learning methods are used to facilitate CLV predictions over flexible time horizons on the basis of product-based propensity models. The authors detail the creation, testing, and deployment of this model, which is currently in production at a large UK lender. In testing, they estimate a 43% improvement in out-of-time CLV prediction error relative to a popular baseline approach. Additionally, the authors reported that the top 10% of customers ranked by their propensity model to take up investment products were 3.2 times more likely to take up an investment product in the next year than a customer chosen at random. The propensity models derived from the proposed CLV model have been used to support customer contact marketing campaigns from the focal organization.

Finally, Prof. Sahni presented a fractional factorial design framework to speed up business policy experiments in collaboration with DoorDash (Tang et al. 2024). This paper investigates an approach to both speed up business decision making and lower the cost of learning through experimentation by factorizing business policies and employing fractional factorial experimental designs for their evaluation. The authors illustrate how this method can be used to estimate both average and heterogeneous treatment effects, elaborating on its advantages and foundational assumptions. The authors empirically demonstrated the implementation and benefits of the proposed approach and assessed its validity in evaluating consumer promotion policies at DoorDash, which is one of the largest delivery platforms in the United States. This approach discovers a policy with 5% incremental profit at 67% lower implementation cost for their client organization.

Discussion

Compared with previous years, the 2024 Practice Prize Competition was distinctive in the following ways. First, the entries to the 2024 Practice Prize Competitions are highly diverse geographically. The four finalist papers showcased considerable and long-reaching impact of marketing science with clients from four distinctive countries (China, Canada, the United Kingdom, and the United States). This year’s competition provided evidence for the prosperity and adoption of rigorous and meaningful marketing science applications around the globe. In the past, winners and finalists of the Practice Prize Competition typically presented impactful collaborations with organizations or government agencies from developed countries. It is worth noting that the winner of the 2024 Practice Prize Competition presented a rigorous and large-scale implementation of an artificial intelligence-based recommender system in a major insurance company in China. The CEO of the client organization and several senior members of the company also attended the 2024 Practice Prize Competition presentations in Sydney, Australia, to demonstrate their support for the author team. This is an encouraging manifestation that our profession now produces impactful, rigorous, and relevant work that is well-received and adopted not only in developed countries, but also in developing economies.

Second, we noticed that three out of four finalists in the 2024 Practice Prize Competition employed machine learning/artificial intelligence methods in their papers. In the last decade, more and more researchers in our field have leveraged the rapid developments in machine learning to help us address various marketing science problems. As an applied domain under the social science umbrella, the field of marketing has always had a tradition to leverage knowledge and toolkits from other domains. It is exciting to witness that our field has once again embraced modern technologies such as machine learning to sharpen our toolkit within the context of not only academic research, but also business practice. We expect our field to continue adopting the latest knowledge and tools from all science and social science domains to further advance our field and broaden our impact on business practices and the broader society.

Last, but not least, we would like to note that the 2024 Practice Prize Competition has the highest female representation on the judging panel ever since the inception of this competition in 2003. This year, about 50% of the judging panel are females, including a female serving as the chair of the panel. In the past, the field of marketing science has been predominately male-centric. We are pleased to witness increasing female representation of leadership roles in our field. The judging panel for the 2024 Practice Prize Competition is one example of this positive movement in our field.

Despite these new developments, our key learnings as VP of Practice also reflect the insights from our predecessors. In line with Lilien et al. (2013), we see multiple approaches for marketing science to add value to client organizations: there is no one silver bullet. Simpler, easier-to-use models that offer robust and improved results can have a stronger impact than academically sophisticated models. Organizational buy-in is still critical and requires researchers to embed themselves and speak the same language as the decision makers. Moreover, we agree with Roberts (2020) that academic papers are too often judged by their perfection on specific criteria in isolation: Are all possible alternative explanations ruled out? Are the data ideal? Is the method the most ground-breaking? Instead, it is in the connections between these criteria where winners shine and less successful applications fall short. In some cases, a simpler approach, which is easier to explain and get buy-in for, can make a long-lasting impact on business practice as compared with a powerful statistical sledgehammer. Lastly, consistent with Luo and Pauwels (2023), we believe that there are many ways to demonstrate impact via different methods, a wide range of collaborating entities, and/or diverse overall objectives. Quantitative marketing can be used to enhance not only the profitability of private entities, but also the welfare of the worlds’ other stakeholders and institutions. In simple terms, all roads lead to Rome, provided we deliver rigorous and relevant work that demonstrates a significant impact on client organizations.

In summary, the discussed projects, and the other entrants, showcase the collaboration of marketing scientists with practitioners. We encourage more academics and decision makers to participate in such projects and are looking forward to your submission to the 2026 Gary Lilien ISMS Practice Prize.

Comment

The prize committee congratulates the finalists and winners for their outstanding work and contributions to the practice of marketing science. ISMS is keen to disseminate the excellent work embodied by the entries in the Practice Prize Competition. The Practice Prize is awarded to outstanding implementation of marketing science concepts and methods. Prior publication of the work in any peer-reviewed journal does not disqualify it, and several past finalists and winners were published before the prize. The Practice Prize committee also welcomes submissions that are working papers, regardless of whether the paper is under review or not. Entry to this competition does not preclude the paper from being submitted/published in any journal of the authors’ choice. All practice prize finalists are also encouraged to submit their papers to Marketing Science for review. The submission and review process at Marketing Science is independent from the submission and review process for the Practice Prize Competition. And videos of finalists’ presentations are available for illustration or classroom use at http://lilienpracticeprizevideos.org/ and on the dedicated YouTube channel, https://www.youtube.com/channel/UCrS2aW6TiebaWVMjGGFH8oQ.