MIT-Janssen’s COVID-19 Forecast Model

Note: The Innovative Applications in Analytics Award (IAAA) is a prestigious award developed by the Analytics Society of INFORMS to recognize creative and unique applications of a combination of analytical techniques in new areas. Presented each year by the Analytics Society along with Kinaxis and Adelphi University, the award attracts submissions from around the world whose work is judged by a panel of experts. Below is the sixth and final in a series of brief articles describing the work of 2021 IAAA finalists, in this case, the 2021 award-winner MIT-Janssen. 

When COVID-19 emerged at the end of 2019, little was understood about the disease. MIT researchers and scientists at Janssen Research & Development (Janssen) leveraged real-world data and applied artificial intelligence and machine learning (AI/ML) to help guide the company’s research efforts into a potential vaccine.

“Data science and machine learning can be used to augment scientific understanding of a disease,” says Najat Khan, chief data science officer and global head of strategy and operations for Janssen Research & Development. “For COVID-19, these cutting-edge disciplines became even more important because ­­­our knowledge was rather limited. There was no hypothesis at the time. We were developing an unbiased understanding of the disease based on real-world data using sophisticated AI/ML algorithms.”

Khan put out a call for collaborators on predictive modeling efforts to partner with her data science team to identify key locations to set up trial sites. Through Regina Barzilay, the MIT School of Engineering Distinguished Professor for AI and Health, faculty lead of AI for MIT’s Abdul Latif Jameel Clinic for Machine Learning in Health, and a member of Janssen’s scientific advisory board, Khan connected with Dimitris Bertsimas, the Boeing Leaders for Global Operations Professor of Management at the MIT Sloan School of Management.

DELPHI Model

When the World Health Organization declared COVID-19 a pandemic in March 2020 and forced much of the world into lockdown, Bertsimas, who is also the faculty lead of entrepreneurship for the Jameel Clinic, brought his group of 25-plus doctoral and master’s students together to discuss how they could use their collective skills in machine learning and optimization to create new tools to aid the world in combating the spread of the disease.

The group started tracking their efforts on the COVID Analytics platform, where their models generate accurate real-time insight into the pandemic. One of the group’s first projects was charting the progression of COVID-19 with an epidemiological model they developed named DELPHI, which predicts state-by-state infection and mortality rates based upon each state’s policy decision using an expanded SEIR model. A key innovation of the model is capturing the behaviors of people related to measures put into place during the pandemic, such as lockdowns, mask-wearing and social distancing, and the impact these had on infection rates.

“By June or July, we were able to augment the model with these data. The model then became even more accurate,” Bertsimas says. “We also considered different scenarios for how various governments might respond with policy decisions, from implementing serious restrictions to no restrictions at all, and compared them to what we were seeing happening in the world. This gave us the ability to make a spectrum of predictions. One of the advantages of the DELPHI model is that it makes predictions on 120 countries and all 50 U.S. states on a daily basis.”

A Vaccine for Today’s Pandemic

Being able to determine where COVID-19 is likely to spike next proved to be critical to the success of Janssen’s clinical trials, which were “event-based” – meaning that “we figure out efficacy based on how many ‘events’ are in our study population, events such as becoming sick with COVID-19,” Khan explains.

The MIT team began collaborating with Khan and her team last May to forecast COVID-19 hot spots where Janssen could conduct clinical trials and recruit participants who were most likely to get exposed to the virus. To understand how the virus was moving around the world, data scientists at Janssen continuously monitored and scouted data sources across the world. The team built a global surveillance dashboard that pulled in data at the country, state and even county level based on data availability of case numbers, hospitalizations, mortality and testing rates. The DELPHI model integrated these data, with additional information about local policies and behaviors, such as whether people were being compliant with mask-wearing, and was making daily predictions in the 300-400 range.

Remarkably, the vast majority of Janssen’s clinical trial sites that DELPHI predicted to be COVID-19 hot spots ultimately had extremely high numbers of cases, including in South Africa and Brazil, where new variants of the virus had surfaced by the time the trials began. In addition, the DELPHI model further evolved with diversity in mind, taking into account biological risk factors, patient demographics and other characteristics. “COVID-19 impacts people in different ways, so it was important to go to areas where we were able to recruit participants from different races, ethnic groups and genders. Due to this effort, we had one of the most diverse COVID-19 trials that’s been run to date,” Khan says.

In April, the MIT and Janssen’s Research & Development Data Science team were jointly recognized by the Institute for Operations Research and the Management Sciences (INFORMS) as the winner of the 2021 Innovative Applications in Analytics Award for their innovative and highly impactful work on COVID-19. Building on this success, the teams are continuing their collaboration to apply their data-driven approach and technical rigor in tackling other infectious diseases.

“This was not a partnership in name only. Our teams really came together in this and continue to work together on various data science efforts across the pipeline,” Khan says. The team further appreciates the role of investigators on the ground, who contributed to site selection in combination with the model.

“It was a very satisfying experience,” concurs Bertsimas. “I’m proud to have contributed to this effort and help the world in the fight against the pandemic.”