Note from the Editor

This is my last message to you as INFORMS Journal on Computing editor-in-chief. Overall, the journal is in very healthy shape. Submissions and usage are stronger than ever, and the bimonthly publication schedule has proven a perfect match for the robustness of the journal. Our unique IJOC GitHub Site (https://github.com/INFORMSJoC) with nearly 300 permanent fully licensed repositories of code, data, and digital artifacts is a powerful and lasting contribution to the field. The implementation of specialized Data and Software Associate Editors has ensured smooth ongoing populating of this site. The addition of a promising new technical area in Quantum Computing as well as the special issues activity reflect the relevance of the journal as technology evolves.

We have recognized those that contribute in an exceptional manner to the discipline and to the journal through the Test of Time Paper Award, Meritorious Reviewer Award, and Meritorious Paper Award. The retrospections published with the Test of Time Award announcements are especially important as they represent the authors’ reflections on their most impactful works. Collectively, these recognized authors are a set of the key scholars of our time (https://pubsonline.informs.org/page/ijoc/test-of-time). I have been especially touched that the unsung heroes of the peer reviewing system—the reviewers—have felt valued after receiving a Meritorious Reviewer award. And we reward the Meritorious Papers with free access. Additionally, each issue we recognize two to three papers as Featured Articles. These are also free access when opened from the featured banner on our website.

I wish we could award a Meritorious Editor award, but I am afraid we would need to give that to nearly all the 100+ individuals who comprise our editors. The Editorial Board has done a good job of managing the growing workload by keeping quality of reviews high and review times reasonable. In fact, we are known for the technical depth of our reviews. The authors and the editors are diverse in terms of geographic area and affiliation, and many are early career professionals.

In closing, we can all be proud of the INFORMS Journal on Computing. We honor its heritage and its hallmarks of progressiveness and excellence with every action of the editorial board, the authors, and the reviewers. As a worldwide team of volunteers, it is really amazing that we can accomplish so much. This is a tribute our devotion to the discipline—past, present, and most importantly, future. It has been both a great honor and a true joy to serve as editor-in-chief for seven years.

All my best,

I am pleased to continue to announce the winners of the INFORMS Journal on Computing (IJOC) Test of Time Paper Award to cover the backlog of awards since the journal’s inception. The energetic and able committee, chaired by John Chinneck with members Bill Cook, Bruce Golden, Karla Hoffman, and David Woodruff, have selected the awardee covering the period 2007 through 2011. What follows is the citation from the award committee and then a reflection from the authors concerning their paper.

I want to thank the committee for their superb efforts and am very pleased to share this recognition of the impactful heritage of our journal.

The Test of Time Award for papers published in the INFORMS Journal on Computing within the years 2007–2011 is awarded to

The Knowledge-Gradient Policy for Correlated Normal Beliefs

Peter Frazier, Warren Powell, Savas Dayanik

INFORMS Journal on Computing 21(4):599–613, Fall 2009

https://pubsonline.informs.org/doi/10.1287/ijoc.1080.0314

Test of Time Award Citation 2007–2011

This paper introduces the now well-known knowledge-gradient (KG) method for gathering information when faced with a black-box objective, where query measurements may be both costly and noisy. The focus of the paper is on ranking and selection, but the KG method can be used also to study multiarmed bandit problems and many other models in Bayesian information collection. This very broad applicability led to a lively KG research area that continues to this day, with new developments studied in modern machine learning. It is an excellent example of work that has stood the test of time.

Below is the retrospective from the authors, Peter Frazier, Warren Powell, and Savas Dayanik, concerning their landmark paper. Note also, they have now published the code associated with this paper on the IJOC GitHub site at https://github.com/INFORMSJoC/1080.0314.

Retrospective from the Authors Peter Frazier, Warren Powell, and Savas Dayanik

We are deeply honored by this recognition and grateful to the selection committee and editor-in-chief for this award.

This work contributes to Bayesian optimization, a powerful approach for optimizing time-consuming-to-evaluate nonconvex derivative-free objective functions. Bayesian optimization builds a probabilistic surrogate (usually a Gaussian process) for the objective function $f(x)$. It chooses where to evaluate $f(x)$ next by optimizing a so-called “acquisition function” that quantifies the value of an evaluation of $f(x)$ as a function of the point evaluated, x. This work also contributes to the broader areas of optimal learning and Bayesian sequential experimental design, which use a Bayesian approach to intelligently choose which data to collect or which experiment to run in settings where data are expensive to collect.

This work played a significant role in developing the KG acquisition function. The KG acquisition function at x is the value of evaluating the objective function at x and then, based on the $f(x)$ observed, selecting a final solution to the optimization problem. It is distinguished from the widely used expected improvement method because the final solution selected can be a point that has not been previously evaluated. We began studying this approach as a one-step lookahead approximation to the full dynamic programming treatment of the problem (later documented by Ginsbourger and Le Riche 2010). What surprised us is that looking only one step ahead can produce an extremely effective method.

This work introduced the KG acquisition function for correlated normal prior distributions—the discrete analog of a Gaussian process. Previous work on KG, comprehensively reviewed in Garnett (2023), only looked at a restricted case of independent priors (Gupta and Miescke 1996, Frazier et al. 2008) or Gauss-Markov priors where the final solution selected is a previously evaluated point (Mockus 1972). The more flexible correlated normal prior distributions enabled by the awarded work are critical for achieving good empirical performance in many problems.

Study of Bayesian optimization grew rapidly following the later discovery by Snoek et al. (2012) that Bayesian optimization effectively tunes hyperparameters of deep neural networks. KG became one of the main acquisition functions used alongside expected improvement, probability of improvement, entropy search, upper confidence bound, and Thompson sampling.

Papers building on the awarded work discovered that KG methods are particularly effective when we can acquire information from sources other than direct objective function evaluation. For example, in multi-information source Bayesian optimization (Poloczek et al. 2017), we can evaluate less expensive but biased surrogates for the objective (e.g., a simulation that replaces a stochastic process by a fluid approximation). Many of the most widely used acquisition functions (including expected improvement) do not extend easily to this setting, but KG methods do so naturally. This led to the development of KG methods for a wide variety of problems. Examples include active learning for allocating crowd workers to labeling tasks (Chen et al. 2013), optimization of compositions of unknown time-consuming black-box functions (Astudillo and Frazier 2021, Buathong et al. 2024), and optimization with gradient observations (Wu et al. 2017), common random numbers (Pearce et al. 2022), or across multiple tasks (Pearce and Branke 2018).

The knowledge gradient has also been adopted outside of Bayesian optimization, especially in the literature on optimal learning (Powell and Ryzhov 2012). This includes work on online learning (Ryzhov et al. 2012), preference exploration (Lin et al. 2022), hierarchical aggregation for settings where each choice is described by a vector of attributes (Mes et al. 2011), and work for parametric belief models (Negoescu et al. 2011, He and Powell 2018, He et al. 2020).

Later, KG methods were implemented in BoTorch (Balandat et al. 2020), a widely used high-quality open-source software package for Bayesian optimization maintained by Meta. The implementation was based on a discovery that sample average approximation on top of the computing infrastructure provided by PyTorch is an effective approach for optimizing KG. KG is more complex than most of the other standard acquisition functions, and this advance simplified and further accelerated the use of KG. Spurred by this award, we have migrated the original code from a zip file on a web page to a more accessible GitHub repository (Frazier et al. 2009).

Looking forward, algorithms enabled by KG continue to be important for core applications in operations research, machine learning, and engineering. New questions and opportunities continue to emerge—for example, how to best allocate computing effort in multi-LLM (large language model) routing or agentic workflows and for directing self-driving labs for materials and drug discovery. KG methods are a powerful and flexible tool in these problems, and we are grateful to the selection committee for drawing attention to them.