What GenAI Taught Us About the Future of Global Optimization

Are your optimization models nonlinear yet? Recent advances in nonlinear optimization technology are paving the way for the use of full-scale nonlinear models in production, thereby mitigating the need for linearized models.

Nonlinear Optimization Coming of Age

For decades, operations research (O.R.) practitioners have been taught to bend reality by modeling applications as linear optimization problems. Nonlinear relationships were routinely approximated, relaxed or ignored, not because they were unimportant but because they were computationally intractable. That situation is currently fundamentally changing. In this article, we share our aha moment that occurred when looking deeper into a recent generative artificial intelligence (AI) success story.

A Wake-up Call from AI

In May 2025, Google DeepMind released a short but striking report [1] on a notable experiment: the researchers used AlphaEvolve, an ensemble of large language models (LLMs) based on Google’s Gemini, to generate Python code to find solutions to various mathematical problems. Many of the benchmark problems it addressed were, from an O.R. perspective, classical nonlinear optimization problems. The codes generated by the LLMs found improving solutions for many problems without relying on classical optimization solvers or algorithms, which is an interesting and innovative use of AI.

Classical Algorithms Strike Back

Because many of these problems can be modeled directly as nonlinear optimization problems, we decided to test how far modern global optimization solvers could go. We formulated several of the AlphaEvolve problems as nonlinear optimization models and solved them using FICO® Xpress Optimization. Somewhat to our surprise, the solver, out of the box, was not only able to match the solutions found by the AlphaEvolve team but also, in some cases, found even better solutions.

One of the test problems was a variant of the classical circle packing problem: packing unit circles into a unit square (or, more generally, into a rectangle of perimeter 4) to maximize the sum of their radii. Figure 1 shows a newly discovered configuration for this problem. A key advantage of modeling-based optimization became immediately apparent. Changing the container from a square to a rectangle of fixed perimeter requires adding just one decision variable and one constraint; the remainder of the model is unchanged. By contrast, custom heuristics must often undergo a major refactoring (or retraining) when the underlying problem changes.

In joint work with Gioni Mexi and Dominik Kamp from Zuse Institute Berlin, we also obtained new solutions for the minimum distance ratio and the hexagon packing problem from the latest AlphaEvolve paper [2]. We summarized our findings in a blog post [3] and a couple of conference talks, focusing on the strengths and weaknesses of the two approaches: direct modeling and optimization with a general-purpose solver versus code generation targeting the specific instance. (The preprint of an upcoming publication is on arXiv [4].)

Based on this experience, we started to look at other packing problems in Erich Friedman’s packing problem collection [5]. We settled on the problem of packing unit circles into an ellipse of minimal area. The most challenging part of the model is characterizing the inclusion of a circle within an ellipse, which we addressed using the S-lemma [6]. Figure 2 shows solutions for ellipses packed with nine and 15 circles.

MINLP: The New MIP?

The usual way to introduce mixed integer nonlinear programming (MINLP) or global optimization is that, unlike local optimization, these algorithms guarantee convergence to a globally optimal solution. Although true, this does not acknowledge the power of modern global optimization solvers, and potential users may be deterred by the perceived difficulty of this class of problems. In our view, a better way to think about these algorithms is that they provide the same kind of guarantees and solutions that we have become accustomed to over the decades from mixed-integer programming (MIP) solvers: Along with a primal solution, they also generate a valid dual bound and report a duality gap to quantify how much more room there is for optimization. You can use a good global optimization solver the same way as you would use a mixed-integer linear optimization solver, but the constraints and objective can be nonconvex and nonlinear and can even include integralities. For practitioners, this means you can now directly model nonlinear aspects of your business problems without sacrificing reliability or decision confidence.

This difference is manifested in the comparison against AlphaEvolve: The heuristics it generated provide only primal solutions without any quality guarantees. Even though the solutions tend to be good, the user is left wanting, not knowing how much is left on the table. In business decision-making, that missing information matters. There are a few remedies for that. One is presented in an independent, open-source replication of the AlphaEvolve study, where in one case, the LLM ended up writing an optimization model and feeding it into HiGHS through SciPy, which it had access to. This can be augmented by the approach explored in a follow-up to the AlphaEvolve study [2], where the relevant literature was provided to the LLM, and it was prompted to provide mathematical insights. This could be used to add valid inequalities to the generated optimization model, strengthening the dual bound. These hybrid directions are promising yet only further reinforce the central role of mature optimization technology.

However, be careful when it comes to validating approaches that affect the dual side of a problem. On the one hand, when the LLM output is a primal heuristic, you can independently verify the solutions it provides. On the other hand, if the output is an optimization model or a custom bound-providing algorithm, then claims about the dual bound are difficult to verify externally without examining the details of the generated model or code.

The Global Optimization Software Landscape

Although BARON had long been the reference in global optimization, the last few years saw a new wave of commercial global optimization solver releases, led by FICO Xpress (2022) and followed by Gurobi (2023) and Hexaly (2024). On the open-source front, SCIP is a strong offering. These tools are multithreaded, support callbacks and solver controls familiar from MIP, and are built around outer-approximation frameworks that leverage the full power of modern MIP engines. (See [7] for more details on how such an implementation works in practice and how the separate components come together.) In addition to the proliferation of new implementations, the performance of existing ones has improved substantially: FICO Xpress Global has achieved a 4.5x speedup since its initial release and a 17.8x speedup on hard instances.

The Future Is Nonlinear

So, are we there yet? Can nonlinear global optimization solvers be used just as if they were mixed-integer linear optimization solvers? We believe the answer is a resounding yes. The technology is mature. The tools are available. The number of possible applications is enormous. It is time for practitioners to revisit their optimization models and ask a simple question: Which of my linear approximations can now be replaced
by reality?

References

1. A. Novikov, N. Vũ, M. Eisenberger, E. Dupont, P.-S. Huang, A. Z. Wagner, S. Shirobokov, et al., 2025, “AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery”, arXiv preprint arXiv:2506.13131.
2. B. Georgiev, J. Gómez-Serrano, T. Tao and A. Z. Wagner, 2025, “Mathematical Exploration and Discovery at Scale,” arXiv preprint arXiv:2511.02864.
3. T. Berthold, 2025, “FICO Xpress Optimization Surpasses AlphaEvolve’s Achievements,” FICO Blog, June 13, https://www.fico.com/blogs/best-global-optimization-solver.
4. T. Berthold, D. Kamp, G. Mexi, S. Pokutta and I. Pólik, 2026, “Global Optimization for Combinatorial Geometry Problems Revisited in the Era of LLMs,” arXiv preprint arXiv:2601.05943.
5. E. Friedman, 2026, “Erich’s Packing Center,” https://erich-friedman.github.io/packing/.
6. I. Pólik and T. Terlaky, 2007, “A Survey of the S-Lemma,” SIAM Review, Vol. 49, No. 3, pp. 371-418.
7. P. Belotti, T. Berthold, T. Gally, L. Gottwald and I. Pólik, 2025, “Solving MINLPs to global optimality with FICO® Xpress global,” Optimization, pp. 1-19, https://doi.org/10.1080/02331934.2025.2595437.