November 17, 2026 in Artificial Intelligence

Enterprise AI is Overengineered

Why Bigger Models Are Not Always Better

SHARE: PRINT ARTICLE:print this page https://doi.org/10.1287/LYTX.2026.03.24

Gartner predicts that more than 40% of agentic AI projects could be canceled by the end of 2027 because of escalating costs, unclear business value, or inadequate risk controls. Notably, model capability is not among the reasons.

That distinction points to a shift in the enterprise AI conversation. For the past several years, organizations have focused primarily on capability: Can artificial intelligence perform the task? Can a large language model (LLM) summarize the document, generate the code, analyze the data, or automate the workflow? The next question is more difficult: Should it?

Generative AI has expanded what machines can do, but enterprises do not need a probabilistic, compute-intensive model for every problem. As AI moves into production, architecture selection is becoming an economic and operational decision. Reliability, predictability, latency, verification, governance, and cost all matter alongside raw capability.

For many workloads, the most advanced model may not be the most advanced solution. Deterministic architectures, specialized models, traditional optimization methods, and carefully designed hybrid systems can outperform generative AI when the business requirement is a controlled, repeatable outcome. The next phase of enterprise AI will therefore be defined less by how much AI organizations deploy and more by how precisely they match technology to the problem.

The Cost of AI Is Bigger Than Compute

The economics of generative AI are often discussed in terms of graphics processing units (GPUs), cloud infrastructure, tokens, and inference costs. Those expenses matter, particularly as demand for AI infrastructure accelerates, but they represent only part of the equation. An enterprise must also account for the cost of making probabilistic outputs acceptable in production.

A generative model can produce a useful answer and then produce a different answer when presented with the same or nearly identical input. In a creative environment, that variability may be desirable. In an operational workflow involving financial calculations, inventory allocation, compliance decisions, manufacturing processes, or other repeatable activities, variability can create another layer of work.

Outputs may need to be verified. Exceptions must be handled. Performance needs to be monitored. Access must be governed. Decisions may require documentation and audit trails. Human review may remain necessary for high-risk actions. The resulting cost is not simply cost per inference. A more useful enterprise metric may be cost per verified business outcome.

Governance illustrates the issue. IBM’s 2026 Cost of a Data Breach research found that 68% of breached organizations lacked AI governance to manage artificial intelligence or detect shadow AI, up from 63% the year before. AI systems create operational requirements that extend beyond the model itself.

As AI systems become more autonomous and interact with more data, applications, and workflows, those requirements increase. The challenge is not simply whether AI can perform an action, but whether the enterprise can reliably govern, verify, and measure that action at scale. The larger question is when probabilistic intelligence creates enough value to justify the additional complexity around it.

When Predictability Is a Feature, Not a Limitation

Generative AI is powerful precisely because it is probabilistic. It can work with ambiguity, interpret natural language, synthesize unstructured information, and generate novel responses. Those capabilities make it exceptionally useful for tasks such as summarization, research, drafting, discovery, and conversational interfaces. But not every enterprise problem benefits from novelty.

Consider a simple mathematical relationship. If a problem can be solved reliably using a known equation with two parameters, building a neural network with millions or billions of parameters to produce the same answer would add complexity without adding meaningful value. Enterprise systems encounter variations of that problem constantly.

A pricing rule may require a defined calculation. A scheduling problem may have known constraints and an optimization objective. A workflow may require the same action whenever a specific set of conditions occurs. A compliance process may need to apply clearly documented business rules. A data transformation may need identical inputs to produce identical outputs.

This is really a question of AI-as-action versus AI-as-advisor: does AI make the call, or does it inform one that a human or a deterministic system should make? The higher the stakes, the more that distinction should drive the architecture decisions.

In these environments, determinism is an architectural advantage.

A deterministic system makes it easier to explain why an outcome occurred, reproduce the result, test behavior, estimate costs, and predict performance at scale. These characteristics become especially valuable in environments where an incorrect result carries a financial, regulatory, safety, or operational consequence.

There is also a reliability issue when probabilistic systems are chained together. An individual AI component may perform well in isolation, yet uncertainty can accumulate as one model’s output becomes another model’s input. In complex agentic workflows, a small error early in the process can influence every decision that follows.

Traditional systems are generally easier to observe because a healthy system following validated rules should behave predictably. A probabilistic AI system can remain technically available while producing an answer that is inaccurate, inappropriate, or inconsistent with the intended objective. That distinction changes what “reliability” means in an AI environment. Infrastructure uptime alone is no longer enough.

Identify the Problem Before You Pick a Solution

The more useful enterprise question is not whether deterministic systems are better than generative AI. It is which architecture best fits a specific workload. That requires organizations to classify problems before selecting technology.

Several questions can help guide that decision:

  • Does the output need to be repeatable? If identical inputs should consistently produce the same result, a deterministic or tightly constrained architecture may be preferable.
  • Does variability create value? Generative AI is well-suited to problems where multiple acceptable answers exist. It is less compelling where variation creates verification work.
  • Is there a defined optimization objective? Operations research, mathematical optimization, rules engines, and classical machine learning may solve structured problems more efficiently than a general-purpose language model.
  • What is the cost of being wrong? The higher the consequence of an incorrect result, the more organizations should consider architectures that provide stronger controls, transparency, and reproducibility.
  • How expensive is verification? A fast AI output provides little efficiency if employees must spend significant time confirming that it is correct.

This workload-first approach also changes how enterprises should think about proofs of concept.

Generative AI can be an excellent prototyping technology because it enables organizations to test ideas quickly. But the architecture used to demonstrate that a process can be automated does not necessarily need to become the architecture that runs the process at scale.

Once a workflow is understood, organizations can examine each component and ask whether the large model is still necessary. Some steps may be handled by deterministic logic, conventional automation, specialized machine learning, smaller language models, retrieval systems, or other targeted technologies. 

The Future Is Likely Hybrid

The strongest enterprise architectures will not necessarily choose between deterministic systems and generative AI. They will combine them deliberately. 

A generative model might interpret an unstructured customer request while deterministic logic enforces the pricing or eligibility rules behind it. Elsewhere, an AI agent might recommend an action while a deterministic policy layer decides whether that action is permitted. We’ve seen this play out in practice: routing a contract-review workflow between a deterministic rules engine and a generative extraction step cuts verification time without giving up the flexibility of the intake step.

Model routing can take the same principle further. Instead of sending every request to the largest available model, systems can route simple tasks toward lower-cost methods and reserve more powerful models for workloads that genuinely require their capabilities. This is where enterprise AI efficiency becomes an architectural discipline.

Deloitte’s 2026 State of AI in the Enterprise report found that only 30% of organizations are redesigning key processes around AI. Another 37% are using AI at a surface level with little or no change to underlying business processes. That suggests many organizations are still adding AI to existing workflows rather than reconsidering how those workflows should operate.

The efficiency opportunity is therefore larger than model optimization. Enterprises need to examine the process itself. The goal should not be to maximize the amount of artificial intelligence inside a workflow. It should be to minimize the complexity required to produce a reliable business outcome.

Efficiency Will Become the Next Measure of AI Maturity

The first wave of enterprise generative AI rewarded experimentation. Organizations learned what the technology could do, developed use cases, and demonstrated capabilities that would have seemed unrealistic only a few years earlier. Production changes the standard. At scale, an AI architecture must do more than work. It must work reliably, economically, securely, and repeatedly enough to justify its place inside the business.

Generative AI will remain indispensable for problems that require language, reasoning, ambiguity, and flexibility, while deterministic systems shine where certainty, reproducibility, and control matter most. The territory between them is where specialized models and hybrid architectures will become increasingly valuable.

The most mature organizations will not ask how to apply generative AI to every problem. They will ask what each problem requires and then choose the least complex architecture capable of solving it well. As compute, energy, governance, and operational demands continue to rise, that discipline will become increasingly important. 

 

 

Ben Prescott

SHARE:

INFORMS site uses cookies to store information on your computer. Some are essential to make our site work; Others help us improve the user experience. By using this site, you consent to the placement of these cookies. Please read our Privacy Statement to learn more.