AI Agents Should Be Mostly Deterministic. Here's Why

During my internship at ANCI AI, I've worked mainly on the brain of our calendar scheduling agent, Zara. A central theme throughout the development process was deciding which parts of the agent should be deterministic and which should be non-deterministic. In this article, I define both terms, share what we learned about them while building Zara, and explain why we are shifting toward a more deterministic design.

01 / Definitions

What determinism and non-determinism mean in an AI agent

A system is deterministic when a specific input always yields the same output, with no randomness involved. In practice this is achieved by writing code that handles predefined use cases. A non-deterministic system, such as a Large Language Model (LLM), can produce different outputs for the same input.

Despite that inherent unpredictability, AI agents depend on LLMs for a reason. Natural language requests are too diverse to handle by hand, and they often carry subtle ambiguity that software alone cannot capture. An LLM can extract structured details from that messy input. It can also select specific tools from a Model Context Protocol (MCP) server to complete the sub-tasks needed to reach a user's goal.

Determinism through code, however, remains essential for implementing guards. Guards keep the agent's workflow controlled and correctable even when the LLM picks the wrong tool or the wrong flow during execution.

02 / What We Tried

Why we are shifting the balance at ANCI AI

Our initial plan put an LLM in charge of the core of the agent's functionality: intent classification, tool selection, and plan revision based on human-in-the-loop feedback. We soon identified two primary limitations.

The only thing we cannot make deterministic is understanding what the user meant. Everything after that, we can.

03 / The Shift

Determinism is the key to fixing this

To address these issues, we are moving toward a primarily deterministic agent, where tools are selected through structured code based on user intentions. The one element we cannot manage deterministically is the structured extraction of those intentions from natural language. For that step we use powerful models and work hard to minimize hallucinations.

Our approach follows a clear sequence:

1. Define the use cases. We start by enumerating the specific scenarios the agent must handle, rather than hoping a model will generalize to all of them.
2. Identify where humans belong. We decide up front which ambiguous scenarios require human intervention instead of a guess.
3. Establish contracts. For each use case we specify exactly which pieces of information must be gathered from the user's prompt.
4. Extract into an Action DSL. The LLM's only job is to fill a custom, structured Action DSL (domain-specific language) that captures the user's intent.
5. Validate, compile, execute. Once the intent is in structured form, deterministic code validates it, compiles it into a plan, and executes the functions needed to fulfill even complex requests.

To make sure the LLM extracts intentions accurately, we thoroughly test a wide range of use cases and tune our zero-shot prompts against real-world scenarios. The model's surface area shrinks to one well-tested job, and everything downstream becomes predictable, inspectable, and cheap to run.

04 / Conclusion

Maximize determinism, minimize the surface for error

Hallucinations and their associated costs are often inevitable when an agent relies solely on LLMs. That is why we have shifted our focus toward maximizing determinism. Through this development process, we have learned that defining exhaustive use case categories and mapping their specific workflows is essential to building a reliable system that successfully fulfills user requests.