Procedures, not prompts

The first version of Pazau Agent, in 2022, ran on a system prompt. By the time we retired that architecture, the prompt for our largest customer was about 14,000 tokens long. It had been edited by eleven people. It contained the sentence "IMPORTANT: never do this unless the customer is a Gold member (see above)", and nobody could find the "above" it referred to.

It worked, mostly. It also failed in ways nobody could predict, explain after the fact, or fix without breaking something else. This post is about why we moved Pazau Agent from prompts to Procedures, and what we learned about getting language models to follow a business process reliably.

The problem with one big prompt

A system prompt asks the model to do two jobs at once: understand the customer, and reconstruct your operating manual from a wall of text on every single turn. The second job is where things go wrong.

We saw four recurring failure modes:

• Instruction interference. A rule written for returns ("always offer store credit first") leaked into warranty claims, where it was wrong and sometimes illegal.
• Position effects. Rules near the end of long prompts were followed less reliably than rules near the start. Teams learned to fix bugs by moving paragraphs around.
• No notion of state. The prompt couldn't say "you are on step 3 of 5". The model had to infer progress from the conversation, and sometimes skipped ahead, or started over.
• Unreviewable changes. Editing the prompt was a text diff with unbounded blast radius. A one-word change to the refund section could shift behavior on billing disputes.

That last one mattered most to our customers. A support operations lead at an insurer can't sign off on a change if nobody can tell them what it affects.

What a Procedure is

A Procedure is a standard operating procedure written in plain language, the way you'd write it for a new hire, with a small amount of structure. Each Procedure has:

• A trigger: the customer intents it handles.
• Steps: an ordered list of things to do, each in natural language.
• Conditions: branches written as plain "if" statements.
• Actions: explicit calls into your systems, referenced by name, with limits.
• Checkpoints: points where the agent must stop and get confirmation, from the customer, a human approver, or both.

Here's a simplified version of a real one, from a retail customer:

Procedure: Damaged item on delivery
Trigger: customer reports an item arrived damaged or broken

1. Find the order. Use @lookup_order with the order number or the
   customer's email. If there are several recent orders, ask which one.
2. Ask for one photo of the damage. If the customer can't provide one,
   continue anyway but note "no photo" on the case.
3. If the order was delivered more than 30 days ago:
     explain the 30-day window and hand off to the Returns team.
4. If the item is marked "final sale":
     offer a 20% goodwill credit instead of a replacement.
5. Offer a replacement or a full refund. Let the customer choose.
6. CHECKPOINT: confirm the choice and the amount with the customer.
7. Replacement → @create_replacement_order (ship method: expedited)
   Refund     → @issue_refund (limit: $400; above that, require approval)
8. Tell the customer what happens next and when.

Nothing here requires an engineer. The support lead who owns this process can read it, edit it, and understand the consequences. That's the point.

Why this works better

At runtime, Pazau Agent doesn't see every Procedure at once. It classifies the conversation, selects the matching Procedure, and executes it as a state machine where each step is a small, focused model call. The model always knows which step it's on, what the step requires, and what it's allowed to do next.

We compared the two architectures on the same 40,000 historical conversations across six customers, graded by human QA reviewers using each customer's own scorecard:

	System prompt (2023)	Procedures (2025)
Correct outcome	71.4%	89.2%
Skipped a required step	8.9%	0.7%
Took an action outside policy	1.6%	0.04%
Handed off when it should have	83%	97%
Median tokens per turn	16,800	2,900
p95 latency per turn	4.1s	1.7s

A few of these deserve comment.

Skipped steps fell by more than 10×. Once the agent is explicitly on step 4, it can't jump to step 7. When a customer tries to skip ahead ("just refund me"), the agent completes the required steps first, politely.

Out-of-policy actions nearly disappeared. Actions are declared inside Procedures, with limits enforced in code, not in prose. A refund Action with a $400 limit can't issue $401, regardless of what the model generates. The model can request the action; the platform decides whether it runs.

Costs and latency both went down. Smaller, focused context per step means fewer tokens and faster responses. This was a surprise to some people on the team, who expected the extra structure to cost us speed.

Natural language is the right interface

We considered, and rejected, a visual flow builder. We've all seen them: boxes and arrows that start clean and end up looking like a subway map. Flow builders force you to anticipate every branch in advance, and real customers don't cooperate.

Natural-language steps give you the best of both. The structure (ordering, checkpoints, Action limits) is explicit and enforced. The content of each step is flexible enough for the model to handle the thousand variations of "my package is broken" without a branch for each.

The test we apply: could your best human agent follow this Procedure on their first day? If yes, Pazau Agent can too. If not, the Procedure needs work, and it would have needed work for a human as well.

This also changes who owns the agent. With prompts, the owner was whichever engineer understood the prompt. With Procedures, the owner is the person who owns the process: a CX operations manager, a claims lead, a compliance officer. They can change a Procedure in an afternoon.

Checkpoints are where trust comes from

The single most important primitive in a Procedure is the checkpoint. It's a step where the agent must stop and get explicit confirmation before continuing.

There are three kinds:

1. Customer confirmation. "I'll refund $86.40 to the card ending 4417. Shall I go ahead?" The agent can't proceed without a clear yes.
2. Human approval. For actions above a threshold, or in sensitive categories, the agent drafts the action and a human approves it in Pazau Inbox. The customer is told it's being reviewed.
3. Policy gates. Automatic checks that must pass, such as identity verification before any account change, or a sanctions-list check before a payout.

Checkpoints are what let a regulated brand turn the agent on for real work. Our insurance customers use them on every claims-related Action. Our financial services customers put a policy gate in front of anything that touches a card.

Every step leaves a trace

Because execution is structured, every decision is recorded in a form a human can read: which Procedure was selected and why, which step the agent was on, what data it looked at, which Actions it requested, and whether a checkpoint passed. We call these Traces, and they're exportable.

When something goes wrong, you don't guess. You open the trace, see that step 3 misread the delivery date because the order system returned it in a different timezone, and fix the step, or the Action. The fix is local and reviewable.

What's still hard

Procedures are not a solved problem. Three areas we're actively working on:

• Procedure selection under ambiguity. "I want to cancel" might mean the order, the subscription, or the account. We now ask a clarifying question when classifier confidence is below a per-customer threshold, but tuning that threshold is still more art than we'd like.
• Cross-procedure handoffs. A conversation that starts as a delivery question and becomes a refund needs to move between Procedures cleanly, carrying state.
• Knowing a change is safe. When someone edits step 4, how do you know it didn't break anything? Today we recommend a sandbox review. We think the answer is automated simulation against historical conversations before a change goes live. We've since written about how we built exactly that in Evaluating agents with simulation.

The short version

A prompt is a suggestion. A Procedure is an operating manual with guardrails. If you want an AI agent to do real work in a regulated business, such as issuing refunds, filing claims or changing bookings, you need the second one.

You can see how Procedures, Actions and Traces fit together on the platform overview, or see Procedures running live in Pazau Agent.