Your AI Agent Works in the Demo. Here Is What Breaks in Production

During the demonstration, the customer asks a tidy question. The agent finds the right record, chooses the correct tool and produces a helpful answer. Everyone sees the successful path. Nobody sees what happens when the record is missing, the customer changes their mind halfway through, or the external service accepts an action but times out before confirming it.

Production contains all three situations, often before the first busy afternoon ends.

The distance between a working demo and a dependable service is not explained by a dramatic failure percentage. Any headline failure rate needs a traceable study, a clear denominator and a matching definition. Without those, the figure should not guide an investment decision. Your own release criteria can provide more useful evidence.

Decide whether this task needs an agent

Some work benefits from a system that chooses its next action dynamically. Other work already follows a known sequence: validate a form, look up a record, calculate a rule and prepare a response. Making the second kind more autonomous can introduce uncertainty without adding business value.

Anthropic distinguishes predefined workflows from agents that direct their own tool use and recommends starting with simple approaches. That distinction is useful when choosing the smallest architecture that can meet a requirement. Building effective agents.

Suppose a service business wants to confirm appointment availability. A deterministic availability query followed by a carefully constrained response may be enough. An agent becomes more relevant when it must investigate an ambiguous request across several sources. Even then, authority to reserve a slot or change a contract should be separately controlled.

Define completion outside the conversation

An agent saying that it has completed a task is not evidence that the task happened. Define success using the system of record. A booking exists when the booking service returns a confirmed identifier and the stored record matches the intended customer, time and service. A drafted email exists when the draft can be retrieved from the correct mailbox.

Write failure outcomes with the same precision. If inventory cannot be checked, the agent should explain that availability is unconfirmed and route the request appropriately. It should not convert an unavailable tool into a confident guess. If a customer changes a material detail, the system should validate again before acting.

This creates a practical release contract. The test does not ask whether the conversation sounds helpful. It asks whether the correct state changed, whether the wrong state remained protected, and whether the customer received an accurate account of what happened.

Design tools for recoverable actions

A broad tool called manage_customer gives an agent too much room to interpret. Narrow operations with explicit inputs and outputs are easier to validate. Separate reading a customer record, preparing a proposed update and applying an approved change when the consequences justify that separation.

Handle duplicate requests deliberately. Imagine a booking operation succeeds but its response is lost. Retrying blindly could create another booking. Use an operation identifier and a server-side duplicate check so that retrying the same intent returns the existing result rather than repeating the side effect. The relevant guarantee belongs in the service, not solely in the prompt.

Treat partial completion as a real state. If an invoice was created but delivery failed, the recovery action is to retry delivery, not create another invoice. Store enough structured progress to resume safely and give an operator an intelligible account of what remains undone.

Build an evaluation set that can embarrass the system

Clean examples are useful for checking basic functionality. Release decisions also need awkward cases: conflicting records, expired policies, incomplete customer details, multiple people with similar names and a tool returning an error. Include requests the agent must decline or escalate.

Use a held-out set that is not repeatedly copied into prompt tuning. Otherwise, the team can become very good at passing familiar examples while learning little about new ones. Evaluate tool choice, final state, permission boundaries and factual accuracy separately. A friendly response should not compensate for an unauthorized action.

Record the expected result before running the test. Have a domain expert review ambiguous examples and explain the acceptable alternatives. This takes effort, but the resulting collection becomes a reusable business asset: a concrete description of how the service should behave when conditions are imperfect.

Put a person at the right decision point

Human review is useful only when the reviewer has information, time and authority to intervene. A screen that asks an employee to approve a long opaque plan is likely to create a rubber stamp. Show the proposed action, relevant evidence, consequences and the specific uncertainty requiring judgement.

Start with approval before consequential external actions. A business might allow automatic categorization while requiring confirmation before issuing refunds or changing contract terms. The exact boundary depends on impact, reversibility and the organization's obligations. Document it, then enforce it in the tool layer where practical.

Measure the review queue. If every request requires extensive intervention, the pilot may have demonstrated an assisted workflow rather than viable autonomous operation. That can still be useful. Describe it honestly and include the human capacity in the operating model.

Roll out in stages that reveal different failures

First, run offline evaluations on approved sample data. Next, use shadow mode: the system produces proposed actions while the existing process remains authoritative. Compare the proposals with actual outcomes without letting the agent change customer records.

Then enable a small, defined live cohort with clear limits and an available operator. Choose the cohort by risk and relevance, not only by how easy it is to impress. Watch for behaviour that offline tests could not reproduce, including delays, repeated customer messages and changing upstream records.

Expand only after the agreed criteria hold over a meaningful observation period. A single quiet morning does not establish reliability. Keep the previous process available and define the conditions that trigger a pause or rollback. Decide who can make that decision before an incident forces a hurried discussion.

Observe the work without collecting everything

Capture the request identifier, model and prompt version, tools called, latency, cost, final status and escalation reason. These fields help explain whether a failure came from reasoning, retrieval, integration or the underlying business data. Redact or restrict sensitive content rather than assuming complete transcript storage is necessary for every diagnostic question.

Give operators a concise view of incomplete work. An alert should identify what failed, which customer action may have happened and the next safe step. A large pile of raw traces is not an operating procedure. Test whether someone outside the development team can use the runbook to recover a routine failure.

Watch changes in the environment. A supplier can modify an API, a policy can expire and a model version can behave differently. Version critical components and rerun relevant evaluations when they change. Production readiness is maintained through these checks, not awarded permanently after one launch.

Make the go-live decision commercially honest

Consider a hypothetical pilot that saves four minutes on a routine request but adds fifteen minutes of review to each exception. The result depends heavily on the exception rate. Measure that rate on representative traffic before forecasting savings or staffing needs.

Include service reliability, review workload, operating cost and customer outcomes in the release decision. If the result is not good enough, narrow the task or keep an assisted mode while fixing the weak point. There is no commercial prize for calling a fragile workflow autonomous.

A dependable agent is a service with explicit authority, verifiable outcomes and a workable recovery path. The demo proves that success is possible. Your production process must show how success remains repeatable when the easy path disappears.