Agents need an operating contract

Delegate an action, not an ambition

An agent changes a customer order. The transaction commits, but the response disappears. Retrying may duplicate the change; declaring failure may mislead the customer. This illustrative case captures the production challenge: flexible reasoning must operate inside a service that knows what happened and who may act next. The executive decision is how much authority to delegate, with what completion evidence and recovery capacity.

Anthropic's engineering guidance distinguishes predefined workflows from dynamically directed agents and describes the associated cost and latency trade-offs. This is vendor practice, not independent proof of business impact. ¹ mayden's recommendation is to preserve deterministic steps wherever the business rule is stable and use model judgment where the path depends on information encountered during execution. An agent can interpret a request while an ordinary service enforces the allowed transaction.

A research assistant may read approved documents and recommend an action. A scheduling assistant may move a meeting within agreed limits. A procurement assistant may prepare an order while a budget owner approves it. These are illustrative designs, not observed mayden implementations. Choose the least discretion that can deliver the intended outcome; additional freedom carries a cost of evidence, supervision and recovery.

Verify the transaction behind the answer

A fluent message saying a task is complete is weak evidence of completion. Business execution should be verified against the relevant state: a booking exists with the required constraints, an approved record changed once, or a draft remains unsubmitted. The original τ-bench evaluates tool-and-user interactions against annotated end states and tests consistency over repeated trials. Its 2024 performance findings describe those models and simulations, not current production capability. ² Its methodological contribution remains useful.

Define acceptable and unacceptable end states before building the agent. Include partially successful outcomes. If a customer asks to change an order and receive a confirmation, changing the order without sending the confirmation is a different failure from sending a confirmation without changing the order. Both are different from changing the wrong order. A single completion flag erases these distinctions and directs engineering effort toward an incomplete view of the service.

Operators need a compact receipt after each consequential action. It should identify the object changed, the actor, the authorization used and the observed result. Preserve uncertainty when the downstream system times out. The correct state may be outcome unknown, requiring reconciliation. A second blind attempt can turn a transient communication problem into a duplicate transaction. The service should be able to inspect whether the first action committed before choosing a recovery path.

Count the work after apparent success

Measure an agent across repeated runs, interruptions and recoveries. A high first-attempt success rate can coexist with unstable behavior on particular customers or long task paths. τ-bench's repeated-trial framing makes that distinction explicit. ² The resulting evaluation should retain case-level variability. Some failures may be concentrated in a narrow, identifiable class that can be routed differently; other failures may be broadly unpredictable. These two patterns call for different operating decisions.

Long-horizon research finds that execution behavior can worsen as task paths lengthen, while more capable reasoning approaches can mitigate some of the observed effects. The experiments also challenge a simplistic assumption that every step has the same independent chance of failure. ³ The enterprise should measure its own path-length and recovery distributions. A mathematical curve based on identical independent steps can illustrate sensitivity, but it should never be presented as an observed reliability forecast.

The hidden workload often sits after nominal completion: reconciling a state, correcting a document, contacting a customer or reconstructing an event. Include this work in cost per accepted outcome. Assign a recovery category and owner when the system fails. Track whether the same operator problem reappears after releases. Measure recurring intervention explicitly in the operations budget and test whether it falls as the service improves.

Enforce the execution envelope outside the model

mayden proposes an execution envelope with five elements: identity, action scope, state checks, resource limits and recovery. This is an original design framework derived from established security and reliability principles. Identity binds the run to the relevant user or service. Action scope constrains the objects and operations available. State checks verify preconditions and results. Resource limits bound time, spend and repeated actions. Recovery determines what the system does when confidence or information becomes insufficient.

OWASP identifies excessive functionality, permissions and autonomy as separate sources of damaging agent behavior. Its recommendations include enforcing authorization downstream of the model. ⁴ For management, this means a prompt telling an agent to respect policy should be accompanied by actual restrictions at the point of action. Removing a dangerous operation from the tool interface can be more dependable than relying on the model to decline it under every possible input.

NIST's Zero Trust Architecture centers access decisions on resources and identities rather than assuming trust from location. ⁵ Applied here as an engineering principle, each tool action should be authorized in its own context. A research run's permission to inspect a supplier file does not imply permission to send that file externally. These controls constrain the consequences of a reasoning failure. They do not make the reasoning correct or remove the need to test the service.

NIST's September 2026 summary of public comments on agent identity and authorization highlights the need to preserve authority through delegation chains and the concern over making probabilistic models the sole authorization mechanism. ⁸ This is a record of stakeholder feedback, not a completed standard. It reinforces mayden's design position: the agent may propose an action, but an external control should decide whether that specific action is permitted for that principal, resource and context.

Keep source content outside the authority boundary

Agents encounter documents, email, search results and tool responses that can contain hostile instructions. AgentDojo provides an extensible evaluation environment for testing attacks through untrusted tool data. Its task and attack collection demonstrates a way to examine the combined system; it does not estimate the prevalence of attacks in a particular company. ⁶ The design implication is to separate what a source says from what the user has authorized.

A supplier document may legitimately contain a bank account or a request to change a delivery address. The agent must distinguish recording this information from executing the requested change. It should present a proposal with a source reference and obtain whatever independent verification the business process requires. A trusted internal storage location does not make every embedded instruction authoritative, particularly when documents originate outside the organization or are editable by a large group.

Test the complete route from intake to side effect. Include instructions embedded in retrieved content, conflicting tool outputs and apparent requests from other agents. Evaluate whether the final service preserves its authority boundary even when the model follows a misleading instruction. Record harmless refusals and blocked legitimate work as well as successful attacks. Controls that stop the service from accomplishing its ordinary purpose create pressure for bypasses, making operational usability part of the security design.

Buy discretion only where it earns its cost

A tightly constrained agent can become an expensive way to run a workflow. Every approval, check and handoff adds latency. For stable processes with clear rules, this objection may be decisive. AI Agents That Matter calls for joint attention to cost and accuracy and warns that benchmark practices can reward unnecessary complexity. ⁷ The company should require an agent to outperform simpler alternatives under the same definition of accepted work, with the chosen level of autonomy justified by that comparison.

There are also situations in which restricting discretion prevents useful discovery. An analyst working with public information may benefit from exploring sources that were not anticipated. The correct response is to distinguish exploration from commitment. Permit broader inquiry inside a bounded environment, while retaining narrower permissions for changes to company systems and communication outside the organization. This separation can preserve creativity without transferring every exploratory judgment into an irreversible business action.

Review capacity belongs in the autonomy decision. Give a reviewer the source evidence, proposed change and consequences together. Measure waiting time and the frequency of changes or rejections. A queue in which reviewers cannot inspect consequential proposals does not provide the intended control. Reduce scope or redesign the handoff before adding traffic.

The order changed; the confirmation did not

Consider a fictional distributor whose customers ask to amend delivery dates and quantities. The proposed agent can interpret the request, inspect order status and prepare a change. The order service remains responsible for checking inventory, customer authority and whether dispatch has started. A validated change request receives a unique operation identifier. These are illustrative design choices; they are not a description of a tested mayden product or a claim of guaranteed reliability.

Exhibit 01 follows the difficult case: the order service commits a change but its response never reaches the agent. The result is unknown, not failed. Query authoritative order state using the unique operation identifier before retrying; confirmed non-commitment may permit a bounded retry, while a confirmed change must not be repeated. If only part of the request completed, reconcile each step and recover or escalate the remainder. A separate notification failure is a communication problem: retrying a permitted message must not replay the order mutation. Until verification succeeds, tell the customer that the change is pending verification rather than asserting success or failure. If the service cannot distinguish a missing result from an operation still in flight, keep the state unknown and escalate. A lookup miss alone cannot authorise replay.

The design has costs. Unique-operation handling may require changes to a legacy system. A human queue needs staffing during peak periods. A narrow tool can prevent legitimate amendments that fall outside its schema. Management should compare these costs with the current service, including manual errors and delays. If the legacy system cannot support safe execution, the agent can still prepare a structured proposal for an operator. This narrower role may deliver useful value while the transaction layer is improved.

Expansion should follow evidence from the actual workflow. First test interpretation and proposal quality. Then test verified execution within a limited class of reversible amendments. Include dispatch races, concurrent staff edits, revoked customer authority and repeated delivery of the same request. Examine the full outcome and operator work. The successful experiment authorizes only the tested operating envelope; adding cancellation, refunds or a new warehouse changes the consequences and requires a separate release judgment.

Expand authority one permission at a time

Select one action class whose result can be checked: preparing an order, amending a reversible field or sending an approved notification. Define the permitted objects, limits and end states. Then interrupt the workflow at each consequential transition. Operators must be able to distinguish completed, failed and unknown outcomes and exercise the appropriate recovery. An untested rollback diagram is not release evidence.

Expand users, tools and financial limits through separate decisions. Each changes a different exposure. Retain the policy, model, tool and evaluation versions behind the accepted baseline, and compare the next configuration on repeated completion and intervention cost. If a new permission adds supervision faster than useful work, retain the narrower service.

The CEO should approve an operating contract for each agent: the work delegated, the actions permitted, the evidence of completion and the owner of uncertain cases. This makes autonomy an economic and operational choice. Scope can grow when the service earns it, and contract when the recovery burden outweighs the benefit.