Skip to content
Muhammet Şafak
tr

Book III of III The Unreliable Systems Trilogy

Engineering Reliable AI Agents

Tools, authority, state, bounds and records for agents in production

A practical guide to building AI agents that can be trusted in production: tools, authority, state, bounds and records, from design to incident review.

  • 487 pages
  • 1st edition
  • Free

The book's thesis

Not

trusting the model to be right

But

making its mistakes survivable

  • 01

    The unreliable decision-maker

    Assume the model will sometimes be wrong, and design for that from the first line.

  • 02

    The control system

    Bound what the agent can do with permissions, checks and hard limits.

  • 03

    The evidence trail

    Record every decision and action so it can be reviewed, measured and investigated when it goes wrong.

About the book

One run called the same lookup 40 times in a row; nobody noticed until a person happened to watch it. The scene is set at Parcelport, a fictional mid-sized fulfillment company, in the week its first agent went live.

By Wednesday the team had learned three things. Besides the lookup loop, one ticket produced a reply that promised a delivery date no tool had returned, and one customer email contained a paragraph addressed to “the assistant”, which the run treated as an instruction from Parcelport. None was a catastrophe, all were predictable, and none could be fully reconstructed, because the only record of what the agent did was its final answer.

The book’s thesis is one sentence: AI agents should be engineered as unreliable decision-makers operating inside reliable control systems. The model proposes and the system decides, so the book starts not with a better prompt but with a question: if the model will sometimes be wrong, what must the surrounding system guarantee so that being wrong is survivable?

It is not a prompt engineering book, nor one about training, a framework or a vendor, and it is not a book of attacks. Every control comes with its cost, and the book says when a control is theater. The numbers in it are illustrations, not recommendations or measurements.

Contents

  1. 01 Preface Sets out the book's thesis: AI agents are unreliable decision-makers that should operate inside reliable control systems. Also covers who the book is for, the framework-free approach and the pseudocode notation.
  2. 02 The Agent Is Not the System Defines an agent as an orchestrator loop, a model, a gate and tools, and the model as a proposal generator whose output is untrusted until the gate allows it. Introduces the six-part control envelope, decision rights and the rule of least agency.
  3. 03 Model Uncertainty Explains why a model's proposal is untrusted input and names four kinds of uncertainty: format, content, intent and omission. Stated confidence is not a control signal; the signals worth building on are computed outside the model.
  4. 04 A Failure Taxonomy for Agents Gives the vocabulary for what goes wrong in a whole run: nine failure classes, a root cause axis and severity. The model's error is the trigger, while the root cause is the control, context or tool that let a wrong proposal matter.
  5. 05 Determinism Boundaries Shows how to draw a determinism boundary per decision, weighing reversibility, verifiability, frequency, tolerance and explainability. The model proposes and code disposes; money, dates and thresholds are computed by code.
  6. 06 Tool Calling Follows the path along which a proposal becomes an effect, from propose to return, and why every station except the first is deterministic code. Covers strict parsing, correction turns, few single-effect tools and treating observations as untrusted data.
  7. 07 Tool Contracts Distinguishes a schema, which says arguments are well formed, from a contract, which says the call may happen in the world as it is now. Covers effect classes, idempotency keys assigned by the runtime and preconditions enforced by the gate.
  8. 08 Permissions and Delegated Authority Places authority in the credential and the code that uses it, not in the prompt. Effective authority is the intersection of the agent's identity, the delegation and the task scope, enforced by a policy engine outside the model.
  9. 09 MCP: Tools Across a Trust Boundary Treats connecting an MCP server as opening a trust boundary. The host owns each tool's contract, a server allowlist and pinned definitions make changes visible, and every MCP call goes through the same gate as any other.
  10. 10 Context Management Treats context as the output of deterministic assembly code that selects, labels, bounds and records items. Covers provenance and trust levels, a written budget with a pinned core, summaries as derived items and retrieval with a relevance floor.
  11. 11 State Puts run state in a durable store owned by the orchestrator, with the model's context as a view rendered from it. Covers state machines, resume from state, deadlines on every wait and concurrency controls at several levels.
  12. 12 Memory Distinguishes memory kept across runs from run state, and asks first whether memory is needed at all. Every memory write is a proposal that code validates, scopes and expires, and memory never carries authority.
  13. 13 Planning and Loops Has the orchestrator drive the loop while the model only supplies the next proposal. Covers plans as proposals, success as a predicate in code and layered stop conditions that include step limits and progress detection.
  14. 14 Budgets and Circuit Breakers Defines a budget as a limit with a unit, a place where it is measured and a defined behavior when exhausted. Covers layered budgets, the effect budget, planning what happens when a budget runs out and circuit breakers that count only dependency failures.
  15. 15 Retries and Fallbacks Classifies a failure before repeating anything: a write that times out has an unknown outcome, exactly one layer retries, and re-asking the model is a new sample, not a repair. A fallback ladder keeps as much primary behavior as each rung allows.
  16. 16 Human Approval Makes approval a policy decision taken by code, never from the model's stated confidence. An approval is bound to the hash of one exact call, and the service that performs the effect must check for a valid approval itself.
  17. 17 Prompt Injection Handles prompt injection outside the model, in the structure of the context and at the gate. Damage needs untrusted content, reach and a channel out in the same run, and the design cuts at least one of the three for each tool.
  18. 18 PII Follows personal data through eight places in an agent, with a control at each one. The cheapest PII to protect is the one the model never saw; tokens, redaction at write time and an outbound check do the rest.
  19. 19 Secrets Keeps secrets out of the model's context: the tool runtime injects the credential and tool schemas are closed to credential fields. Prefers short-lived, narrow, run-bound credentials, scrubs at the boundary and gates pull requests with a secret scan.
  20. 20 Sandboxing Treats running model-produced code as a separate trust boundary. A temporary directory or a cleared environment is not a sandbox; isolation is chosen by who wrote the code, and the orchestrator, not the model, sets the profile and limits.
  21. 21 Observability and Tracing Treats the run as a trace and every step, model call, gate check, tool call and approval wait as a span. The orchestrator writes the record from deterministic events; content is tiered and redacted at write time.
  22. 22 Evaluation Separates evals, which measure the probabilistic parts, from tests, which check the deterministic ones. Covers four eval levels, a golden set built from real traffic and incidents, deterministic graders first and results reported as rates.
  23. 23 Cost Control Covers the spend that stays under every limit and is still wrong. Record raw usage on every span, manage to cost per resolved case including human minutes, and apply cost controls only with evidence that quality held.
  24. 24 Incident Response Defines an agent incident and why it is harder than a service incident: runs cannot be reproduced and behavior may have moved with an unlisted change. Covers a containment ladder written in advance, impact analysis and review that ranks causes.
  25. 25 A Reliability Checklist and Closing Thoughts Gathers the book into a reliability checklist grouped by the control envelope, where each item is answered with an artifact you can open and is tied to what makes it stale. Moves from prototype to production by evidence, per capability.
  26. 26 Appendix: Glossary, Gate Verdict Reasons, Tool Gateway Error Codes, Pattern Quick Reference, Tool Contract Template, Failure Taxonomy Table, Pre-Launch Checklist, Agent Incident Review Template Reference material: a glossary, gate verdict reasons, tool gateway error codes, a pattern quick reference, a tool contract template, the failure taxonomy table, a pre-launch checklist and an incident review template.

Sample chapter: Permissions and Delegated Authority

An excerpt from one chapter; the full text is in the downloadable edition.

The gate and authority

Now put it in the gate. Chapter 4’s tool-call branch checks the model’s proposal against a choice set and refuses with not_in_choice_set, and the contracts of Chapters 5 and 6 list “the order belongs to the customer of this run” as a precondition. The choice set is an earlier, separate layer: it narrows what the model is offered, it is checked against what the model proposed, and its refusal keeps its own code. I would not implement the ownership sentence twice more, though. The ownership decision has one owner, checkAuthority below, and one code, not_permitted, and the contract’s precondition is satisfied by calling it. A proposal can fail the choice set and never reach this function, and that is fine. What matters is that when an order outside the delegation does reach the gate, the verdict about ownership comes from here, with one observation text and one count in the record. It also runs before preconditions that read more data, because there is no reason to read an order the principal may not see.

function checkAuthority(run, call):
    delegation = run.delegation                       // assigned at run creation
    if delegation.expired or delegation.revoked:
        return { kind: "refuse", reason: "not_permitted" }    // an expired delegation is no authority
    authority = intersect(grantsFor(run.agent, call.tool),
                          delegation.grantsFor(call.tool),
                          run.taskScope)
    if authority is empty:
        return { kind: "refuse", reason: "not_permitted" }
    for each ref in resourceRefs(call):               // order id, shipment id
        owner = ownerOf(ref)                          // read from the owning service
        if owner is unknown:
            return { kind: "refuse", reason: "upstream_unavailable" }   // cannot verify: refuse, retryable
        if owner != delegation.on_behalf_of:
            return { kind: "refuse", reason: "not_permitted" }
    return evaluatePolicies(run, call, authority)     // ceiling, window, environment

The block is correct under the failure it is about. The reasons come from the gate’s vocabulary in Chapter 5 (the Appendix lists it), not from a vocabulary of this chapter’s own, and checkAuthority is a partial view of the gate in Chapter 5: it is the permitted step, and an approval outcome it returns is acted on last, after the preconditions (Chapter 6). When the owner lookup fails, the call is refused with upstream_unavailable, which is retryable: an unreachable orders-service must not turn into permission, and it must not turn into a permanent denial either, so the retry runs under the retry policy (Chapter 14) and not at the model’s request. When the delegation has expired because the ticket was closed while the run was suspended, the call is refused with not_permitted. When the model proposes an order that is not the customer’s, the call is refused the same way, without the model ever being asked whether it is sure.

The refusal text matters. The observation returned to the model should say that the order is not available for this ticket, and nothing more. It should not confirm that the order exists, and it should not include any field of it. A refusal that echoes the other customer’s data is a read that happened by another route.

Authority is re-checked after waiting

A suspended run may sit in a queue for minutes or hours before Sam approves it. In the meantime the ticket may have been reassigned, the delegation revoked, or the order refunded by someone else. If the orchestrator resumes the run and applies the call on the strength of the earlier verdict, it acts on stale context (F5, runaway loop or stale context). I would re-run checkAuthority and the preconditions when the run resumes, and treat an approval as one input to that second check, never a replacement for it. The approval says a human accepted the exact call. It does not say the world is still as it was.

Glossary

The key terms used in the book, with short definitions.

agent
A component: an orchestrator loop that calls a model, asks a gate to check its proposals, and applies approved effects through tools. Not a person.
control envelope
The six-part model of this book: Context, Proposal, Gate, Effect, Bound, Record.
determinism boundary
A line, drawn per decision, between what the model may propose and what code must decide.
gate
The deterministic check between a proposal and an effect, with three outcomes: allow, refuse, needs approval. In pseudocode: `"allow"`, `"refuse"`, `"needs_approval"`.
proposal
Anything the model returns: a tool call, a plan, a final answer. Treated as untrusted input.
effect
What happens in the world after the gate allows a tool call.
tool contract
The written, machine-checkable description of a tool: name, description, input schema, effect class, preconditions, idempotency, permissions, timeouts, error shape.
delegated authority
Authority a principal holds on behalf of another, always narrower than the delegator's own and tied to a task.
confused deputy
A component that uses its own authority to do what a less-privileged party asked.
prompt injection
Content that is data being treated as instruction. Direct: the user's own input tries to override instructions. Indirect: the content arrives through a tool result, a document or an email.
kill switch
The manual end of the agent-level breaker: no new runs, in-flight runs wait at the next step boundary. Kept outside the agent.
effect budget
A budget on the cumulative size of effects, such as the total refunded. It closes the gap between a per-call ceiling and the sum.
degraded mode
A stated, reduced behavior (answers without tools, or hand-off only) that the system enters when a bound or breaker trips.

Frequently asked

8 questions

  • Who is this book for?

    Backend and platform engineers, SREs, tech leads and the teams that take an agent into production, including people asked to review a design they did not make.

  • Does it depend on a particular model, framework or vendor?

    No. The book treats the model as a component you call and works the same way when you replace it. Real products and open protocols appear only as illustrations; the pattern is the point.

  • Is it a prompt engineering book?

    No. Prompts appear because they lower the rate of bad proposals, but a prompt is a request, and no property in the book rests on a request alone. Wherever a chapter relies on an instruction, it names the deterministic control that holds when the instruction does not.

  • What do I need to know before reading it?

    You are expected to be a competent software engineer. The book does not explain what a language model is and uses machine-learning vocabulary only where it changes a control decision. You do not need to have used any particular framework, and the book names none as a recommendation.

  • Is the book free?

    Yes. The license line reads "© 2026 Muhammet Şafak — free to download and read". PDF and EPUB files are offered in the download panel on this page. This edition is 487 pages.

  • Which languages and formats is it available in?

    Seven languages: English, Turkish, Spanish, Simplified Chinese, Portuguese, German and French. Each is offered as a PDF and an EPUB.

  • Does the book contain code examples?

    It contains pseudocode. All code is language-independent, written in plain text blocks, and diagrams use Mermaid. The book never shows a provider API or an SDK; the model call is always one abstract function, model.propose(context), that returns a proposal.

  • Is this a standalone book or part of a series, and in what order should I read the series?

    It is the third of three books in The Unreliable Systems Trilogy: I Access (The Production Database Access Guide), II Events (Designing Reliable Event-Driven Systems) and III Agents (this book). The preface asks for general engineering skill, not for another book, so you can start here if an agent is the problem in front of you.

Blog posts that touch on this book's subject.

Search the site

Start typing to search posts, projects and pages.

Esc to close Powered by Pagefind