https://fastaiagent.net/blog.html Notes from the harness: replay, guardrails, evaluation, and what it takes to operate agents in production. en Fri, 25 Sep 2026 12:00:00 GMT https://fastaiagent.net/blog/traces-the-flight-recorder/ https://fastaiagent.net/blog/traces-the-flight-recorder/ Fri, 25 Sep 2026 12:00:00 GMT Every run the SDK pushes lands on the plane as a trace — every model call, tool call, retrieval and guardrail verdict, in order, with what it cost. Including the runs that failed, and exactly why. Observability Upendra Bhandari https://fastaiagent.net/blog/online-evaluation-policies/ https://fastaiagent.net/blog/online-evaluation-policies/ Mon, 21 Sep 2026 12:00:00 GMT An offline eval tells you how your agent handles the cases you thought of. Online evaluation policies score what actually happened — sampled from live traffic, on a schedule, with the failures surfaced rather than averaged away. Evaluation Upendra Bhandari https://fastaiagent.net/blog/how-many-answers-has-anyone-read/ https://fastaiagent.net/blog/how-many-answers-has-anyone-read/ Wed, 16 Sep 2026 12:00:00 GMT Most teams rely on an LLM judge to grade the rest, and nobody has checked whether the judge is right. When someone asks who reviewed this, a model's score is not an answer. Evaluation Upendra Bhandari https://fastaiagent.net/blog/which-agents-would-fail-an-eu-ai-act-audit/ https://fastaiagent.net/blog/which-agents-would-fail-an-eu-ai-act-audit/ Sun, 13 Sep 2026 12:00:00 GMT Most teams answer that with a document. This answers it with production evidence — two agents failing the audit, captured automatically, every record hash-chained for the auditor. Compliance Upendra Bhandari https://fastaiagent.net/blog/prove-the-control-was-operating/ https://fastaiagent.net/blog/prove-the-control-was-operating/ Sun, 06 Sep 2026 12:00:00 GMT The question for CISOs and Compliance Officers is no longer whether you have an AI policy. It is whether you can prove a control was operating when the agent made the decision — with a 60-second demo of evidence bound to live runtime. Compliance Upendra Bhandari https://fastaiagent.net/blog/evidence-not-attestation/ https://fastaiagent.net/blog/evidence-not-attestation/ Sun, 06 Sep 2026 12:00:00 GMT An auditor does not want a signed claim that a control exists. They want the record that it ran. Compliance evidence produced by the system rather than asserted about it. Compliance Upendra Bhandari https://fastaiagent.net/blog/guardrails-at-every-checkpoint/ https://fastaiagent.net/blog/guardrails-at-every-checkpoint/ Thu, 27 Aug 2026 12:00:00 GMT It's what comes in, what tools it calls, and what goes out. Three rules created on the Control Plane and blocking live — with zero guardrail code in the agent. Guardrails Upendra Bhandari https://fastaiagent.net/blog/guardrails-block-before-the-provider/ https://fastaiagent.net/blog/guardrails-block-before-the-provider/ Thu, 20 Aug 2026 12:00:00 GMT FastAIAgent guardrails block before the provider ever sees the data. Define one in code, or author it once on the Control Plane and every connected agent enforces it. Guardrails Upendra Bhandari https://fastaiagent.net/blog/where-should-ai-actually-be/ https://fastaiagent.net/blog/where-should-ai-actually-be/ Sat, 15 Aug 2026 12:00:00 GMT Use the simplest thing that reliably does the job, and only let the model decide what happens next when you genuinely cannot know the next step until you have seen the last one. Architecture Upendra Bhandari https://fastaiagent.net/blog/five-people-evaluating-your-agent-platform/ https://fastaiagent.net/blog/five-people-evaluating-your-agent-platform/ Mon, 10 Aug 2026 12:00:00 GMT The developer, the operator, the CTO, compliance and finance each judge an agent platform by a different test. Ship for one and you lose the other four. Platform Upendra Bhandari https://fastaiagent.net/blog/guardrails-that-actually-block/ https://fastaiagent.net/blog/guardrails-that-actually-block/ Sun, 26 Jul 2026 12:00:00 GMT Most guardrails observe and report. A guardrail that cannot stop the action is a log line with good intentions. Policy checked before the agent acts. Guardrails Upendra Bhandari https://fastaiagent.net/blog/agent-framework-is-not-a-harness/ https://fastaiagent.net/blog/agent-framework-is-not-a-harness/ Thu, 16 Jul 2026 12:00:00 GMT A framework helps you build an agent. A harness runs it in production — and the difference is everything you need after the demo works. Harness Upendra Bhandari https://fastaiagent.net/blog/strong-model-reads-balance-sheet-wrong/ https://fastaiagent.net/blog/strong-model-reads-balance-sheet-wrong/ Fri, 10 Jul 2026 12:00:00 GMT A capable model still misreads a financial table. The fix is not a bigger model — it is an eval that finds the failure and a prompt change you can prove worked. Evaluation Upendra Bhandari https://fastaiagent.net/blog/give-your-agent-memory/ https://fastaiagent.net/blog/give-your-agent-memory/ Thu, 02 Jul 2026 12:00:00 GMT Your agent references a customer trip that never happened. It came from memory — a fact recalled from someone else's conversation. Memory as one object, with scopes you can see. Memory Upendra Bhandari https://fastaiagent.net/blog/ship-an-agent-without-evals/ https://fastaiagent.net/blog/ship-an-agent-without-evals/ Wed, 24 Jun 2026 12:00:00 GMT Nobody ships a payment service on three manual clicks, yet that is exactly how most agents reach production. Datasets, scorers and a gate that actually blocks. Evaluation Upendra Bhandari https://fastaiagent.net/blog/your-agent-shouldnt-decide-alone/ https://fastaiagent.net/blog/your-agent-shouldnt-decide-alone/ Thu, 18 Jun 2026 12:00:00 GMT The hard part of human-in-the-loop isn't letting a human approve. It's letting them approve later — hours or days later, on a different machine — without holding a live process hostage the whole time. HITL Upendra Bhandari https://fastaiagent.net/blog/agent-was-right-data-was-wrong/ https://fastaiagent.net/blog/agent-was-right-data-was-wrong/ Thu, 11 Jun 2026 12:00:00 GMT Perfect reasoning over the wrong document is the most dangerous kind of agent bug — no hallucination, no error, green across every dashboard. To find it you have to look at what the agent actually retrieved. Knowledge Base Upendra Bhandari https://fastaiagent.net/blog/see-everything-your-agent-does/ https://fastaiagent.net/blog/see-everything-your-agent-does/ Thu, 04 Jun 2026 12:00:00 GMT Your choices used to be free but heavy, easy but hosted, or self-hosted for an enterprise fee. Nobody offered the fourth option — free, easy and local. One pip install, one command, one SQLite file, nothing leaving your machine. Observability Upendra Bhandari https://fastaiagent.net/blog/your-agent-needs-transactions/ https://fastaiagent.net/blog/your-agent-needs-transactions/ Fri, 29 May 2026 12:00:00 GMT Databases solved this 40 years ago. Every framework gives agents tools to act; none give them transactions to act safely. ACID — atomicity, consistency, isolation, durability — mapped onto action-taking agents. Durability Upendra Bhandari https://fastaiagent.net/blog/prompt-changed-agent-broke-nobody-noticed/ https://fastaiagent.net/blog/prompt-changed-agent-broke-nobody-noticed/ Mon, 25 May 2026 12:00:00 GMT Code changes have PRs, reviews, CI and tests. Prompt changes have someone eyeballing three responses before they deploy. Here is the gate that catches a bad prompt before it ships. Prompts Upendra Bhandari https://fastaiagent.net/blog/every-production-failure-becomes-a-test/ https://fastaiagent.net/blog/every-production-failure-becomes-a-test/ Thu, 21 May 2026 12:00:00 GMT Finding the bug is an afternoon. Keeping it dead across every future change by every future contributor is the real cost — and that only works if the failure becomes a permanent test case the moment you fix it. Replay Upendra Bhandari https://fastaiagent.net/blog/done-complaining-heres-what-im-shipping/ https://fastaiagent.net/blog/done-complaining-heres-what-im-shipping/ Sun, 10 May 2026 12:00:00 GMT Five articles on what's broken in agent infrastructure. This is the one where I stop complaining and ship: FastAIAgent, open source, Apache 2.0, runs locally. Announcement Upendra Bhandari