Fixing agent reliability before runtime is cheaper because many failure causes are visible in code and can be corrected in a pull request. When an agent fails in production, someone has to notice, reproduce it, read traces and find the cause. Often the cause turns out to be one line — a tool described as "handles orders," an HTTP call with no timeout, or a max_turns that was never set.
Those causes were in the repository the whole time. Software teams learned this decades ago with linters and static analysis. Agents need the same thing, adapted to how agents fail. It is one of the core ideas behind agent reliability: you engineer reliability before deployment as well as watching for it after.
Three recent studies show that tool definitions, system design and missing fault handling drive a large share of agent failures, and all three can be seen in code.
| Study | Finding | Visible in code? |
|---|---|---|
| MCP Tool Descriptions Are Smelly! (Hasan et al.) | 97.1% of 856 MCP tools had a description flaw; rewriting descriptions raised task success by a median of about 6 percentage points across four models (GPT-4.1: 18% to 29%) | Yes, in tool definitions |
| Why Do Multi-Agent LLM Systems Fail? (Cemri et al.) | 44.2% of failures came from system design; nearly 16% of traces showed step repetition | Largely, in config and orchestration |
| ReliabilityBench (Gupta) | Rate limiting was the most damaging injected fault | Yes, as missing timeouts and retries |
Static analysis can catch at least seven common agent reliability gaps before runtime. The examples are real rule IDs from the open Agent Reliability Rules.
| Gap | Example rule | Why it breaks reliability |
|---|---|---|
| Placeholder tool description | CSDK-017 | The model can't choose the right tool consistently |
| Description too short to guide selection | CSDK-018 | Inconsistent tool choice |
| Network call with no timeout | CSDK-003, OAI-005, MCP-004 | One slow dependency stalls or crashes the run |
| No explicit max turns | CSDK-206 | Runaway loops and runaway cost |
| No input guardrails on shell or filesystem tools | OAI-101 | Unbounded harm when the agent is wrong or manipulated |
| Permissions bypassed | CSDK-103 (bypassPermissions) |
Removes the human check entirely |
| Over-broad tool grants | Agent-scope rules | The agent can do more than its purpose requires |
Deterministic analysis returns the same findings on the same code every time. Using an LLM to review an LLM agent brings back the same inconsistency you are trying to remove. Consistent findings make it safe to use as a pull request gate, and your code never has to leave your machine.
Yes. The earliest place to fix a reliability gap is before it is written. Most agent code is now written with help from a coding agent, so Trustabl Forge gives that coding agent the rules up front.
Forge reads your repo's dependency manifests, works out which agent SDKs you use and generates a SKILL.md from the matching Trustabl rules. Claude Code and Cursor read the skill and apply the rules as they write: specific tool descriptions, timeouts on network calls, explicit max-turns limits and scoped permissions. The skill is generated per repository and updates as your dependencies and the rules change. Install it as an MCP server:
claude mcp add agent-reliability-skills -- npx -y https://github.com/trustabl/agent-reliability-skills/releases/download/v0.1.0/agent-reliability-skills-0.1.0.tgz
Forge prevents gaps and the Analyzer catches whatever still gets through.
Trustabl's open-source Agent Reliability Analyzer scans agent repositories with 350+ rules across the Claude Agent SDK, OpenAI Agents SDK, Google ADK, MCP, LangChain/LangGraph, CrewAI, AutoGen/AG2, Pydantic AI and the Vercel AI SDK. It runs locally with no account, no LLM and no network.
# install
brew install trustabl/tap/trustabl
# scan
trustabl scan .
# SARIF for GitHub Code Scanning
trustabl scan . --format sarif --sarif-out trustabl.sarif
# optional: AI-suggested fixes with your own key, written only with --apply
trustabl enrich
Each finding includes a severity, confidence level, explanation and suggested fix. You also get readiness scores for tools, agents, sub-agents, skills and the repo as a whole. Add the GitHub Action to check every pull request, or install the Claude Code, Cursor, Codex or Gemini CLI plugin so the coding agent checks its own work as it writes.
Yes. Static analysis reads code, so it can't see what a tool actually does when it runs. The Agent Network Access Probe covers part of that gap before deployment. It runs an agent tool in an OpenShell or Docker sandbox, records every network destination it contacts, and drafts a least-privilege network policy for the sandbox to enforce in production.
Even so, pre-runtime checks won't catch everything. Live behavior still needs evaluation (see Agent Reliability Metrics), tracing and runtime enforcement. Each gap you remove before production is one less incident to debug after it. For the code-level practices, see Best Practices for Building Reliable Agents. For what to add in production, see What Production Controls Do Reliable Agents Need?.
Static analysis for AI agents inspects agent code, configuration and tool definitions for known failure causes without running the agent. It catches problems such as vague tool descriptions, missing timeouts and over-broad permissions before deployment.
Both check code before it runs. ESLint looks for general JavaScript bugs and style problems. Static analysis for AI agents looks for problems that matter because a model reads and acts on the code: vague tool descriptions, missing loop limits, over-broad tool grants, missing guardrails, and unsafe skills or MCP server configs. It also understands agent SDKs such as the Claude Agent SDK, OpenAI Agents SDK and LangGraph, across Python, TypeScript and other languages.
No. Static analysis removes known failure causes before production. Observability and evaluation catch behavior that only appears at runtime. Reliable agents need both.
Related Blogs
Agent Reliability Resources