November 12, 2026 

Reliable Agents Start Before Runtime

Executive summary

  • 97.1% of MCP tools in a Queen's University audit had at least one description flaw (Hasan et al., 2026). That flaw sits in the code before the agent ever runs.
  • 44.2% of multi-agent failures in UC Berkeley's MAST study came from system design (Cemri et al.).
  • An ounce of prevention is worth a pound of cure. Fixing these in a pull request costs far less than debugging them in production, and preventing them while coding with Trustabl Forge costs less still.

Why fix agent reliability before runtime?

Fixing agent reliability before runtime is cheaper because many failure causes are visible in code and can be corrected in a pull request. When an agent fails in production, someone has to notice, reproduce it, read traces and find the cause. Often the cause turns out to be one line — a tool described as "handles orders," an HTTP call with no timeout, or a max_turns that was never set.

Those causes were in the repository the whole time. Software teams learned this decades ago with linters and static analysis. Agents need the same thing, adapted to how agents fail. It is one of the core ideas behind agent reliability: you engineer reliability before deployment as well as watching for it after.

Which agent failures are visible in code?

Three recent studies show that tool definitions, system design and missing fault handling drive a large share of agent failures, and all three can be seen in code.

Study Finding Visible in code?
MCP Tool Descriptions Are Smelly! (Hasan et al.) 97.1% of 856 MCP tools had a description flaw; rewriting descriptions raised task success by a median of about 6 percentage points across four models (GPT-4.1: 18% to 29%) Yes, in tool definitions
Why Do Multi-Agent LLM Systems Fail? (Cemri et al.) 44.2% of failures came from system design; nearly 16% of traces showed step repetition Largely, in config and orchestration
ReliabilityBench (Gupta) Rate limiting was the most damaging injected fault Yes, as missing timeouts and retries

What reliability gaps can static analysis catch?

Static analysis can catch at least seven common agent reliability gaps before runtime. The examples are real rule IDs from the open Agent Reliability Rules.

Gap Example rule Why it breaks reliability
Placeholder tool description CSDK-017 The model can't choose the right tool consistently
Description too short to guide selection CSDK-018 Inconsistent tool choice
Network call with no timeout CSDK-003, OAI-005, MCP-004 One slow dependency stalls or crashes the run
No explicit max turns CSDK-206 Runaway loops and runaway cost
No input guardrails on shell or filesystem tools OAI-101 Unbounded harm when the agent is wrong or manipulated
Permissions bypassed CSDK-103 (bypassPermissions) Removes the human check entirely
Over-broad tool grants Agent-scope rules The agent can do more than its purpose requires

Why use deterministic analysis for AI agents?

Deterministic analysis returns the same findings on the same code every time. Using an LLM to review an LLM agent brings back the same inconsistency you are trying to remove. Consistent findings make it safe to use as a pull request gate, and your code never has to leave your machine.

Can you prevent reliability gaps while coding?

Yes. The earliest place to fix a reliability gap is before it is written. Most agent code is now written with help from a coding agent, so Trustabl Forge gives that coding agent the rules up front.

Forge reads your repo's dependency manifests, works out which agent SDKs you use and generates a SKILL.md from the matching Trustabl rules. Claude Code and Cursor read the skill and apply the rules as they write: specific tool descriptions, timeouts on network calls, explicit max-turns limits and scoped permissions. The skill is generated per repository and updates as your dependencies and the rules change. Install it as an MCP server:

claude mcp add agent-reliability-skills -- npx -y https://github.com/trustabl/agent-reliability-skills/releases/download/v0.1.0/agent-reliability-skills-0.1.0.tgz

Forge prevents gaps and the Analyzer catches whatever still gets through.

How do you add pre-runtime checks to an agent?

Trustabl's open-source Agent Reliability Analyzer scans agent repositories with 350+ rules across the Claude Agent SDK, OpenAI Agents SDK, Google ADK, MCP, LangChain/LangGraph, CrewAI, AutoGen/AG2, Pydantic AI and the Vercel AI SDK. It runs locally with no account, no LLM and no network.

# install
brew install trustabl/tap/trustabl

# scan
trustabl scan .

# SARIF for GitHub Code Scanning
trustabl scan . --format sarif --sarif-out trustabl.sarif

# optional: AI-suggested fixes with your own key, written only with --apply
trustabl enrich

Each finding includes a severity, confidence level, explanation and suggested fix. You also get readiness scores for tools, agents, sub-agents, skills and the repo as a whole. Add the GitHub Action to check every pull request, or install the Claude Code, Cursor, Codex or Gemini CLI plugin so the coding agent checks its own work as it writes.

Do you still need runtime tools?

Yes. Static analysis reads code, so it can't see what a tool actually does when it runs. The Agent Network Access Probe covers part of that gap before deployment. It runs an agent tool in an OpenShell or Docker sandbox, records every network destination it contacts, and drafts a least-privilege network policy for the sandbox to enforce in production.

Even so, pre-runtime checks won't catch everything. Live behavior still needs evaluation (see Agent Reliability Metrics), tracing and runtime enforcement. Each gap you remove before production is one less incident to debug after it. For the code-level practices, see Best Practices for Building Reliable Agents. For what to add in production, see What Production Controls Do Reliable Agents Need?.

FAQ

What is static analysis for AI agents?

Static analysis for AI agents inspects agent code, configuration and tool definitions for known failure causes without running the agent. It catches problems such as vague tool descriptions, missing timeouts and over-broad permissions before deployment.

How is static analysis for AI agents different from ESLint?

Both check code before it runs. ESLint looks for general JavaScript bugs and style problems. Static analysis for AI agents looks for problems that matter because a model reads and acts on the code: vague tool descriptions, missing loop limits, over-broad tool grants, missing guardrails, and unsafe skills or MCP server configs. It also understands agent SDKs such as the Claude Agent SDK, OpenAI Agents SDK and LangGraph, across Python, TypeScript and other languages.

Does static analysis replace observability for agents?

No. Static analysis removes known failure causes before production. Observability and evaluation catch behavior that only appears at runtime. Reliable agents need both.



Share Now!

Like what you see? Share it with your network.

Related Blogs

Agent Reliability Resources

Agent Reliability Engineering
Explore the open discipline for reliable, governable autonomous AI agents.
Visit ARE →
Agent Reliability Engineering Tools
Explore Trustabl's open-source agent reliability tools on GitHub.
View Tools →
downloadchevron-down