Software development has already shifted from writing every line by hand to conducting an orchestra of models, tools, agents and subagents. That shift is real and irreversible. But something critical is still missing.
We can spin up agents in minutes. We can give them tools, memory, subagents, and long context windows. What we cannot yet do reliably is keep them working under production pressure.
Agents fail. They loop. They call the wrong tool. They exceed context. They forget constraints. They burn money. They succeed in demos and collapse when the stakes are real. Gartner has already warned that a large share of agent projects will be canceled for exactly these reasons. The research community is now quantifying why.
This is the moment Agent Reliability Engineering (ARE) becomes unavoidable.
Three recent papers make the diagnosis unmistakable.
Towards a Science of AI Agent Reliability (Princeton, arXiv 2602.16666) argues that raw task accuracy is an incomplete proxy for what production actually needs. The authors decompose reliability into consistency, robustness, predictability, and safety, then propose concrete metrics that stand independent of simple success rates. Accuracy without reliability is a dangerous illusion.
Why Do Multi-Agent LLM Systems Fail? (Cemri et al., arXiv 2503.13657) introduces the Multi-Agent System Failure Taxonomy (MAST). Apparently, “Failure” failed to make the cut in the acronym. Across more than 1,600 annotated traces, the team identified 14 distinct failure modes clustered into system design issues, inter-agent misalignment, and task verification problems. The striking finding is that the majority of failures are not model capability failures. They are specification, coordination, and verification failures. Better prompts alone will not fix them.
Model Context Protocol (MCP) Tool Descriptions Are Smelly! (arXiv 2602.14878) examined 856 tools across popular MCP servers and found that 97.1% of tool descriptions contain at least one “smell.” More than half fail to state their purpose clearly. Unstated limitations, missing usage guidelines, and opaque parameters are the norm. When descriptions were improved, success rates rose—but so did execution steps and cost. Tool quality is not a documentation afterthought; it is a reliability surface.
Taken together, these papers point to the same conclusion: the bottleneck has moved. Model intelligence is advancing rapidly. The systems we wrap around those models, the specifications, configurations, tool interfaces, memory, fallbacks, and policies, are lagging far behind.
This matches what practitioners are seeing in the field. The CEO of Moonshot AI (the team behind the Kimi K models) has been explicit that many agent failures stem not from the underlying model but from context management and configuration. The harness decides what the model sees, which tools it can reach, how errors are returned, when history is compacted, and when the run is allowed to stop. Change any of those levers and both success rate and cost shift.
Agents do not mostly fail because the model is “not smart enough.” They fail because the surrounding system is underspecified, inconsistent, or drifting.
That is an engineering problem. And engineering problems require an engineering discipline.
ARE is the emerging practice of treating autonomous agents as production systems that must be specified, configured, validated, observed, diagnosed, and continuously improved.
It asks different questions than traditional model evaluation:
These are the questions Site Reliability Engineering answered for web services. ARE must answer them for agents.
At Trustabl we believe the answer is not another observability dashboard bolted onto an existing framework. It is a practical engineering layer that helps teams find and fix the reliability and safety gaps that keep agents from surviving contact with production.
Today Trustabl is an open-source static analyzer purpose-built for agent systems. It parses repositories written against the major agent SDKs (Claude Agent SDK, OpenAI Agents SDK, Google ADK, LangChain/LangGraph, CrewAI, AutoGen, Pydantic AI, Vercel AI SDK) and the Model Context Protocol. It models the tools, agents, subagents, skills, and the relationships between them, then checks those surfaces against a growing catalog of reliability and safety rules.
What teams get back is concrete and actionable:
In short, Trustabl already answers the first hard questions of Agent Reliability Engineering: What is actually declared in this agent system? Where are the weak tools, missing guardrails, overly broad permissions, and poorly described skills? What needs to be fixed before this is safe to run?
That foundation is deliberate. Static analysis and scoring are the necessary first step. Trustabl then generates least-privilege policies consumed by much of the agentic stack. From here the roadmap expands toward richer specification of agent intent, deeper configuration intelligence, and closed-loop improvement from runtime signals. The open-source analyzer is already in use; the broader control plane is what we are building next.
The goal remains the same: turn the emerging discipline of Agent Reliability Engineering into something teams can actually practice, starting with the gaps that are visible in the code and configurations they already have.
The teams that treat agents as clever demos will keep shipping systems that work until they don’t. The teams that treat agents as production systems, with configuration intelligence, policy, observation, and continuous improvement, will ship agents that keep working.
We are still early. The research is clarifying the failure modes. The frameworks are maturing. The hard isolation layers such as OpenShell are arriving. What has been missing is the intelligence layer that sits above them and makes reliability systematic rather than heroic.
That is the work Trustabl is doing.
The orchestra is already playing. The question is whether we will learn to keep it in tune.
I often hear the AI accelerationists talk about how overall jobs will grow, but there will be a transition from “old economy” jobs to new AI-enabled jobs. The Forward Deployed Engineer is a hot new category, but I expect that Agent Reliability Engineer will be a new role with plenty of upside.
If you are building or deploying agents and care about making them production-grade, we should talk. The future belongs to the teams that engineer reliability instead of hoping for it.
