Making an autonomous agent reliable enough to ship for an AI SaaS
The challenge
The product was an agent that could take multi-step actions on a user behalf. When it worked it was impressive. When it failed it called the wrong tool, repeated steps, or acted outside its intended scope, and each failure was a support incident and a trust hit.
Our approach
- Enumerated the agent tools and the ways each could be misused, then wrote tests for those paths.
- Ran scope-violation and prompt-injection probes against the agent, not just the base model.
- Built a regression suite that replays known-hard scenarios on every change.
- Added CI gates and tracked latency and cost alongside correctness.
Agents raise the stakes because they act, not just answer. A wrong sentence is embarrassing. A wrong action is an incident. Testing an agent means testing the tools it can call and the ways a user or an attacker can push it off task.
What we did
We enumerated the agent tools and the failure paths for each, then wrote tests that exercised them. We ran scope-violation and injection probes against the full agent rather than the model alone, and we built a regression suite that replays the known-hard scenarios on every change. Correctness was tracked alongside latency and cost, because an agent that is right but slow or expensive is still a problem.
Result
The team shipped faster with fewer incidents, because the gate caught regressions before release rather than users catching them after. The regression suite lives in their repository, so the safety net stays with the product.
Representative engagement. Client details are anonymized and figures are illustrative ranges shown to convey the type and breadth of work, not a specific named result.
