Monitoring AI agents is different from monitoring ordinary software. Traditional metrics such as uptime, latency, and error rates still matter, but they do not show whether the agent used the wrong tool, exposed sensitive data, followed a malicious instruction, or took an action outside its approved purpose.
Agent monitoring should focus on behavior and governance evidence.
Monitor tool calls
Tool calls show what the agent actually did. Capture tool name, action type, input summary, target system, permission context, outcome, timestamp, user, and whether human approval was required. For sensitive tools, keep enough detail to reconstruct the decision without storing unnecessary secrets.
Unexpected tool combinations can be early warning signs. An agent that reads customer records and then sends external messages may require tighter controls than one that only summarizes internal documents.
Monitor policy decisions
Record when guardrails allow, block, redact, escalate, or require approval. This helps teams understand whether controls are active and whether users or prompts are repeatedly pushing boundaries.
A high number of blocks may mean the control is working. It may also mean the system design or user instructions need improvement.
Monitor data exposure
Track sensitive data access, retrieval sources, output destinations, and permission mismatches. For retrieval-based systems, verify that responses respect the user's access rights and do not leak unrelated records.
Monitor human review
Capture approval decisions, rejections, edits, escalation, reviewer comments, and override rates. Human oversight is only defensible when the review happened and the reviewer had enough context.
Monitor changes and incidents
Prompt changes, model changes, new tools, new data sources, and expanded user groups can change risk. Monitoring should connect these changes to review requirements. Incidents should be tied back to the agent, root cause, remediation, and control updates.
Good monitoring does not mean collecting everything forever. It means collecting the right events, connecting them to the system record, and using them to improve controls.