Nordic APIs Summit 2026 - October 12-14 Early Bird Registration Open!
Supported by Curity Logotype

Session

From Outcomes to Procedures: The Hidden Failure Modes of AI Agents

Current agent evaluations predominantly reward what gets done (end states, correct API calls) while overlooking how it is done. This outcome-first view not only creates a misleading picture of performance but also hides harmful behaviors. An agent might “succeed” at a technical task yet fail the user with hallucination, policy violation, or sociolinguistic failure. Similarly, simple success rates mask the danger of failures: A safe handoff to a human is treated the same as a costly mistake, like issuing a refund without permission. We propose a framework to answer these questions holistically and systematically. We identify 11 consistency relationships between what agents observe, what they do, and what they tell the customer, and formalize procedure contracts as typed constraints over these relationships: what must never happen (Forbid), what must precede what (Order), what communications must be grounded in retrieved data (Ground), and what falls within authorized scope (Scope). When we applied this framework to frontier models on realistic service scenarios, we found 27–78% of benchmark successes are corrupt: correct outcome, broken process. Each model fails differently, and smarter agents make smarter mistakes that are harder to catch before they reach your customers.

Smarter Tech Decisions Using APIs

Smarter Tech Decisions Using APIs

API blog

High impact blog posts and eBooks on API business models, and tech advice

API conferences

Connect with market leading platform creators at our events

API community

Join a helpful community of API practitioners

API Insights Straight to Your Inbox!

Can't make it to the event? Signup to the Nordic APIs newsletter for quality content. High impact blog posts on API business models and tech advice.

By clicking below, you agree that we process your information per the terms in our Privacy Policy.

Join Our Thriving Community

Become a part of our global community of API practitioners and enthusiasts. Share your insights on the blog, speak at an event or exhibit at our conferences and create new business relationships with decision makers and top influencers responsible for API solutions.