Evaluating AI Agents Live at the Grounded Reasoning Cup

This year, Databricks hosted the inaugural Grounded Reasoning Cup, a first-of-its-kind live AI competition to evaluate AI agents’ ability to reason over complex, enterprise-style document collections. By testing agents on a newly released corpus under live competition conditions, the Grounded Reasoning Cup was designed to help answer one of the hardest questions in AI evaluation: how well do performance improvements on a benchmark generalize to similar, real-world tasks?

The competition brought together 11 top academic teams from across the U.S. and Canada, paired with resources and mentorship from frontier labs including OpenAI, Anthropic, and Google DeepMind. Over the course of two months, teams developed and optimized their agents on OfficeQA, our flagship grounded-reasoning benchmark designed to reflect economically valuable enterprise workflows. On competition day, they were challenged to apply those systems in real time to a newly released grounded-reasoning benchmark, OfficeQA Pro V2, designed to test whether their improvements generalized.

Stanford won with a system that achieved 63.3% accuracy, beating out the average team by approximately +22 points, and the average frontier agent offline baseline by approximately +35 points. The top teams demonstrated substantial gains through document preprocessing, targeted retrieval, parallel agents, structured tool use, and verification. At the same time, 18.8% of questions went unsolved by every team, underscoring how much headroom remains in enterprise grounded reasoning.