EnterpriseOps Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings EnterpriseOps Gym is a containerized, resettable enterprise simulation benchmark for evaluating LLM agents on stateful, multi step planning and tool use across realistic enterprise workflows About EnterpriseOps Gym is a large scale benchmark for evaluating the agentic planning and tool use capabilities of LLM agents across enterprise operations. It comprises 1,150 expert curated tasks spanning 8 enterprise domains , each running against live containerized MCP servers backed by realistic, fully synthetic databases. Unlike static QA benchmarks, EnterpriseOps Gym evaluates agents on final environment state using SQL verifiers meaning agents are rewarded for achieving the correct outcome, not for following a rigid action sequence. Tasks require long horizon multi step reasoning, strict policy compliance, and precise tool invocation under complex data dependencies. Best model performance: 34.1% success rate leaving significant headroom for future research. Key Features 🛠️ 512 tools across 8 enterprise domains 🗄️ 164 database tables with avg 1.7 foreign key dependencies per…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy