Atomicwork and New Measure Release ITSMBench for AI Service Desk Tests

In a press release, Atomicwork and New Measure announced ITSMBench, an open source benchmark for testing how frontier AI models handle enterprise IT Service Management work. The benchmark evaluates models and agent frameworks from OpenAI, Anthropic, xAI, GLM, and other developers across service desk tasks.

ITSMBench recreates enterprise environments with 42 mocked software systems, nearly 1,800 database tables, and more than 2,000 REST endpoints. It covers 89 L2 and L3 service desk tasks, including identity and access management, device issues, networking, security operations, infrastructure, and engineering escalations.

Initial results show that the best performing model completed 50.56% of requests typically handled by enterprise IT service teams. Grok 4.5 found the right APIs 83.0% of the time but performed worse on task completion, while Opus 5 completed 63.5% of tasks after the correct tools were identified. GPT-5.6 Sol placed between those models on discovery and execution, and GLM-5.2 cost $0.25 per trial but trailed Opus 5 and GPT-5.6 Sol on execution.

The benchmark also found that agent frameworks affected accuracy and cost. Models often stopped after finding a plausible explanation instead of verifying the root cause, and sometimes left related systems, records, or follow up actions incomplete. The benchmark, methodology, results, and environments are available at atomicwork.com/itsm-bench.