Atomicwork and New Measure Release ITSMBench for AI Service Desk Tests
In a press release, Atomicwork and New Measure announced ITSMBench, an open source benchmark for testing how frontier AI models handle enterprise IT Service Management work. The benchmark evaluates models and agent frameworks from OpenAI, Anthropic, xAI, GLM, and other developers across service desk tasks.
ITSMBench recreates enterprise environments with 42 mocked software systems, nearly 1,800 database tables, and more than 2,000 REST endpoints. It covers 89 L2 and L3 service desk tasks, including identity and access management, device issues, networking, security operations, infrastructure, and engineering escalations.
Initial results show that the best performing model completed 50.56% of requests typically handled by enterprise IT service teams. Grok 4.5 found the right APIs 83.0% of the time but performed worse on task completion, while Opus 5 completed 63.5% of tasks after the correct tools were identified. GPT-5.6 Sol placed between those models on discovery and execution, and GLM-5.2 cost $0.25 per trial but trailed Opus 5 and GPT-5.6 Sol on execution.
The benchmark also found that agent frameworks affected accuracy and cost. Models often stopped after finding a plausible explanation instead of verifying the root cause, and sometimes left related systems, records, or follow up actions incomplete. The benchmark, methodology, results, and environments are available at atomicwork.com/itsm-bench.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Enterprise AI Brief or Daily AI Brief.
Also, consider following us on social media:
More from Enterprise
Nov 17 HCLTech and Cohere partner on sovereign AI for companies and governments Nov 17 ControlUp launches add-on that shows IT which AI tools employees use Nov 17 Argentum AI signs $630 million, 20-year data centre deal with Kyndryl for Nvidia B300 systems Nov 17 Uber and Zipline start commercial drone delivery for Uber Eats in Texas Nov 17 OpenAI signs OpenAI for Chile partnership with the Chilean governmentEnterprise AI Brief
Weekly report on AI business applications, enterprise software releases, automation tools, and industry implementations.
Industry analysis
2025 Global Business Services Agenda: Gen AI Takes Center Stage
This industry analysis by The Hackett Group explores the transformative impact of generative artificial intelligence (Gen AI) on global business services (GBS) in 2025. The study highlights the shift from exploration to acceleration of Gen AI initiatives, with 89% of executives advancing these projects to improve customer satisfaction, innovate products, and reduce costs. The report also discusses the challenges and strategies for successful Gen AI adoption, emphasizing the need for a technology-enabled operating model and the importance of reskilling the workforce.
Read moreYou may also like
OpenAI Introduces BioBasicsBench for Wet Lab Analysis
New coding benchmark puts Claude Opus 5.5 on top and opens a 35-point gap that DeepSWE hid
SemiAnalysis GDPx benchmark: best frontier model scores 12% across 11 high-value domains
OpenAI Tests GPT-6 Astra on Shopping and Meal Planning
MLPerf Training v6.1 adds first LLM post-training benchmark as Rubin and TPUv7 debut
Daily AI Brief: the AI news that matters, in your inbox.