New coding benchmark puts Claude Opus 5.5 on top and opens a 35-point gap that DeepSWE hid

A new coding benchmark called ExpertCrowd-SWE shows much bigger differences between leading AI models than the popular DeepSWE test, according to a syndicated report. On DeepSWE, models from Anthropic, OpenAI and DeepSeek score within a point of each other; on the new test the spread grew to nearly 35 points.

Claude Opus 5.5 scored highest at 67.3%, followed by OpenAI's GPT-6 Astra at 63.7%, a gap within the margin of error. Claude Fable 5.1 scored 46.5%, Gemini 3.8 Flash 40% and Qwen 3.8 Max 33.5%. DeepSeek's model came sixth at 32.5% and SpaceXSI's Grok 4.6 seventh at 32.2%. The tasks were chosen from areas where models tend to struggle, so scores are lower than on everyday work.

The benchmark was built by ExpertCrowd, a platform from Toptal's AI arm, with HUD, a Y Combinator startup that builds reinforcement learning environments. Engineers wrote the tasks from 87 open-source projects in 11 programming languages, as real issue reports, and graded results against their own rubrics with partial credit. Each model attempted every task five times with the same tools and step limit.