Coding agents finish fewer than 8% of assignments in SWE-CoWork benchmark

Nov 22, 2026
The benchmark makes agents collect requirements from 26 simulated colleagues; 31% of requirements agents reported as done failed their tests.

A new benchmark called SWE-CoWork finds that leading coding agents rarely finish whole assignments when they have to gather requirements from colleagues, Superpower Daily reports. Across 11 models, the two leaders passed about 63% of tests for required changes but completed fewer than 8% of assignments. The paper, by researchers led by Xinping Lei, was published on 15 November.

The benchmark has 101 assignments in 101 real code repositories, each placing an agent in a simulated engineering organisation with 26 colleagues who hold scattered and sometimes conflicting requirements. Giving Seed Evolving and GPT-6 Astra the complete requirements upfront improved their test results by about 16 percentage points but did not raise full-task success. Claude Opus 5.5 sent 93% of its first questions to the right owners but obtained only 64% of the requirements colleagues held, and 31% of requirements that agents reported as completed failed their tests.

We hope you enjoyed this article

Consider subscribing to one of our newsletters like AI Programming Weekly or Daily AI Brief.

Also, consider following us on social media:

Free newsletter

AI Programming Weekly

Weekly news about AI tools for software engineers, AI enabled IDE's and much more.

Whitepaper

Tensordyne Napier: What If One Rack Could Do the Work of Nine?

Tensordyne

This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.

Read more
Free, six days a week

Daily AI Brief: the AI news that matters, in your inbox.