Epoch AI finds no model meets its employee standard, but Fable 5.2 matches contractors on data center research
No AI model met the standard Epoch AI expects of its own employees on any task in its latest real-work evaluation, though Anthropic's Fable 5.2 and Opus 5.5 reached contractor level on one of them.
Epoch tested six models on an updated version of Epoch Automation Reports, a benchmark of tasks taken from its own work, and graded each output by hand against its internal standards. Fable 5.2 and OpenAI's GPT-6.2 Astra roughly tie for a record on the Epoch Capabilities Index, but GPT-6.2 Astra averaged 65% on the real tasks, 17 points below Fable 5.2.
On adding a data center to Epoch's AI Data Centers database, Fable 5.2 scored 97.5% and Opus 5.5 scored 87.5%, which Epoch's analyst judged comparable to contractor work. Opus 5.5 finished in just under three hours at a lower cost than a contractor, who typically spends about 16 hours. GPT-6.2 Astra scored 65%, leaving out critical information because it was overly cautious. Epoch is now considering moving contractors' time toward reviewing AI output.
Epoch found models still struggle to design, run and present research, and in all but one comparison the models preferred their own outputs over higher-quality human work, according to Epoch AI.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Daily AI Brief.
Also, consider following us on social media:
Daily AI Brief
Daily report covering major AI developments and industry news, with both top stories and complete market updates
Whitepaper
Tensordyne Napier: What If One Rack Could Do the Work of Nine?
Tensordyne
This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.
Read moreYou may also like
OpenAI releases GPT-6.2 Astra and Sol, cutting Astra's API prices by 20%
OpenAI Tests GPT-6 Astra on Shopping and Meal Planning
New coding benchmark puts Claude Opus 5.5 on top and opens a 35-point gap that DeepSWE hid
Artificial Analysis Index v5: open-weight models fall 12 points behind
OpenAI Finds GPT-6 Astra Underperforms on Safety Research
Daily AI Brief: the AI news that matters, in your inbox.