OpenAI's Codex-powered Performance Factory investigates ChatGPT slowdowns on its own
OpenAI has built Performance Factory, an internal system powered by Codex that automates the investigation and fixing of performance regressions, and is using it to keep ChatGPT fast. In just over three months the system tracked more than 1,700 investigations and identified the root causes of more than 600 cases, about 17 automated investigations a day, according to an engineering report shared with VentureBeat.
The system turns each regression into a task that moves through stages such as investigation, root cause, code change, review and verification. A dispatch loop starts a Codex session when a task is ready to advance, agents run in cloud environments with access to OpenAI's code and observability tools, and tasks can pause until an engineer reviews a pull request on GitHub.
OpenAI found that agents sometimes stopped at the first plausible explanation, even when the evidence did not fully support it. It added an "adversarial review" in which a separate Codex session with a fresh context is told to find gaps in the diagnosis, VentureBeat reports.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like AI Programming Weekly or Daily AI Brief.
Also, consider following us on social media:
AI Programming Weekly
Weekly news about AI tools for software engineers, AI enabled IDE's and much more.
Whitepaper
Tensordyne Napier: What If One Rack Could Do the Work of Nine?
Tensordyne
This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.
Read moreYou may also like
Gilbert + Tobin Rolls Out ChatGPT Enterprise and Codex, Reports 87% Active Use
External test found gaps in OpenAI agent monitoring
Holiday Extras rebuilt its main app in two weeks with OpenAI's Codex
OpenAI details how it contains frontier models, including a pause after a DNS exploit
OpenAI Expands Monitoring for Frontier Model Misalignment
Daily AI Brief: the AI news that matters, in your inbox.