OpenAI Builds GPT-Red to Attack and Improve Its Own AI Models

OpenAI has developed GPT-Red, an automated adversarial model built to identify and exploit weaknesses in its own AI systems, according to a company announcement. GPT-Red functions as an internal red-teaming agent that searches for prompt injection vulnerabilities before models are widely deployed.

The system is trained through self-play reinforcement learning. It faces off against a group of defensive models and is rewarded for successful attacks, while defenders earn rewards for resisting them. GPT-Red has been trained using a large compute scale typically reserved for production models, allowing it to discover sophisticated attack methods that human testers might miss.

In practical evaluations, GPT-Red successfully manipulated an AI agent operating a physical vending machine by altering prices and canceling orders. OpenAI disclosed these vulnerabilities and used the findings to improve safeguards. The model also demonstrated effectiveness against earlier OpenAI models, achieving higher success rates than human testers in controlled red-teaming tests.

OpenAI incorporated results from GPT-Red’s attacks to train GPT-5.6, which now shows significantly greater resistance to prompt injection attempts compared to prior versions. Despite GPT-Red’s strength, the company has stated it will not release the model publicly to prevent misuse of its offensive capabilities.