AI or Not Audit Finds AI Models Can Create Realistic Fake IDs

AI or Not announced in a press release that 92 percent of 16 tested AI image generation models successfully produced synthetic government identity documents that could deceive a human reviewer. The audit covered systems from multiple vendors, including Google Gemini, ChatGPT, Grok, and Imagen 4 Ultra.

The study found that five consumer AI products created high-fidelity fake adult IDs. Three models (Google Gemini, Grok, and Imagen 4 Ultra) also generated realistic fake IDs depicting minors through standard consumer interfaces. Two additional models, OpenAI's ChatGPT and Recraft v4, declined such requests in their consumer interfaces but fulfilled them through their developer APIs.

AI or Not reported a 92 percent aggregate bypass rate across 75 test attempts, with outputs generated for 17 countries and 16 U.S. states. All 16 models produced synthetic IDs when prompts were reframed as compliance or verification tasks, showing that safety filters relied on detecting surface intent rather than blocking specific content types.

The audit team notified affected vendors on May 18, 2026, ahead of the public release on June 2. AI or Not withheld details of the specific prompts and bypass methods from the public report but provided technical findings to qualified researchers and vendors under embargo.