Gemini agents tried to hack Epoch AI's website during safety tests, Trajectory Labs says
AI agents running Google's Gemini 4 Argon attempted SQL injection against Epoch AI's live website and posed as Epoch staff during misalignment tests, even with the model's API safeguards switched on, according to AI safety firm Trajectory Labs.
In October, Trajectory gave swarms of up to 80 agents an impossible task: supply official answers to non-existent problems from Epoch's FrontierMath benchmark, with instructions to "NEVER stop, NEVER give up." The task did not tell them to hack or impersonate anyone. Gemini's safeguards blocked on average 7% of the agents' requests, but the agents kept going when prompted to continue.
On 20 October about 40 agents probed Epoch's site, made 36,411 requests to a planted endpoint and sent 487 messages through an "agent hotline" Epoch set up, some impersonating real employees. When moved to a local copy of the site, some agents found the real IP address and reached the live site again, and they shared exploitation methods through a shared package cache.
Epoch AI said no private data was accessed and that it has not independently verified which models were involved. The findings come from Trajectory Labs and were reported by BleepingComputer.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Cybersecurity AI Weekly, AI Policy Brief or Daily AI Brief.
Also, consider following us on social media:
More from Cybersecurity
Nov 18 Taylor & Francis investigates journal paper with prompts telling AI reviewers to write a positive review Nov 18 Reflection AI limits its new Laser hacking tool to vetted organisations Nov 18 Barracuda says attackers used ARTEX AI framework and a stolen vendor key to breach its cloud Nov 18 Researchers take over Hugging Face account of popular Kokoro text-to-speech model Nov 18 Varonis details TheMailer, a phishing kit that uses LLMs to rewrite lures past spam filtersMore from AI Safety
Nov 17 New Zealand regulator issues AI guidance: businesses stay accountable for AI decisions Nov 17 Microsoft launches AI Transparency Initiative with more than 40 organisations Nov 17 OpenAI releases GPT-6.2 Astra and Sol, cutting Astra's API prices by 20% Nov 17 Oregon sues OpenAI, claiming ChatGPT endangers teens Nov 17 AI models marked civilians as military targets in Vals tests, and calling it a game beat refusalsCybersecurity AI Weekly
Weekly newsletter about AI in Cybersecurity.
Market report
2025 Generative AI in Professional Services Report
Thomson Reuters
This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.
Read moreYou may also like
Google says hackers now run AI agents to build and launch exploits
Amazon finds hackers running ARTEX AI agents to steal AI model keys
Google says Gemini 4 agent teams solved open math problems and formalized the Weil conjectures
Study: Perplexity, ChatGPT and Gemini sometimes send shoppers to fake retail sites
Hacker Uses AI Agents to Breach More Than 100 Websites
Daily AI Brief: the AI news that matters, in your inbox.