Dragos seeded two million AI prompts with secrets and found breaches at three providers
Researchers at Dragos sent about two million AI prompts containing unique, trackable secrets to widely used AI services between July and November 2026, then watched where those secrets turned up. They found evidence that data from at least three AI providers had been exposed: seeded prompts were used in attempts to access Dragos's controlled test systems and canary AWS accounts, and none of the three providers knew about the breach until Dragos told them.
Multiple resellers on an AI inference marketplace could view and keep customer prompts using standard logging, without exploiting any vulnerability. Requests often passed through more than one router before reaching a model, and every hop could see the full prompt and response, in one case through an undisclosed extra hop.
Dragos also saw providers that appeared not to honour privacy settings or their own policies. A Chinese AI lab's systems fetched the secret URLs from more than half of the prompts sent to its models, including about two thirds of requests marked "no data collection" or Zero Data Retention, and one seeded link appeared in an internal company chat in China, Dragos reports.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Cybersecurity AI Weekly, AI Policy Brief or Daily AI Brief.
Also, consider following us on social media:
More from Cybersecurity
Nov 19 Meta's Muse AI agent mixed up conversations and exposed private data, employees say Nov 19 Extrordinas raises $1.5 million for AI-assisted cyber incident response Nov 19 Nadella warns AI could cause more outages as companies rush security fixes Nov 19 EU says an AI-assisted cyberattack targeted it in October Nov 19 Disseqt AI raises $8 million to test and police enterprise AI agentsMore from AI Safety
Nov 20 Protesters rally against Anthropic in New York and San Francisco on its stock market debut Nov 19 Van Hollen puts AI regulation at the centre of his bid to lead Senate Democrats Nov 19 Anthropic's IPO could channel about $156 billion to charity, TechCrunch estimates Nov 19 OpenAI says an internal model wrote scripts to secretly copy emails during a training run Nov 19 Tumbler Ridge shooter used ChatGPT for months for firearms advice, Mother Jones reportsCybersecurity AI Weekly
Weekly newsletter about AI in Cybersecurity.
Industry analysis
2025 Global Business Services Agenda: Gen AI Takes Center Stage
This industry analysis by The Hackett Group explores the transformative impact of generative artificial intelligence (Gen AI) on global business services (GBS) in 2025. The study highlights the shift from exploration to acceleration of Gen AI initiatives, with 89% of executives advancing these projects to improve customer satisfaction, innovate products, and reduce costs. The report also discusses the challenges and strategies for successful Gen AI adoption, emphasizing the need for a technology-enabled operating model and the importance of reskilling the workforce.
Read moreYou may also like
Northflank Breach Leads to Unauthorized AI API Use
Arrakis Security Finds 76% of AI Agents Dormant but Still Connected
Amazon finds hackers running ARTEX AI agents to steal AI model keys
Attackers used an open-source AI hacking agent with GLM and DeepSeek models to breach 10 organisations, Unit 42 finds
Leaked files show Russian propagandist used Claude and Qwen to churn out pro-Kremlin content
Daily AI Brief: the AI news that matters, in your inbox.