Google Threat Intelligence Group Reports Surge in AI Misuse for Cyber Operations
The Google Threat Intelligence Group has published new findings showing a global rise in the misuse of artificial intelligence by both state-sponsored and private-sector threat actors. The report identifies widespread use of AI tools to enhance phishing, reconnaissance, and malware creation activities across multiple regions.
According to the report, government-backed actors from North Korea, Iran, China, and Russia are using large language models to generate realistic phishing content, conduct target research, and support coding tasks. These actors leveraged AI systems, including Gemini, to automate reconnaissance and improve the credibility of their social engineering campaigns. Google has taken actions to disable accounts and assets linked to these malicious operations.
The report also highlights a growing number of model extraction or “distillation” attacks, in which adversaries attempt to replicate the capabilities of proprietary AI models through repeated API queries. These activities were primarily attributed to private-sector entities and researchers seeking to clone model logic. Google stated that it has detected and mitigated these attacks to protect its AI systems.
In addition, the report outlines experimental uses of AI in malware development. Examples include the HONESTCUE malware family, which used Gemini’s API to generate code for secondary payloads, and the COINBAIT phishing kit, likely built using AI code generation tools. These cases illustrate how attackers are integrating AI into traditional cyber operations to increase speed and efficiency.
Google noted that while no direct attacks on frontier AI models have been observed from advanced persistent threat groups, the company continues to strengthen its security measures to prevent misuse. GTIG emphasized that organizations operating AI services should monitor for extraction patterns and apply safeguards to protect proprietary systems.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Cybersecurity AI Weekly, AI Policy Brief or Daily AI Brief.
Also, consider following us on social media:
More from Cybersecurity
Nov 19 Arizona opens criminal investigation into OpenAI over what it calls rogue AI agents in banks Nov 19 Citi tells customers they are on their own if third-party AI agents misuse their accounts Nov 18 Google releases Gemini 4 Neon, ranking first on a coding agent index at $3 per million input tokens Nov 18 Palo Alto Networks to buy Applied Compute for $4.5 billion to secure open-weight AI Nov 18 Rapid7 finds thousands of exposed Ollama servers, many already used by outsidersMore from AI Safety
Nov 19 US Catholic bishops form AI task force and push for guardrails and worker protections Nov 18 Nvidia scandal complicates US-China AI talks ahead of Trump-Xi meeting, Wired reports Nov 18 Beijing AI safety group Concordia AI briefed OpenAI staff on China's approach to AI risk Nov 18 OpenAI details how it contains frontier models, including a pause after a DNS exploit Nov 18 Coalition including Hugging Face and xAI co-founders says Anthropic "is not a safe AI company"Cybersecurity AI Weekly
Weekly newsletter about AI in Cybersecurity.
Market report
2025 Generative AI in Professional Services Report
Thomson Reuters
This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.
Read moreYou may also like
Microsoft Finds Broad AI Use Among North Korean Threat Groups
Recorded Future Finds AI Accelerating Familiar Cyberattacks
GAO tests got Claude, ChatGPT, Gemini and Grok to produce viable attack plans
Attackers Used 44 AI Agents in CyberStrikeAI Campaign Against Japan
NCSC Says AI Will Accelerate Cyberattacks and Widen Security Gaps
Daily AI Brief: the AI news that matters, in your inbox.