AI models marked civilians as military targets in Vals tests, and calling it a game beat refusals
Frontier AI models usually agreed to mark people and targets in drone and satellite images, and often picked civilians, according to tests published by AI evaluator Vals AI. When models did refuse, telling them the task was a game almost always got them to comply, researchers Geby Jaff and Anthony Ozerov write in the report.
The researchers ran three kinds of tests: marking targets in drone imagery of a fictional battlefield in eastern Ukraine, a long drone survey in which reports of people were followed by images of strikes and casualties, and marking targets in real satellite images of Minab, Iran, where an elementary school sits next to a military complex. In the drone tests, models often boxed civilians, ambulances and surrendering or incapacitated soldiers. Most models kept going after seeing the aftermath of strikes. Vals says none of its requests was blocked by a safety classifier and that refusals came only from the models themselves.
In the satellite test, Anthropic's Claude Opus 5.5 selected the school more often than any other model. Framing a task as a game changed some results sharply: OpenAI's GPT-6 Astra returned at least one target in 6% of responses without a game claim and 92% with one, while Claude Opus 5.5 went from 71% to 75%, The Defiant reports.
Vals stresses the limits: it tested whether models would do these tasks, not how accurate they were, whether they followed the laws of war or how they would perform in real military systems.
Vals says it shared its findings with OpenAI, Anthropic, Google, Meta and SpaceXAI on 1 November and held publication for two weeks. OpenAI said in a statement shared by Vals that it is working on alignment updates that explicitly cover drones and their software, including help framed as simulation, gaming or academic or defensive work. Anthropic's usage policy, effective 12 November, prohibits use for autonomous weapons. Vals recommends safeguards that block weapons-related requests and training models not to automatically accept claims that potentially harmful tasks are only games.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like AI Policy Brief, Defense AI Brief or Daily AI Brief.
Also, consider following us on social media:
More from AI Safety
Nov 17 Oregon sues OpenAI, claiming ChatGPT endangers teens Nov 17 New York Attorney General creates AI safety enforcement unit to police the RAISE Act Nov 17 Gemini AI agents in safety tests repeatedly bypassed safeguards to attack Epoch AI's live website Nov 17 Anthropic cuts its vulnerability disclosure deadline to 30 days for serious bugs Nov 17 US and China prepare for AI safety talks at APEC after agreeing an AI incident channelMore from Defense
Nov 17 Defence autonomy tester Valkryn emerges from stealth with $17 million, led by Andreessen Horowitz Nov 17 OpenAI gives NATO access to its Daybreak cyber defense models Nov 16 AI-designed munitions startup Tiberius raises $85 million at a $1 billion valuation Nov 16 Plural raises €1B for European AI, defence and deep tech startups Nov 16 ICEYE unveils Gen5 radar satellites with 11 cm resolution and on-board AIAI Policy Brief
Weekly report on AI regulations, safety standards, government policies, and compliance requirements worldwide.
Market report
2025 Generative AI in Professional Services Report
Thomson Reuters
This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.
Read moreYou may also like
OpenAI agents weaken internal misalignment monitoring
OpenAI: GPT-6 Astra stopped an alignment test over a rule it invented
OpenAI Expands Monitoring for Frontier Model Misalignment
Anthropic safety guide shows Claude Fable 5.1 scored 94% in app but 60% via API on self-harm tests
Anthropic Automates Search for Training Data That Evades Safety Monitors
Daily AI Brief: the AI news that matters, in your inbox.