New York limits AI chatbots for minors, Anthropic backs court power to halt dangerous models, and more - AI Policy Brief #93

In this edition, covering 26 October to 2 November:

• New York Governor Kathy Hochul signed a law restricting AI chatbot features that can harm minors, with civil penalties of up to $25,000 per violation.
• Anthropic told California Governor Gavin Newsom that courts should be able to suspend AI models posing a near term risk of catastrophic harm.
• The US Department of Justice is examining how Nvidia's planned $12.93 billion acquisition of Hugging Face could affect AI competition, Reuters reports.

This spot is open for 2027. One company puts its name on every AI Policy Brief, which has 900+ subscribers. See what's included

Anthropic

Anthropic told Governor Gavin Newsom that courts should be able to suspend AI models posing a near term risk of catastrophic harm. Senator Elizabeth Warren asked CEO Dario Amodei how Anthropic will address risks such as loss of control, by 3 November. Anthropic gave $900,000 to Democratic committees in California. Letter to Newsom · Warren · California donations

AI company employees gave tens of thousands of dollars to Minnesota candidates from both parties. A 40% GSA discount on Claude could save the US government an estimated $1 billion. Anthropic says an unreleased model found and chained flaws in Chrome, Safari, Android, Windows and Linux. Minnesota · GSA discount · Exploits

A new self assessment layer flagged 78% of potentially harmful conversations on Claude Fable 5. Anthropic researchers built fine tuning datasets that made smaller models misaligned while passing a monitor. In a study of 50 causal analysts using Claude Code, they followed their agents' assigned preferences 86% of the time. Self assessment · Training data · Analyst study

Anthropic's amended IPO filing reports a $104.0 billion net loss for the nine months ended 30 September. IPO filing risks

Anthropic released FlashProve, an open source prototype that uses zero knowledge proofs to show that a response came from a specific model and its committed weights. FlashProve

OpenAI

OpenAI will limit weapons related information for users under 18 and let ChatGPT end chats fixated on violence. The Observer and Tortoise report that OpenAI's investigations team faced a backlog of flagged ChatGPT conversations. OpenAI rewrote its public Model Spec around faithfulness to human intent. Teen safeguards · Backlog · Model Spec

A 29 October update made GPT-6 Luna give shorter and less agreeable answers. In a replay of the Hugging Face incident, OpenAI's monitors alerted security staff about 30 minutes after agents escaped. OpenAI wants California to require reporting of AI safety incidents that cause no death or injury. GPT-6 Luna · Monitoring · SB 53

OpenAI plans a policy and public relations campaign in Washington after the midterms. Twenty House Democrats asked Sam Altman why three safety researchers were fired, by 13 November. OpenAI will honor a shared robots.txt signal for AI training preferences. Washington campaign · Researcher firings · robots.txt

OpenAI lobbyist Chris Lehane is advising prospective 2028 presidential candidates on AI. Delaware's Department of Justice issued a broad subpoena to OpenAI over the July 2026 Hugging Face hack. OpenAI uses probes that read model activations in real time to flag unsafe activity. 2028 candidates · Delaware subpoena · Probes

Models and safety research

Fangcun AI found that DeepSeek V4.1 Flash-1017 disabled a shutdown script in 39% of runs after a warning. Palisade Research found Alibaba Qwen 3.8 models routing web requests through Alibaba Cloud proxies. Reuters reports that DeepSeek and Moonshot AI trained on outputs from OpenAI's GPT-6 Astra. DeepSeek shutdown test · Qwen proxies · Distillation

In Y Combinator and Olam Labs experiments, GPT 6.1 Sol agents beat Claude Opus 5.5 agents 6 to 3 in Minecraft Bed Wars, and GPT agents beat Claude agents 6 to 1 in Mindustry. Minecraft and Mindustry

United States

The Justice Department is examining Nvidia's planned $12.93 billion Hugging Face acquisition, Reuters reports. House Democratic leaders asked five frontier AI companies for their misalignment and safety incidents by 11 November. House Oversight is investigating Delaware's proposal for companies run by AI agents. Nvidia and Hugging Face · Incident reports · Delaware

Robert Garcia and Jamie Raskin asked the White House to preserve President Trump's chats with Grok. Maxine Waters and Greg Casar introduced a resolution opposing bailouts for insolvent AI companies. HHS's HTI-5 rule removes 14 of 60 health IT certification criteria. Grok chats · Bailouts · HTI-5

The GAO found that some of 10 reviewed Pentagon AI tools did not closely follow responsible AI guidelines. POGO reports that Rhombus Power's military AI tools often failed to perform as advertised. Governor Kathy Hochul signed a New York law restricting chatbot features that can harm minors. GAO · Rhombus Power · New York minors law

RAISE Act 2.0 would require independent safety audits. California's AB 1609 requires large businesses to connect customers with a person within 15 minutes from 1 January 2027. Florida's attorney general sued the owner of A2E over sexually explicit images of children. RAISE Act 2.0 · California handoffs · A2E

The New York State Bar Association adopted AI guidelines for lawyers. Lawyer guidelines

Courts and copyright

In The New York Times case against OpenAI, the Wikimedia Foundation argued that fair use may cover training. OpenAI published excerpts from filings backing fair use. The News/Media Alliance and the Open Markets Institute support the publishers. Wikimedia · OpenAI filings · News/Media Alliance

SAG-AFTRA and other unions oppose a categorical fair use exemption for AI training. The Human Consent Foundation argues creative works can be licensed through existing rights systems. Susan Feinberg sued OpenAI and Luka, alleging ChatGPT and Replika contributed to her husband's suicide. Unions · Human Consent Foundation · Feinberg

An Amazon engineer's parents sued OpenAI, alleging ChatGPT helped her taper her epilepsy medication before she died. British Columbia's privacy commissioner is investigating OpenAI over the Tumbler Ridge shooter. A judge allowed Character.AI's asset sale despite objections from 41 state attorneys general. Seizure medication · British Columbia · Character.AI

Apple's trade secrets lawsuit against OpenAI goes to trial on 19 July 2027. A proposed class action says Allegheny Health Network and Highmark did not properly disclose that Abridge records patients. WME urged AI companies to license and pay for training material. Apple trial · Abridge · WME brief

Newspapers including The New York Times sued Brave Software, accusing it of scraping more than 30 billion webpages. Brave lawsuits

A Singapore intellectual property tribunal rejected OpenAI's five objections to Porolis Technologies' GemGPT trademark, finding the shared letters GPT did not show likely confusion, and awarded Porolis S$10,000 in costs. GemGPT ruling

Elections

OpenAI opened an Election Operations Center to monitor misuse of its tools until all midterm votes are counted. Washington Post tests found chatbots failed to name a congressional district in 48% of AI voter guides. A Forum AI study found factual errors in nearly 40 percent of election answers. OpenAI center · Voter guides · False information

A Stacey Garrity ad in Pennsylvania uses an AI generated Governor Josh Shapiro. In Texas, a pro Paxton super PAC is using conversational AI calls against James Talarico. Pennsylvania ad · Texas robocalls

Europe and UK

The European Commission published EN 18286:2026, the first harmonised standard under the EU AI Act. The 53 WHO European Region states adopted a 2026 to 2033 roadmap for AI in health care. Canada joined the EuroHPC Joint Undertaking. EU standard · WHO Europe · EuroHPC

The Dutch data protection authority warned 29 debt collectors about AI deciding whom they contact first. The Dutch government plans to cut 25,000 civil service jobs by 2035, partly with AI. Norway's Havtil found uneven AI safety oversight among oil and gas companies. Debt collectors · Dutch civil service · Havtil

The UK health department found that 90,835 of 118,756 responses to its vapes consultation came from bots. A Grant Thornton audit found that Capita failed to deliver promised AI for the Civil Service Pension Scheme. The borders inspectorate asked about two Home Office AI tools used in asylum cases. Vape consultation · Capita · Asylum tools

Government figures put UK AI revenues at £33.6 billion in 2025. UK business AI adoption reached 32% in 2026, up from 16%. A Bank of England and FCA survey found AI agents in production at 80% of 196 financial firms. AI revenues · Adoption · Financial firms

London Mayor Sadiq Khan called for a cap on robotaxi numbers. Germany's AI task force proposed 18 measures to reach sovereign AI status by 2030. Digital Minister Karsten Wildberger urged faster AI adoption at the Digital Summit in Berlin. Robotaxis · German blueprint · Digital Summit

Austria proposed a United Nations center in Vienna to study risks from advanced AI. Vienna UN center

TÜV-Verband and TÜV AI.Lab proposed independent testing of frontier AI with access to models, training processes and safety information, and called for Germany's AISI to become an independent institution. TÜV

The CMA, FCA, ICO and Ofcom, through the Digital Regulation Cooperation Forum, say existing UK law already applies to personal AI agents and deployers remain responsible for their actions. UK regulators

Asia and other national AI plans

India's telecom regulator proposed rules requiring operators using AI to report serious AI incidents, with comments due by 27 November. Singapore Prime Minister Lawrence Wong proposed an international pilot for sharing AI incident data. Japan's Digital Agency is testing AI drafting of answers to Diet questions. India telecom rules · Singapore pilot · Japan

South Korea plans to give about 1,700 AI robots to public bodies in 2027. East African Community ministers adopted a regional AI strategy. ADNOC will deploy OpenAI's Frontier technology across its operations. South Korea robots · East Africa · Abu Dhabi

Canada's privacy regulators called for stronger privacy protections for AI in health care. Hong Kong's SFC found that 30% of 20 large financial firms use agentic AI. The HKMA plans Project SAGE to test AI agents in finance, and HSBC plans an AI institute with HKUST. Canada health · SFC survey · HSBC and HKMA

Eswatini's Standards Authority and AICE Africa launched an AI Safety and Standards Center of Excellence. India launched a handbook for training AI on data that stays under providers' control. Eswatini center · DEPA handbook

South Korean President Lee Jae-myung said the country will allocate 21.3 trillion won to AI and semiconductors next year in a proposed national budget of 821 trillion won. South Korea budget

China will explore AI in financial regulation and services, according to a State Council report submitted to the Standing Committee of the National People's Congress. China finance

Data centers and energy

An xAI subsidiary has bought five homes in Southaven, Mississippi, amid a lawsuit over turbine noise. A federal court refused to dismiss an NAACP lawsuit over 27 gas turbines running without air permits. San Diego's council unanimously approved a 45 day data center moratorium. Southaven homes · NAACP lawsuit · San Diego

Four California representatives plan a bill to block a data center at San Diego's NAVWAR site without local approval. The UK Autumn Budget offers data centres priority grid connections for local investment. UK officials met 137 businesses and no environmental groups while planning AI growth zones, Spotlight on Corruption found. NAVWAR · UK covenant · AI growth zones

Malaysia will require new data centers to source at least 30% of their electricity from renewables. Finnish police are investigating Google's local subsidiary over forest clearing at Muhos. Investigate Europe reports abuses of migrant workers on European data centre projects. Malaysia · Muhos · Worker abuses

Security

In Irregular's tests, Z.AI's GLM-5.3 completed three of 13 cyberattack scenarios and refused no offensive task. Malaysian customs seized 30 servers with US AI chips worth about $12.2 million. US intelligence reportedly surveilled Megaspeed executive Alice Huang over suspected China ties, The New York Times reports. GLM-5.3 · Chip seizure · Megaspeed

The European Cybersecurity Competence Centre launched €42 million of projects on generative AI for cyber defence. ECCC cluster

CrowdStrike introduced an AI Kill Switch in Falcon Guardian that lets authorized security teams stop rogue AI agents, block relaunches and cut network access, recording each action for review. AI Kill Switch

The UK National Cyber Security Centre says AI will mainly speed up and scale existing attack methods, and organisations with weaker security and fewer resources will face greater risk. NCSC

Austin based Wraithwatch earned FedRAMP Class D, the High level certification through Knox Systems, so US federal agencies can deploy its AI cyber defense platform. Wraithwatch

Police and public services

Sydney drafted a policy letting AI flag unusual behaviour in CCTV footage, without facial recognition. California's Judicial Council cancelled AI translation at court counters after mistranslations in a pilot. Sydney CCTV · California courts

Schools

South Korea will seek criminal charges against students who cheat on the CSAT with AI glasses. A Dutch lower house majority backs banning generative AI for primary school pupils. Türkiye plans to score open ended student answers with AI. CSAT cheating · Dutch ban · Türkiye

A SETDA report finds that 79% of US state education leaders rank AI as a top technology priority. SETDA

Also

A Queensland bill would make nonconsensual intimate images made with digital tools punishable by up to three years in prison. AI use among 2,221 surveyed Victorian lawyers rose from 37% to 55% in one year. The University of Kentucky let a Humanist administrator avoid using AI at work. Queensland · Victorian lawyers · Kentucky

The Business Development Bank of Canada launched Mission AI to help 250 entrepreneurs test Canadian made AI tools. BDC Mission AI

Google will provide more than $6 million in grants to the National University of Singapore, Nanyang Technological University and INSEAD for work including Alzheimer's research, AI safety and regional languages. Google Singapore