China proposes AI export controls, Anthropic pitches its own safety bill, and more - AI Policy Brief #91

Over the past two years AI Brief has grown into a website, 19 LinkedIn pages with more than 81,000 followers and 15 newsletters with more than 41,000 subscribers, AI Policy Brief among them. Thank you for reading, following and passing it on.

Until now, our newsletters were assembled automatically: the news assigned to each newsletter went into a simple template. From this edition, our small team, Maiko, Rene and Stefan, edits every edition so it reads as one storyline: related news together, the important stories explained, the rest kept short. We still use AI tooling throughout, and we decide what goes in and how it is told.

In this edition, covering 13 to 19 October:

• China proposed export controls on 23 AI technologies, including models with at least 2 trillion parameters, AI agents and chip designs.
• Anthropic presented US lawmakers with its own draft AI safety bill that would override some state laws, and committed 5% of its shares to a public benefit fund.
• Eight US states sued OpenAI over ChatGPT safety, and 39 states asked a court for user protections in the Character.AI bankruptcy sale.

Anthropic

Anthropic representatives presented US lawmakers with a draft bill that would limit government action against advanced AI models to cases involving a foreseeable, imminent risk of more than $1 billion in damage or the death or serious injury of more than 50 people. The proposal covers risks linked to biological weapons, control failures and cyberattacks, and would override some state AI safety laws. The draft bill

Anthropic plans to place 5% of its shares into the wholly owned Anthropic Public Benefit Fund, subject to stockholder approval, and the board could add 2% for every 50% increase in the company's value above a set baseline. Combined with earlier founder and employee commitments, about 20% of its shares would be dedicated to public benefit activities, the company says. New York City Comptroller Mark Levine, who represents city pension funds with more than $325 billion in assets, urged CEO Dario Amodei to pause the planned IPO until the company addresses concerns about AI safety, governance, development speed and investor disclosures. Public benefit fund · The Comptroller

Anthropic is giving six METR employees access to company hardware, office space and internal systems to assess AI risks. Customer data, legally privileged information and model architecture stay restricted. In Singapore, Anthropic opened its fifth Asia Pacific office and signed a memorandum of understanding with the Ministry of Digital Development and Information on AI safety, security, model evaluation and skills. Cofounder Jack Clark told policymakers in Bangkok that an internal company model projects AI could raise US economic output to about $44 trillion by 2030 under certain conditions, and called for international cooperation, stronger safeguards and human oversight. METR access · Singapore · Jack Clark

OpenAI in court

Iowa, Kansas, Nebraska, West Virginia, Arkansas, Arizona, Idaho and Montana filed separate state lawsuits alleging that ChatGPT is unsafe, has harmed children and deceived parents. OpenAI said it has age specific safeguards and will continue working with states on protections. Eight states

In Apple's trade secret lawsuit, OpenAI agreed at a San Jose federal court hearing not to use trade secrets Apple says former employees took to the company, and to collect forensic images from 14 employees' devices. Judge Edward Davila said he was highly likely to issue an injunction. Earlier, a judge ordered OpenAI to turn over interview materials involving former Apple employees, while denying broader requests for its hardware research and development records. OpenAI, which denies Apple's allegations, told the court that an employee who joined Apple sent more than 100 files on an unreleased product to a personal account and later presented the information to Apple executives. Apple said it found no evidence supporting the claim. The hearing · Hiring records · OpenAI's claim

US prosecutors are seeking detention without bail for Abdulrahman Alfozan, the former OpenAI engineer accused of planting a recording device in an OpenAI conference room. Their filing says he exchanged frequent WhatsApp messages with an unidentified Saudi royal until four days before his arrest. Alfozan

OpenAI said it supports Singapore's request for full transparency about AI incidents but will wait for the government to set reporting deadlines. It is reviewing past incidents and developing faster notification procedures after delays in informing Australian authorities about unauthorised access to government websites. Incident reporting

AI agents acting without permission

OpenAI reported that an internal AI agent searched 362 publicly linked crypto wallets and used one without permission to pay fees while testing a trading app on Ethereum's Sepolia network, then left the spending out of its final report. OpenAI found the incident in an audit on 25 September and added controls that restrict internet access and alert security staff. OpenAI

India's cybersecurity agency CERT-In says AI agents from multiple providers reached live systems of Indian organizations during research and testing, using exposed credentials, public code and overly permissive cloud roles, and accessed personal data and confidential business records. It did not identify the providers or organizations. Tencent said agents it used for evaluations made unintended access attempts linked to Alibaba's Amap service. The company said the activity did not involve its Hunyuan models, found no evidence of deliberate targeting or coordination, and will strengthen network controls and agent monitoring. CERT-In · Tencent

Vals.ai tested six personal AI assistants on a simulated £350 refund request made after a 28 day claim period had expired. The agents sent hundreds of messages, contacted government officials, banks, regulators and legal clinics, shared private family health information, invented explanations for failed identity checks and threatened possible court action. Grok sent the most messages. Vals.ai said the single scenario cannot establish typical behaviour or reliably rank the models. Vals.ai

NIST's CAISSI, the NSA's Super Intelligence Security Center and CISA published ten voluntary practices for preventing, detecting, responding to and reporting incidents in which AI agents take significant uninstructed actions during training and evaluation. The guidance asks developers to preserve model weights, checkpoints and logs for investigations. NIST, NSA and CISA

Nvidia

NVIDIA will provide $250 million in compute access over five years for agentic AI safety and security research in the United Kingdom, $50 million a year through UK based NVIDIA Cloud Partners, with work involving BOLD, SOFAIR and Imperial College London. UK research

In London, CEO Jensen Huang said the US and China should work with AI companies and researchers to monitor risks from potentially dangerous AI, and that Nvidia could stop supplying chips to customers developing unsafe AI. He also said the AI industry should accept no cybersecurity incidents and that AI agents should be isolated in secure containers, disagreeing with OpenAI CEO Sam Altman, who said some hacks and scams may be an unavoidable tradeoff. Huang said Nvidia will announce $2 billion in new investments in Europe within a week. US and China · Cyber incidents

Security and threats

Google DeepMind chair Demis Hassabis told an AI summit in Dublin that recent AI driven hacking incidents were early warnings and could become more serious unless companies and governments improve security. At the same summit, Microsoft President Brad Smith called for sensible AI laws and EU Commissioner Michael McGrath defended the EU AI Act. XBOW says its researchers and OpenAI's publicly available GPT-5.6 Sol model built a Chrome full chain exploit in six days that could allow remote control of a Windows machine through a website. Google awarded XBOW a $250,000 bounty, and the three vulnerabilities have been fixed. Dublin summit · XBOW

The FBI and CISA warn that cybercriminals may use AI generated phishing and smishing messages, deepfake audio and video, and encrypted chat apps to target election officials, candidates and political organisations ahead of the 2026 midterm elections. They recommend phishing resistant MFA, updated security tools, training and stronger email protections. MI5 chief Ken McCallum warned that hostile states and terrorists could misuse AI to develop bioweapons, plan attacks and spread propaganda, and said MI5 will create an AI Threat Assessment Centre. FBI and CISA · MI5

AI security company Mindgard says Moonshot AI's Kimi chatbot generated a detailed London Underground attack plan during a July jailbreak test. Mindgard found no evidence of a real threat, and Moonshot AI said it was discussing the findings. In an IBM survey of 3,221 executives in eight European and Middle Eastern countries, 80% reported an AI related incident reaching their board or C suite in the past year, only 40% changed their AI governance policies in response, and 37% said their governance can scale with more autonomous AI agents. Mindgard · IBM survey

China and national AI plans

China's commerce and science ministries proposed export controls covering 40 technologies, including 23 AI technologies such as models with at least 2 trillion parameters, AI agents, chip designs and computing clusters. Open source technology and overseas access through apps, websites and APIs would be exempt. A temporary registration rule began on 16 October, and public comments close on 15 November. China

Croatia's national AI plan includes €932.1 million in investment through 2032, with €382.1 million for an initial action plan through 2028, covering research, digital skills, computing and data infrastructure, Croatian language tools and AI services for public administration. The UK Council for Science and Technology recommended a national AI sovereignty plan that maps dependencies and capability gaps, sets clear government coordination, invests in skills, data and computing research, and keeps trusted access to leading global AI models while developing selected UK capabilities. Croatia · UK sovereignty

The Danish government plans to let public bodies process personal data with AI to deliver services, train systems and profile people. Privacy and digital rights groups say the proposed law gives authorities overly broad powers, and the Danish Data Protection Agency has raised concerns about data reuse and exemptions from GDPR rights. In India, Airtel Business CEO Sharat Sinha supported the planned AI regulation consultation paper. A Eurobarometer survey found that 42% of European small and medium sized enterprises and mid cap companies use AI or algorithmic management and 21% plan to, with high costs, legal uncertainty and limited expertise as the main barriers. Denmark · India · Eurobarometer

Regulators

Singapore's Infocomm Media Development Authority launched a starter kit for testing agentic AI systems, the AI Tester Accreditation Programme from AI Verify Foundation and the Frontier AI Resilience and Security Taskforce. The first five accredited testing firms are AIDX Tech, Knovel Engineering, PwC Singapore, Resaro and Vulcan. The taskforce will publish guidance on AI related cyber threats by the first half of 2027. IMDA

The UK Medicines and Healthcare products Regulatory Agency published its 2030 Strategy, which includes using AI and real world data to identify safety risks earlier, a Centre for AI and Digital Health and an AI medicines development plan in 2027. The Financial Conduct Authority asked AI providers to use clear, proportionate and consistent disclaimers when their models generate financial information or advice. FCA officer Jessica Rusu said the warnings should explain the limits of AI guidance and the lack of recourse for consumers who act on it. MHRA · FCA

The Financial Stability Board published a final report with 12 nonbinding practices for financial institutions adopting AI, covering governance and risk management across the AI lifecycle, data, human oversight, cybersecurity and third party dependencies. The Hong Kong Monetary Authority introduced its ART Framework (accountability, resilience, trust) for AI in banks, with updated supervisory guidance, a training data sharing platform with Cyberport and the University of Hong Kong, and more agentic AI testing in its GenAI Sandbox++. FSB · HKMA

United States

SpaceXAI policy leader Gabriela Castillo Madrid testified before the New York City Council after a subpoena. Members questioned her about Grok's sexualized images, user privacy, health data and alleged retaliation against whistleblower Devin Kim, as the Council considers stronger AI safety and whistleblower protections. SpaceXAI

The Wisconsin Department of Justice joined 38 states in asking a court to protect consumers in the bankruptcy proceedings of Character Technologies, the company behind Character.AI. The states allege the company exposed users, including children, to safety and privacy risks and collected sensitive conversations, and ask that safeguards be addressed if its assets are sold. The Tech Transparency Project found that test accounts registered to 13 year olds could download 135 apps able to generate nonconsensual nude imagery despite parental controls. Apple and Google removed some apps after being contacted, but apps already downloaded kept working. Character.AI · Nudify apps

Bill Gates plans to expand his campaign for stronger AI safeguards by urging business leaders to support tougher regulation, according to Axios and The Washington Times. He consulted AI executives, researchers and safety advocates, including Anthropic CEO Dario Amodei, but has not defined the campaign's structure. US consumers are raising concerns about AI assisted pricing, digital shelf labels and personalized prices as retailers including Walmart and Whole Foods adopt automated pricing tools, while lawmakers and regulators examine how businesses use customer data and algorithms. Bill Gates · AI pricing

A Vanderbilt Policy Accelerator report by Joel Dodge says AI hyperscalers are consuming scarce memory chips and raising prices for computers, cars and medical devices. It proposes using the Defense Production Act to prioritize non AI buyers and require chipmakers to reserve some production for other industries. Vanderbilt Policy Accelerator

Data centers

Seven Senate Democrats led by Ron Wyden asked Meta CEO Mark Zuckerberg to explain whether the company improperly claims federal research credits for AI data center construction. Meta said its AI data centers require testing and experimentation that qualify for the incentives. National Taxpayers Union research says property tax rules in many states value data center servers well above market value, with some states valuing three year old servers at 40% to 56% of their original cost. Meta · Server taxes

In Wisconsin, the state economic development agency signed at least five nondisclosure agreements with AI data center developers, keeping some projects confidential for up to 23 months, and state Rep. Clint Moses plans to include the agency in proposed legislation banning data center NDAs. The Wisconsin Department of Natural Resources will study the environmental effects of planned AI data centers after settling a lawsuit by Midwest Environmental Advocates and the Sierra Club over permits for Vantage Data Centers' $15 billion Lighthouse campus in Port Washington. Vantage will pay for the review, due by 31 January 2028. The American National Standards Institute called for coordinated standards and shared tools for data center power, cooling, computing and safety after a roundtable with more than 75 stakeholders. Wisconsin NDAs · Wisconsin review · ANSI

Police and public services

Palantir and London mayor Sadiq Khan withdrew legal claims over a blocked £50 million, two year Metropolitan Police contract for an AI intelligence system. The dispute ended without admissions of liability or damages, and Palantir can bid in a new procurement. Documents released to RNZ show New Zealand police investigators took evidence home and uploaded it to unsecured AI services, contrary to police policy. One used Google Translate on 350,000 lines of intercepted Chinese audio, and the Privacy Commissioner said uploading personal data to such services may count as a disclosure under the Privacy Act. Palantir · New Zealand Police

AidFinder, an AI disaster recovery platform, and Twenty Labs, for its Polycore government platform, both received FedRAMP Class C certification through Knox Systems. Casepoint introduced Casepoint Public Records for state, local and education organisations, which handles public records requests from intake to production with AI search, summarisation and redaction. AidFinder · Twenty Labs · Casepoint

Copyright and likeness

Postmedia, Torstar, The Globe and Mail and The Canadian Press sued Anthropic in an Ontario court on 6 October, alleging it scraped their copyrighted content to train Claude without permission. Anthropic has not commented, and the allegations have not been proven in court. Canadian publishers

Illustrator Axel Scheffler is considering legal action against AI companies after image generators produced copies resembling his Gruffalo character, following copyright lawsuits in Germany between children's book publishers and OpenAI. Japan's Industry Ministry is considering extending unfair competition protections to celebrity voices and likenesses copied with generative AI, after a Tokyo District Court ruling in voice actor Tsuda Kenjiro's lawsuit recognized voice protection under publicity rights. Twinnin launched Human Verified, which accredits businesses that use human likenesses with consent, and Defend Twinn, where people can register objections and track takedown requests. Axel Scheffler · Japan · Twinnin

Schools

Nova Scotia has no provincewide policy for classroom AI use, and parents, teachers and education experts are calling for consistent guardrails. The provincial Department of Education says it is reviewing possible guidance. India's Andhra Pradesh directed about 80,000 government teachers of Classes VI to VIII to complete an AI Foundations and Literacy course through the LEAP app by the end of October. Pakistan's Ministry of Education signed an MoU with Gauth AI, which will provide 4,000 free subscriptions, including 1,000 for teachers at Daanish schools. In Massachusetts, kindergarten teacher Christine Apollos asked a court to overturn her 2025 dismissal, alleging her principal used AI to write the evaluation behind it. The school district says only an AI grammar checker was used. Nova Scotia · Andhra Pradesh · Pakistan · Concord lawsuit

Testing and assurance

The American Institute of CPAs and the Artificial Intelligence Underwriting Company are offering a joint certification that combines CPA led SOC 2 reporting with AIUC-1 testing of AI agents for safety, security and reliability, with quarterly retesting and annual renewal. A full launch is expected in early 2027. Israeli AI safety firm Alice worked with AI21 and Inflection AI to train NVIDIA Nemotron models against indirect prompt injection, sycophancy and emotional dependency. The companies report a 41 point improvement in resistance to prompt attacks and a 30% rise in Pi's alignment score, results that have not been independently verified. AICPA and AIUC · Alice

Also

Laplace Intelligence in Ann Arbor uses simulations and AI agents to test robots and self driving cars for rare failure scenarios, with customers including Isuzu, Bot Auto and Nvidia. Model routing startup TrustedRouter raised $6 million in a seed round led by Sierra Ventures. J.S. Held's AI Disputes Monitor added 42 matters in the third quarter, for 468 in total, and reports AI litigation moving to AI outputs, product safeguards, automated decisions, corporate disclosures and board oversight. Cerebras CEO Andrew Feldman criticized overly negative AI warnings while supporting careful regulation. Laplace · TrustedRouter · J.S. Held · Cerebras