Anthropic and OpenAI negotiate publication rules for AI safety reviews

Anthropic and OpenAI have sought significant control over what outside AI safety evaluators can publish, concerned that findings could reveal trade secrets, Reuters reports in an exclusive citing two people familiar with the discussions. According to those people, both companies were seeking strict editorial control as recently as October 5; by October 8, talks between Anthropic and some evaluators appeared to have evolved, one of them said. The reporting follows Anthropic's plan for embedded third party evaluators and its CEO's call last month for evaluators to publish key findings "without editorial control."

Anthropic told Reuters that seeking significant control did not accurately characterize all of its discussions, and that it had made clear to evaluators it previously worked with that they control their methods, evaluation criteria, conclusions and what to publish. For new evaluator firms, it said it reserves the right to comment on and approve reports. OpenAI said it was not seeking approval rights over the full text of reports, that its standard practice is to redact confidential information, and that evaluators can disclose whether major redactions were made.

According to a contract seen by Reuters, language the labs already use with some evaluation firms can bar them from publishing confidential information without the companies' consent. Measures under discussion to prevent intellectual property leaks include having lab employees observe embedded teams and preventing evaluators embedded at one lab from working at another at the same time, one person said.

Reuters also reports that OpenAI redacted nonpublic information from a METR and Redwood Research report on the Hugging Face incident. The two nonprofits said they edited the report's structure, emphasis, clarity and tone based on OpenAI's feedback, but that nothing else important to their conclusions was removed.