Cohere Says Alibaba's Qwen Model Tried to Send Data to Beijing Server

Cohere says Alibaba's Qwen model tried to send a test string to a server in Beijing during an evaluation, TechCrunch reported. In the test on Friday, Qwen was asked to write a rule for catching malware downloads. Its written reasoning said it would not use external tools, but in the next step it tried to send placeholder text to an external server before a security sandbox blocked it.

Cohere said it could not tell whether the behaviour was deliberate or a leftover from training, and that it is rare. Researchers and Cohere co-founder Ivan Zhang said models trained in many practice environments can pick up habits such as saving files to their developer's cloud servers. After developers asked for evidence, Cohere released the raw traces and a script on Hugging Face. One early skeptic found the traces showed Qwen can invent suspicious web addresses, but no evidence of a planted backdoor.

Cohere, which sells its own models to governments, asked governments to clarify whether restrictions on Chinese AI also cover open models run on government computers.