Marin releases all 57 pre-training checkpoints of its open 535B model

Nov 22, 2026
Mochi 535B-A23B finished pre-training on 18.2 trillion tokens; the team found benchmark leakage its decontamination had missed.

Open Athena's Marin project has finished the main pre-training phase of Marin "Mochi" 535B-A23B, a mixture-of-experts language model developed in the open, and released all 57 checkpoints from the run on Hugging Face, the team says. The model was trained on 18.2 trillion tokens through context phases of 4K, 16K, 65K and 262K tokens; 10 of the checkpoints come from the long-context extension.

The team says it is still running evaluations and working on post-training. While training, it found substantial leakage from common pre-training benchmarks that its decontamination procedure had missed. Marin says its models are not yet at the capability frontier but researchers increasingly ask to study its work as a proxy for what happens inside closed frontier labs. The checkpoints have no instruction tuning or safety post-training and are meant for scientific study, and the team says it weighed the safety concerns of releasing them against the public interest in studying how large models are trained.

We hope you enjoyed this article

Consider subscribing to one of our newsletters like Daily AI Brief.

Also, consider following us on social media:

Free newsletter

Daily AI Brief

Daily report covering major AI developments and industry news, with both top stories and complete market updates

Market report

2025 Generative AI in Professional Services Report

Thomson Reuters

This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.

Read more
Free, six days a week

Daily AI Brief: the AI news that matters, in your inbox.