Marin releases all 57 pre-training checkpoints of its open 535B model
Open Athena's Marin project has finished the main pre-training phase of Marin "Mochi" 535B-A23B, a mixture-of-experts language model developed in the open, and released all 57 checkpoints from the run on Hugging Face, the team says. The model was trained on 18.2 trillion tokens through context phases of 4K, 16K, 65K and 262K tokens; 10 of the checkpoints come from the long-context extension.
The team says it is still running evaluations and working on post-training. While training, it found substantial leakage from common pre-training benchmarks that its decontamination procedure had missed. Marin says its models are not yet at the capability frontier but researchers increasingly ask to study its work as a proxy for what happens inside closed frontier labs. The checkpoints have no instruction tuning or safety post-training and are meant for scientific study, and the team says it weighed the safety concerns of releasing them against the public interest in studying how large models are trained.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Daily AI Brief.
Also, consider following us on social media:
Daily AI Brief
Daily report covering major AI developments and industry news, with both top stories and complete market updates
Market report
2025 Generative AI in Professional Services Report
Thomson Reuters
This report by Thomson Reuters explores the integration and impact of generative AI technologies, such as ChatGPT and Microsoft Copilot, within the professional services sector. It highlights the growing adoption of GenAI tools across industries like legal, tax, accounting, and government, and discusses the challenges and opportunities these technologies present. The report also examines professionals' perceptions of GenAI and the need for strategic integration to maximize its value.
Read moreYou may also like
MBZUAI's Institute of Foundation Models releases K2-Horizon, six fully open models up to 375B parameters
Meituan releases LongCat-2.5, a 1.6 trillion parameter model for agentic coding
OpenAI Expands Monitoring for Frontier Model Misalignment
OpenAI embeds six Apollo Research evaluators to test whether models try to subvert safety controls
OpenAI Resumes Frontier Model Training After Containment Upgrades
Daily AI Brief: the AI news that matters, in your inbox.