MBZUAI's Institute of Foundation Models releases K2-Horizon, six fully open models up to 375B parameters
The Institute of Foundation Models at MBZUAI has released K2-Horizon, a fleet of six open language models ranging from a 0.9 billion parameter dense model to a 375 billion parameter mixture-of-experts model with 23 billion active parameters.
The models are built for reasoning and agentic work across maths and science, software engineering, deep research and office automation, and the team says they lead or are competitive with strong open models of their size on agentic and coding benchmarks. The release follows the LLM360 definition of open source, publishing intermediate checkpoints from every training stage, pre-training data, training logs and the training, evaluation and infrastructure code, as well as the pipelines that produced roughly 10 trillion synthetic pre-training tokens.
The paper introduces Mixture-of-Value Attention, which applies mixture-of-experts to attention value projections. The team says its 36B-A4B model, which uses it, nearly matches a 32B dense model trained on the same data with one-seventh of the active parameters. It also introduces Uno, adapters that speed up decoding by up to 2.2 times without a separate draft model, according to the paper and AI Weekly.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Daily AI Brief.
Also, consider following us on social media:
Daily AI Brief
Daily report covering major AI developments and industry news, with both top stories and complete market updates
Whitepaper
Tensordyne Napier: What If One Rack Could Do the Work of Nine?
Tensordyne
This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.
Read moreYou may also like
Liquid AI Releases LFM3, Six Open-Weight Models for On-Device AI
Baidu releases 280B parameter ERNIE 5.5 Flash Base
Shanghai AI Laboratory releases Sage-A1.5 for scientific agents
Artificial Analysis Index v5: open-weight models fall 12 points behind
Deep Principle releases Hydrogen-1.0 scientific model weights
Daily AI Brief: the AI news that matters, in your inbox.