Tencent open-sources Hy4, a 770 billion parameter multimodal model
Tencent has open-sourced the weights of Hy4, its new-generation flagship mixture-of-experts model, on Hugging Face, ModelScope and GitCode, along with an FP8 quantised version and the pre-trained Hy4 base model.
Hy4 supports text, image and video understanding. Its language backbone has 770 billion total parameters, of which 49 billion are activated per token, across 78 layers with 256 routed experts per MoE layer, and it has a context length of 1 million tokens. A built-in 3.3 billion parameter drafter called DFly2 supports speculative decoding, and the attention design draws on DeepSeek and GLM, according to the model card.
The model card lists use cases including vision and computer use, frontend and game development, finance and office workflows, scientific research and long-horizon personal assistance. Hy4 can be served with vLLM or SGLang through an OpenAI-compatible API and is released under the Tencent Hy License, the Hugging Face model card says.
We hope you enjoyed this article
Consider subscribing to one of our newsletters like Daily AI Brief.
Also, consider following us on social media:
Daily AI Brief
Daily report covering major AI developments and industry news, with both top stories and complete market updates
Whitepaper
Tensordyne Napier: What If One Rack Could Do the Work of Nine?
Tensordyne
This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.
Read moreYou may also like
Tencent releases Hy4 flash, a 224B-parameter model with 12B active parameters
MBZUAI's Institute of Foundation Models releases K2-Horizon, six fully open models up to 375B parameters
Baidu releases 280B parameter ERNIE 5.5 Flash Base
Tencent and Inferact Partner on Hy4 Serving and vLLM Support
Tencent Releases Hy-Nomas for Interactive Video World Generation
Daily AI Brief: the AI news that matters, in your inbox.