Novita AI and SGLang Partner to Enhance AI Inference

Novita AI has announced a strategic partnership with SGLang to enhance AI inference capabilities, announced in a press release. Novita AI will supply high-performance GPU cloud resources to support SGLang's research, benchmarking, and optimization efforts.

SGLang is known for its fast-serving engine for large language and vision-language models, which includes innovations like RadixAttention cache reuse and zero-overhead batch scheduling. This collaboration aims to empower developers to build complex generation workflows and multi-modal applications with improved reliability and scale.

Novita AI's contribution has already facilitated the development of SGLang's multi-turn reinforcement learning framework and the Prism multi-large language model serving system. The partnership also involves collaboration on SGLang's large-scale expert parallelism project, an open-source initiative to meet throughput benchmarks outlined in the DeepSeek blog.

This partnership underscores Novita AI's commitment to advancing an open ecosystem of inference engines and supporting diverse research initiatives through shared infrastructure and joint development efforts.