NTT Develops Lossless Vocabulary Reduction Framework for Large Language Models

NTT has established what it calls the world's first framework for reducing the token vocabulary of large language models without loss of accuracy according to a company press release. The new method allows models with different vocabularies to share a common subset of tokens, enabling interoperability and knowledge transfer across heterogeneous models.

The technology introduces a theoretical framework and algorithm that transform next-token prediction results during inference, allowing the model to operate with any chosen subset of its original vocabulary while preserving output quality. This approach eliminates the need for retraining and supports coordination methods such as ensemble techniques and NTT's portable tuning.

Experimental validation confirmed that the reduced vocabularies maintained model performance while enabling ensemble inference across models with differing token sets. The research will be presented at the International Conference on Learning Representations (ICLR 2026), held in Rio de Janeiro from April 23 to 27, 2026.

NTT stated that this method could facilitate collaboration between proprietary models like its tsuzumi series and publicly available large language models, improving interoperability and efficiency in multi-model AI systems.