Google Releases LiteRT Tensor API for On Device Model Authoring

Oct 19, 2026
Google released the LiteRT Tensor API, a C++ authoring layer that lets developers build and optimize model graphs for deployment across CPUs, GPUs, and NPUs.
Google Releases LiteRT Tensor API for On Device Model Authoring
Image: By Anthony Quintano from Mount Laurel, United States

Google released the LiteRT Tensor API, a C++ authoring layer for on device models, as detailed in a company blog post. Developers can define computation graphs one operation at a time and run the same graph through LiteRT on CPUs, GPUs, and NPUs, or directly on CPUs through XNNPACK.

The API can load weights from checkpoints such as Hugging Face .safetensors files without intermediate conversion scripts. It also connects preprocessing, inference, and postprocessing through shared accelerator buffers, keeping intermediate data off the CPU.

Google demonstrated the API with a browser application using Meta's Segment Anything Model 2 on WebGPU. A coding agent modified the C++ implementation and increased performance from fewer than 10 to more than 30 frames per second at a resolution of 512 by 512. In Google's tests, the same model code ran about 1.5 times faster than Apple MLX on a MacBook Pro.

We hope you enjoyed this article

Consider subscribing to one of our newsletters like Daily AI Brief.

Also, consider following us on social media:

AI Brief AI Brief (X)
Free newsletter

Daily AI Brief

Daily report covering major AI developments and industry news, with both top stories and complete market updates

AI Token Webinar

Whitepaper

Tensordyne Napier: What If One Rack Could Do the Work of Nine?

Tensordyne

This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.

Read more

You may also like

Free, six days a week

Daily AI Brief: the AI news that matters, in your inbox.