Deepgram Runs Nova-3 on AWS Trainium2 With Up to 70% More Audio per Dollar

Oct 19, 2026
Deepgram ported its Nova-3 speech recognition model to AWS Trainium2 for offline batch workloads. The resulting setup processed 11% to 70% more audio per dollar than its previous compute options.

Deepgram and Amazon Web Services ported the Nova-3 speech recognition model to Trainium2 for offline batch workloads, as detailed in an AWS blog post. A single Amazon EC2 trn2.3xlarge instance ran eight model replicas and processed 11% to 70% more audio per dollar than the compute options Deepgram previously used.

The teams compiled the model with TorchNeuron and replaced several layers with optimized kernels. They also separated the encoder and decoder into static graphs and checked model accuracy against CPU results and word error rate targets.

Deepgram built a Rust runtime that executes compiled model files through the Neuron Runtime API without Python or PyTorch in the serving path. At the batch size with the highest throughput, the Rust runtime delivered 7.5% more throughput than a Python multiprocessing baseline.

We hope you enjoyed this article

Consider subscribing to one of our newsletters like Silicon Brief or Daily AI Brief.

Also, consider following us on social media:

AI Chips & Datacenters AI Brief AI Brief (X)
Free newsletter

Silicon Brief

Weekly coverage of AI hardware developments including chips, GPUs, cloud platforms, and data center technology.

AI Token Webinar

Whitepaper

Tensordyne Napier: What If One Rack Could Do the Work of Nine?

Tensordyne

This Tensordyne whitepaper presents Napier, an inference-focused AI processor and rack-scale system based on the company’s TDN Math logarithmic number system. It examines infrastructure requirements for large mixture-of-experts and agentic models, compares major inference architecture approaches, and details the TDN AIP processor, TDN72 pod, TDN Link fabric, and Napier Ultra configuration. The paper reports simulation-based performance, cost, and accuracy-validation results, including Tensordyne’s projected comparison of one Napier rack with a nine-rack Nvidia Rubin plus Groq deployment; the chip is reported as taped out and in fabrication.

Read more

You may also like

Free, six days a week

Daily AI Brief: the AI news that matters, in your inbox.