Nemotron 3.5 ASR ONNX

[!Note] This model is the quantized INT4 ONNX version of nvidia/nemotron-3.5-asr-streaming-0.6b, adding language-ID prompt conditioning to support transcription across 40 language-locales from a single model. It supports streaming inference with 0.56 seconds of latency and is simshipped alongside the baseline NVIDIA model.

Nemotron 3.5 ASR is a multilingual, streaming Automatic Speech Recognition (ASR) model engineered to deliver high-quality multilingual transcription across both low-latency streaming and high-throughput batch workloads. Developed by NVIDIA, this 600M parameter model transcribes speech into text with native support for punctuation and capitalization, and offers runtime flexibility with configurable chunk sizes, including 80ms, 160ms, 320ms, 560ms, and 1120ms. This ONNX model was exported with optimization for the 560ms chunk size.

By leveraging a state-of-the-art Cache-Aware FastConformer-RNNT architecture, the model eliminates redundant overlapping computations common in traditional "buffered" streaming. This allows it to process only new audio chunks while reusing cached encoder context, significantly improving computational efficiency and minimizing end-to-end delay without sacrificing accuracy.

It was trained on a massive ASR dataset and is engineered to perform across diverse and challenging acoustic conditions.

This model is ready for commercial use.

This model is for transcription of multilingual audio.

Release Date

• Hugging Face [06/04/2026] via https://huggingface.co/onnx-community/nemotron-3.5-asr-streaming-0.6b-onnx-int4

Why Choose Nemotron 3.5 ASR?

• 🌍 Single Multilingual Model: Transcribes 40 language-locales from one model through language-ID prompt conditioning, with optional automatic language detection.
• ⚡ Native Streaming Architecture: Cache-aware design enables efficient processing of continuous audio streams, designed and optimized for low-latency voice agent applications.
• 💰 Improved Operational Efficiency: Delivers superior throughput compared to traditional buffered streaming approaches. This allows for a higher number of parallel streams within the same GPU memory constraints, directly reducing operational costs for production environments.
• 🎛️ Dynamic Runtime Flexibility: Choose the optimal operating point on the latency-accuracy Pareto curve at inference time. No re-training is required to adjust for different use-case requirements.
• 📝 Punctuation & Capitalization: Built-in support for punctuation and capitalization in output text.

Supported Languages

The model supports 40 language-locales in total, across three tiers:

• Transcription-ready (19 locales): highest-accuracy ASR, ready out of the box.
• Broad-coverage (13 locales): production ASR across an additional 13 locales.
• Adaptation-ready (8 locales): recognized by the tokenizer; fine-tune on in-domain data to unlock full transcription.

Note: Transcription-ready and broad-coverage locales (32 total) produce ASR transcription out of the box; adaptation-ready locales require fine-tuning on in-domain data to enable full transcription. The model supports uppercase and lowercase letters, punctuation, spaces, and apostrophes.

How to Use this Model

The model is available for use in via https://github.com/microsoft/onnxruntime-genai.

Results (for the ONNX model)

Questions and issues

For any questions open an issue or send an email to Nenad Banfic (nebanfic@microsoft.com).