Nemotron 3.5 Transcribe: Setup, Memory and Audio Costs

Transcribe supports 40 transcription languages, translation between English and 31 languages, and speaker attribution in 13. Audio question answering is trained in English. Check the task-by-language matrix before selecting it for a multilingual workflow; support for transcription does not imply every other task is supported in that language.

The model handles recordings up to one hour and up to eight speakers, with a 65,536-token context. It accepts audio plus text instructions and returns text. For a live voice assistant that needs streaming speech output, use a model designed for that interaction; Transcribe’s release focuses on recorded audio.

Start with a short file whose words and speakers you can check. Try a longer recording only after names, speaker changes and the requested transcript style look right. This makes quality review manageable before you allocate a large GPU batch.

The 4.3B active-parameter figure includes the audio encoder and describes computation per token. The model card lists approximately 34B total parameters across the full model. A simple 34B × 2-byte BF16 calculation is about 68GB in decimal units, or 63.3GiB, before runtime allocations, audio features, state and batching. It is a weight-only estimate, not a minimum-VRAM measurement.

NVIDIA’s BF16 quick start uses one H100 80GB. Begin with its model-specific vLLM and NeMo Speech environment, then measure peak allocated memory with the intended audio length and concurrency. Test hour-long recordings and concurrent requests separately before sizing a multi-user service.

The current model card links FP8 and NVFP4 variants, but those repositories were not publicly retrievable on October 29. In discussion 8, shuuul reports the same availability problem and asks for a publication date. Plan a reproducible deployment around the accessible BF16 checkpoint until you can obtain the intended quantized artifact and its instructions.

For a serving endpoint, follow NVIDIA’s environment setup: vLLM 0.32.0 plus the NeMo Speech plugin in one environment. Its BF16 launch recipe includes the audio-duration limits and timestamp middleware. A generic text-model vLLM command omits parts of this setup.

For in-process use, the model card also gives a Transformers path. Install Transformers from its current source as the guide specifies, along with a matching PyTorch and audio-decoding environment. After that setup, the following example downloads the BF16 checkpoint and transcribes a local recording named audio.wav. Allow storage and memory for the full model.

The printed text is the recognized transcript; compare it against the recording. The example limits generation to 256 new tokens, so use a short clip first and increase the output budget for longer content. For speaker labels, translation and word timestamps, use the corresponding task recipe rather than assuming the transcription helper changes its output automatically.

NVIDIA’s release evaluates transcription, long-form audio, multilingual speech and speaker attribution separately. Word error rate measures recognition mistakes; diarization error measures speaker-time assignment; speaker-attributed word error also depends on matching words to people. A strong result on one does not settle the others for your recordings.

The model card’s highest serving-throughput example uses an optimized Nemotron Transcribe NIM on an RTX PRO 6000 Blackwell Server Edition, with NVFP4 and DFlash2. NVIDIA says those serving optimizations are outside the open-source release and expects the release-day NeMo/vLLM integration to be roughly twice as slow. That NIM result is not a latency promise for the downloadable BF16 example.

Measure wall-clock time from accepted audio to a complete usable transcript, including decoding and retries. Keep batch audio-seconds per second separate from one recording’s response latency. Check accented speech, overlap, silence and vocabulary from the workload you actually plan to process.

The checkpoint is governed by OpenMDW-1.1. Downloading weights gives you a self-hosting route; GPU rental or ownership, storage and transcript review remain costs. The cited release does not give a hosted per-minute tariff, so there is no provider price to substitute for a measured deployment budget.

For a hypothetical batch of 100 audio hours, assume your measured complete processing job occupies a GPU for two hours at $2 per GPU-hour. Compute costs $4, or $0.04 per input audio hour. If setup and retries occupy another 30 minutes, the total becomes $5, or $0.05 per input audio hour. These are assumed inputs, not NVIDIA performance or rental quotes.

Track accepted output as well as input volume. If only 90 of those 100 hours meet your quality threshold, that $5 compute bill becomes about $0.056 per accepted audio hour before storage and human review. For a small occasional batch, setup time may outweigh faster batch throughput.