Hy-SimulMT: Live Translation, Setup and Running Cost

The official family provides Hy-SimulMT-30B-A3B and Hy-SimulMT-30B-A3B-Interleave, with separate FP8 and GGUF variants. The base model outputs translated text. Interleave also outputs source-language transcription, which helps when listeners need both captions.

The model card lists Apache 2.0. Check the selected TTS model’s own terms if you add speech generation. The translation release focuses on Chinese and English in both directions; unrelated Hy-MT2 and Hy-Nomas checkpoints serve different tasks.

For the one-command vLLM path, prepare Linux, Python 3.12+, Git, an NVIDIA GPU and a compatible CUDA driver. The model environment needs vLLM 0.31 or newer with HYV3ForCausalLM support and the simulst plugin. SimulInfer itself can run on a separate CPU machine.

The command below installs dependencies into separate deployment environments, prepares the model and starts the model and web services. The first run downloads the weights. Re-running with --install-server repeats installation, so use the documented existing-environment option when you need to preserve a custom vLLM/CUDA build.

Open the web service at http://127.0.0.1:8080, select the translation direction, then upload an audio file or start a real-time conversation. Stop and finish flushes the remaining translation. Remote microphone access needs HTTPS and a proxy that supports WebSockets.

SimulInfer can send translated text to a configured TTS service and use incoming speech as the voice-cloning reference. Its guide includes a vLLM-Omni example using Qwen/Qwen3-TTS-12Hz-0.6B-Base. Start that service separately, set its connection details in SimulInfer, restart the application and enable speech output.

Standard input cadence uses 1,600 ms blocks. Short-chunk mode keeps a 1,600 ms first block and uses 800 ms for subsequent blocks. End-to-end listening delay also includes translation, queuing, TTS generation and playback buffering. Measure first translated text and first audible output separately.

The 30B-A3B name describes a mixture-of-experts model with roughly 30 billion total parameters and 3 billion active per token. A simple two-byte calculation gives about 60 decimal GB for 30B weights, before audio components, cache and runtime allocations. This arithmetic is a capacity starting point; it does not establish minimum GPU memory.

Budget the model server, audio processing, optional TTS, concurrency and storage together. If the translation and TTS services use separate machines, add both full-instance rates and their billed hours. For an assumed $2/hour translation server plus a $1/hour TTS server used for ten billed hours, compute costs are $30 before storage and network charges. At 300 accepted translation minutes, that is $0.10 per accepted minute. These are chosen budget inputs, not a hosting quote or measured throughput.

A GGUF download needs an audio-capable compatible runtime. The SimulInfer guide says upstream llama.cpp support is pending and recommends waiting for a supporting release. Its bridge connects an already-running model server to the web/API service; it does not add missing model support to an arbitrary llama.cpp binary.

Tencent reports approximately three seconds of StreamLAAL on its RealSI long-form evaluation. StreamLAAL measures translation lag; it is not a guarantee of three seconds to the first audible translated word. A voice-output service adds TTS and playback to the translation path.

For a useful acceptance test, record the checkpoint, runtime, hardware, audio length, input cadence and concurrency. Test names, numbers, interruptions and long utterances in both directions. Judge missing or incorrect meaning alongside lag, then check voice continuity separately when TTS is enabled.