How it works
Everything happens on your own machine or on a server you run: the audio never leaves your network. This page follows the sound to the interpreter's voice, and shows what we measured on the way.
From sound to voice, in four steps
- Live Caption Pro – the caption window on your machine
- AccountServer (port 8100) – the account service: users, roles, API keys, audit log and usage metering
- TranslatorServer (port 8080) – the translator: translates each sentence with TranslateGemma
- VoiceServer (port 8090) – the voice service: speaks the translation, optionally in the speaker's cloned voice
- CaptionServer (port 5111) – the caption server: splits the audio into sentences and transcribes it (Whisper)
Capture
The application records what you hear – the other party in a call, a video, a lecture – through Windows audio, or a single application's sound on its own. It watches for silence: a pause closes a sentence, so the caption server gets whole thoughts, not fixed slices.
Transcribe
The caption server runs Whisper on the graphics card and returns each sentence as text with timestamps. It recognizes the language on its own or keeps to the one you set; with several possible languages it votes over the whole session, so one short sentence cannot switch it.
Translate
The translator runs a TranslateGemma model with llama.cpp and translates each sentence as soon as it closes. The first word arrives within a tenth of a second, so the translation shows under the original almost as it is spoken.
Speak
If you want the interpreter heard, the sentence goes to a voice: the speaker's cloned voice or a built-in one from the voice service, or a Piper voice inside the application without a graphics card. The interpreter never falls behind for long: when sentences pile up, it speeds up a little and skips the oldest ones rather than lagging minutes behind.
Two directions
Incoming
The other party's words appear as captions and a translation in a transparent band over any window, and the interpreter speaks them into your headphones. The application excludes its own process from the capture, so it does not caption the interpreter.
Outgoing
Your microphone is transcribed and translated the same way, and the interpreter's voice goes into a virtual microphone (VB-CABLE) that you select in Teams, Zoom or any other call application. The other party hears the translation, in your voice.
Three voices, one setting
- Cloned: the voice service learns the speaker from a few seconds of their speech and says the translation in that voice. The best for calls – the other party hears you, not a robot.
- Built-in: one of the voice service's speakers, without cloning – steady and predictable.
- In-process: a Piper voice inside the application, on the CPU. The fastest, and it works without the voice service and without a graphics card.
There is no silent fallback: if the chosen voice is not available, the interpreter stays quiet and the window says why.
Languages and voices
| What we measured | Result | Under what conditions |
|---|---|---|
| Languages the caption server recognizes | 99 | Whisper; automatic detection, or a fixed language and a candidate list. |
| Languages of the voice service | 17 | XTTS-v2, for cloned and built-in voices alike. |
| Built-in voices of the voice service | 58 | XTTS-v2 speakers; a cloned voice comes on top of these, from a few seconds of the speaker. |
| Languages of the in-process voice | 42 | Piper voices, 50 language variants; no graphics card needed. |
| In-process voices | 185 | Piper models. |
Measured in detail
| What we measured | Result | Under what conditions |
|---|---|---|
| First translated word | 62 ms | TranslateGemma 12b Q8, 20 sentences; the whole sentence in 270 ms, 19 of 20 flawless. For comparison, the cloud model gpt-4o-mini took 1106 ms to the first word. |
| First translated word, 95th percentile | 71 ms | The same 20 sentences; 19 of 20 were faster than this. |
| Whole translated sentence | 270 ms | TranslateGemma 12b Q8, 20 sentences, median; none cut short. |
| Whole translated sentence, 95th percentile | 452 ms | The same 20 sentences. |
| Flawless translations | 19 of 20 | TranslateGemma 12b Q8, the same 20 sentences. |
| First word from a cloud model, for comparison | 1106 ms | OpenAI gpt-4o-mini, median, the same sentences; the local 12b model answered in 62 ms. |
| Transcribing 6.3 s of speech | 108 ms | Whisper large-v3-turbo on the GPU, through the caption server. |
| Round trip to a remote caption server | 406 ms | 6.3 s of audio, Quadro RTX 6000 on another machine; locally, on the RTX 5090, it took 108 ms. |
| Network cost of a remote caption server | 1 ms | 6.3 s of audio over the local network; the rest of the round trip is the server's own work. |
| Sentence end to the interpreter's first sound – in-process voice | 0.99 s | Median of the 24 spoken sentences (13 of 37 skipped under load); 95th percentile 5.56 s. Piper voice, no GPU needed for the voice. |
| Sentence end to the interpreter's first sound – in-process voice, 95th percentile | 5.56 s | The same 37 sentences. |
| Sentences spoken – in-process voice | 24 of 37 | Three minutes of continuous speech with the product defaults; the rest were skipped so the interpreter keeps up. |
| Sentence end to the interpreter's first sound – cloned voice | 2.93 s | Median of the 22 spoken sentences (16 of 38 skipped under load); 95th percentile 6.86 s. XTTS-v2 in the voice service. |
| Sentence end to the interpreter's first sound – cloned voice, 95th percentile | 6.86 s | The same 38 sentences. |
| Sentences spoken – cloned voice | 22 of 38 | The same three minutes; a cloned voice takes longer to synthesize, so more sentences are skipped under load. |
| Speaking one sentence – in-process voice | 84 ms | Piper model on the CPU, inside the application. |
| Speaking one sentence – cloned voice, to the first audio chunk | 282 ms | XTTS-v2 in the voice service; the audio is streamed, so the interpreter starts before the whole sentence is synthesized. |
| Residual of the interpreter's own voice in the captured audio | −97 dBFS | The application excludes its own process from loopback capture, so the interpreter does not caption itself. |
| Video memory for the three engines together | 22.3 GB | Measured on one machine with the largest translator model. On a 20 GB card the translator runs as 12b Q4_K_M (measured 7.5 GB), on a 32 GB card as 12b Q8_0. |
| Video memory of the translator with the smaller model variant | 7.5 GB | TranslateGemma 12b Q4_K_M; the Q8 variant needs 16.7 GB. |
| Video memory of the voice service | 2.3 GB | XTTS-v2; the in-process voice needs none. |
Measurement machine: NVIDIA GeForce RTX 5090, Intel Core i9-13900K, 128 GB RAM; September 2026. Other machines give other numbers; this page publishes no numbers measured on a Mac. The interpreter timings are medians (p50): half of the sentences were faster; the 95th percentile and the share of sentences skipped under load are on the How it works page.
Every number comes from the measurement records in the source repository – the site keeps the source file and a verbatim quote for each –, and a test checks that the page and the record still agree. p50 is the median: half of the sentences were faster. p95 is the value 19 of 20 sentences stayed under. "Skipped" means the interpreter dropped the sentence to keep up.
Where the parts run
The application runs on your Windows machine or Mac. The four services can run on the same machine or on any other one on your network; the application only needs their addresses and a key. The video memory the three engines need together is in the table above. On a smaller card the translator picks a smaller model variant – or you move the services to the machine that has the card, and then the captioning machine needs none.