Captions and translation for lectures and video

What this mode is for

This is the simplest use: whatever plays on your speakers – a video, a lecture, a recording – the program captions and translates. No virtual audio cable is needed: the loopback listens to your speaker output, and the caption server's live endpoint transcribes the audio sentence by sentence. The caption appears in a transparent, always-on-top window, and stays out of screen sharing.

Two small things worth knowing. The loopback only delivers data while sound is actually playing on the device, so the program starts an inaudible silent playback on the same device next to it – this can be turned off on the Audio source tab. If the audio stalls anyway (muting, a sleeping audio driver), the server closes the open sentence by the wall clock: the caption does not get stuck.

Setting up step by step

1. Start the services with the daily launcher (launch-system.cmd), then the caption window with LiveCaption\start-livecaptionpro.cmd (see the first start guide): the caption server, the translator and – if you want an interpreter too – the voice service should be running.
2. On the settings window's Usage tab choose "The machine's audio (video, talk)" mode. The mode sets the audio source; the other tabs' fields are filled in from it.
3. On the Translation tab turn on translation and set the language pair: the language of the speech and the target language of the translation.
4. In the caption window's Incoming audio section, open the settings panel with the gear icon and choose the Accurate profile (see below).
5. Start the video or the lecture stream – the caption arrives sentence by sentence, the translation under it.

The Accurate segmentation profile

Segmentation decides where the program closes a sentence and when it shows preview text. It has three time values (in milliseconds): the silence that counts as a sentence boundary, the longest section, and the time of the first preview text. On the Incoming audio section's settings panel the three values can be chosen as a profile; the panel's sliders change them even during a running session. The outgoing direction has its own segmentation on the Outgoing direction section's panel.

For a lecture, Accurate is the one: the speaker says longer thoughts, and the translator does better the more context it gets. Measured on a real lecture, speech sections are 9–28 seconds long – that is too long to wait without text, so the default hybrid mode shows preview text after 2 seconds and refreshes it every second. If you change the three values by hand, the picker shows "Custom".

The interpreter with or without headphones

The translated sentence can also be spoken aloud: this is turned on by the speaker icon in the Translation and interpreting section's icon row, the Translation tab or the Ctrl+Alt+I key. The first switch-on asks once: it asks for headphones and tells you that it ducks the playing program's audio while the interpreter speaks, then restores it.

Do not turn it on without headphones. From speakers the interpreter's voice gets back into the loopback, and the program would caption its own interpreter – excluding the process only protects against our own playback, not against sound coming back through the air. If you only want to read the caption, for example in front of an audience listening on speakers, leave it off: the caption and the translation are complete without the interpreter.

The volumes are set on the Translation and interpreting section's settings panel with immediate effect: how much the playback is ducked while the interpreter speaks (Call), and how loud the interpreter is (Interpreter). If you switch to another video, the same panel's relearn button (Ctrl+Alt+V) lets go of the learned voice and learns a new one from the next long enough sample. If the voice service is unavailable, the interpreter stays silent and the fault shows on the status dot – there is deliberately no automatic fallback.

Recordings and SRT export

Every session's text and translation is recorded in the sentence's final state, so the transcript is complete even with translation turned off. One file is made per start:

<adatgyökér>\livecaption\recordings\<dátum>.jsonl

The Recordings window opens from the tray menu's Recordings… item or the clock icon of the small icon row that slides in at the caption window's top right corner. You can search the text and export: bilingual text, an SRT subtitle with the spoken text or with the translation, or a CSV table. The SRT can be put next to the recording if you watch the lecture back later. Recording and the retention period are set on the settings window's Recordings tab: it is on by default and keeps the files for 30 days.

If the caption stalls or the translation lags, the troubleshooting guide lists the symptoms.