A Teams call with interpreting in both directions

What happens in the call

Two directions run at once. In the incoming direction the caption server transcribes the other party's voice, the translator translates it, and the voice service – the interpreter – speaks it into your headphones while the call's audio is ducked. In the outgoing direction your own speech goes through the same chain into the other party's language and reaches the call through a virtual microphone: they hear the interpreter, preferably in your voice. The cloned voice comes from the voice service's XTTS-v2 model (CPML licence, non-commercial use); starting the services: first start.

What you need

Measured, no loopback hears Teams' audio, so the call comes in through a virtual audio cable: a driver that gives Windows a speaker and a microphone, where whatever plays on the speaker is heard on the microphone. The outgoing direction needs a second cable, because Teams receives the interpreter's voice as a microphone. Headphones are needed too: from speakers the interpreter would get back into the microphone.

Installing the cables

VB-Audio's products are donationware: download the installer packages by hand from the vendor and put them into the repository's Resources folder, where the installers find them. The Audio cable tab's buttons start the installer with elevation. The installer makes the cable the default device; the program restores the headphones, or by hand:

LiveCaption\set-default-audio.cmd -Name "…" -Role all

Setting up Teams and the Usage tab

1. In Teams the speaker is the input of the incoming cable, the microphone is the recording end of the outgoing cable. Leave the secondary ringer on the headphones: without the program running the cable is silent.
2. On LiveCaption's Usage tab choose the "Call – both directions through the call app" mode: it sets the audio source and the two cables. With one cable choose "Call – incoming direction only".
3. The headphones stay the Windows default device and the interpreter's player. The tab's map also lists the Windows defaults, so a cable standing in them shows up there.
4. The "What does the call program use?" section under the map reads back what Teams actually has set: ✓ matches, ✗ differs, ? cannot be checked. The program only reads; it writes nothing into Teams.

Your own voice sample

During the call the outgoing interpreter builds a voice profile from your speech, but only after a few sentences – until then the other party hears a foreign voice. From a sample read out in advance the voice service makes a persistent profile, and the interpreter speaks in your voice from the first word. The recording is in the outgoing section of the Translation tab, after choosing the saved own sample mode:

1. The profile's name: suggested from your Windows user name (for example own-anna); no accented letters.
2. Recording: read the text shown on the screen; it closes by itself after fifteen seconds.
3. The result: length and a quality score; a sample shorter than six seconds cannot be uploaded.
4. Listening: on the Translation tab's playback device, never on the cable.
5. Saving to the voice service: the fingerprint stays on the service's machine (livecaption\voices\<name>.pt).

While the outgoing direction runs, recording is blocked: your microphone goes to the cable, and the other party would hear you reading.

The outgoing direction switch

The call-forwarding icon in the Outgoing direction section's icon row sends the translation of your own speech into the call. The first switch-on asks once (headphones, and what to choose as microphone in Teams); if the cable, the microphone, the language pair or the voice service is not in order, the switch springs back with the reason. When switching off, the program warns you to set the Teams microphone back to the headset – the program cannot do that, and until then the other party hears nothing. The other party's language is decided at switch-on and does not change during the call; if they speak differently, a warning sign (triangle) appears in the Outgoing direction icon row. The outgoing volumes – what the other party hears, and separately what you hear yourself – are set on the section's panel that opens with its gear icon. Hearing the interpreter in your own ear can be turned on on the Translation tab – off by default, and safe only with headphones.

Things worth knowing

• The call's audio in your ear lags by about 0.2 seconds behind the direct path; not disturbing in a conversation.
• The audio route belongs to the settings, not to the session: the call keeps playing during pause, a server outage and reconnection.
• If you do not hear the call, first check whether the program is running; other symptoms are in the troubleshooting guide.