Repository navigation
fix(voice): end a turn on a late STT final after a later END_OF_SPEECH - #7739
Merged
chenghao-mou merged 4 commits intoOct 30, 2026
Merged
Conversation
With turn_detection="stt", a final transcript only ran end-of-turn detection while _user_turn_committed was set, and every commit cleared it. When the user resumed between an END_OF_SPEECH and its final, the next END_OF_SPEECH committed the earlier text, and that utterance's final, arriving after the commit, waited for another END_OF_SPEECH, leaving every later turn one utterance late. Track the end of a turn in STT mode separately (_stt_turn_ended, set when a turn ends and cleared on START_OF_SPEECH) and let a final run end-of-turn detection while it is set. _user_turn_committed is unchanged, so the VAD end-of-speech path and the other modes behave as before. Fixes livekit#7699
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
_stt_turn_ended was set whenever a turn ended, so a VAD-driven commit armed it for an utterance the user had already resumed, and clear_user_turn left it set. Set it on the STT's END_OF_SPEECH instead, and clear it on START_OF_SPEECH, on clear_user_turn and when the turn detection mode changes.
Contributor
Author
|
Hi @longcw , @chenghao-mou this PR is ready for review and all checks are passing. Could you please review it when you have a chance? Thank you! |
3 tasks
chenghao-mou
approved these changes
Oct 30, 2026
chenghao-mou
left a comment
Member
There was a problem hiding this comment.
lgtm, thanks for the PR!
iamdgarcia
added a commit
to iamdgarcia/agents
that referenced
this pull request
Oct 31, 2026
Rebased onto main (post livekit#7285, livekit#7739). Un-stacked from the silero VAD branch (livekit#7735) so this PR carries only its own changes. Pipelines with manual_flush keep the livekit#7285 flush-first path; the reuse shortcut applies to plain streaming pipelines where a second final will never arrive.
This was referenced Oct 31, 2026
chenghao-mou
added a commit
that referenced
this pull request
Nov 2, 2026
Port #7795 and #7796 onto the shared STT connection coordinator, with #7739 that they build on. Each provider connection restarts the input clock. RTZR detects idle in the send loop, and Gladia and Reson8 re-anchor each session on its first frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7699
With
turn_detection="stt", a final transcript only runs end-of-turn detection while_user_turn_committedis set, and every commit clears it. If the user resumes between anEND_OF_SPEECHand itsFINAL_TRANSCRIPT(Sarvam realtime sends END a few hundred ms before the final), the nextEND_OF_SPEECHcommits the earlier text. That utterance's final then arrives after the commit, finds the latch cleared and waits for anotherEND_OF_SPEECH, so every later turn is committed one utterance late.Change: track the end of a turn in STT mode with a separate flag,
_stt_turn_ended. It is set when a turn ends, cleared onSTART_OF_SPEECH, and lets a final that arrives in between run end-of-turn detection._user_turn_committedis unchanged, so the VAD end-of-speech path and the other turn detection modes behave as before. The flag is only read in STT mode, so switching to manual turn detection right after a turn doesn't let a final end the next one.Tests (events sent through
_on_stt_event, which every STT stream event passes through):test_stt_final_after_a_later_end_of_speech_still_ends_a_turnreplays the sequence from the issue. With no endpointing delay, the late final commits as its own turn; onmainit is stranded. With a 0.5 s delay it joins the pending turn. In both cases the next turn commits on its ownEND_OF_SPEECH.test_stt_pending_end_of_speech_survives_a_resume: anEND_OF_SPEECHnot yet committed still lets a delayed VAD end commit its final when the user resumes (same asmain).test_stt_turn_is_not_ended_by_vad_after_the_user_resumes: after the user resumes, a final that follows a VAD pause doesn't end the turn early.test_stt_turn_end_does_not_carry_into_manual_turn_detection: after switching to manual turn detection, a final doesn't end the turn.Removing any one part of the change fails one of these tests.
test_transcription_delay_anchor.pybuildsAudioRecognitionby hand, so its helper sets the new attribute.