Skip to content

Short dictations lose their cleanup when the rewrite's letters appear inside an instruction word ("Done." matches "abandoned") #330

Open

Description

@Pulkit7070

Summary

RewriteGrounding.isSpeech rejects a rewrite if its normalized text appears anywhere in the normalized instruction. The check is a plain substring test over characters, not words, so a short rewrite is thrown away whenever its letters happen to sit inside a longer word of the instruction. The verbatim transcript, fillers and all, is pasted instead.

With default settings (no custom style, Fast or Smart), saying "Um, done." pastes Um, done. because the base instruction contains "abandoned". The same happens for "Uh, ten." ("content"), "Um, sent." ("sentence") and "Uh, OK." ("okay"). With a custom style the user's style is dropped for these replies too: under "Write everything in ALL CAPS." the reply "HI." is rejected because "everything" is in the style text, so hi is pasted.

The type's own doc says a spoken phrase that sits inside the instruction costs that utterance its cleanup. The current check goes further than that: it also matches fragments inside unrelated words.

Introduced with RewriteGrounding in #315 (#315). The same code is in open PR #311 (#311).

Code:

if rules.contains(words) { return false }
return !sentences(of: instruction).contains { words.contains($0) }

Repro

Add this test to Tests/BlurtEngineTests/ on main (6739585) and run DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift test --filter RewriteGroundingSubstring:

import Testing
@testable import BlurtEngine

@Suite("RewriteGroundingSubstring")
struct RewriteGroundingSubstringTests {
  @Test(arguments: [RewriteMode.fast, .smart])
  func shortRepliesKeepTheirCleanup(mode: RewriteMode) {
    let instruction = CleanupInstruction.sendable(appending: nil, mode: mode)
    for (verbatim, rewrite) in [("Um, done.", "Done."), ("Uh, ten.", "Ten."), ("Um, sent.", "Sent.")] {
      #expect(
        AssemblyAITranscriber.chooseTranscript(
          verbatim: verbatim, rewrite: rewrite, enhanced: true, instruction: instruction) == rewrite)
    }
  }

  @Test func styleAppliesToAShortReply() {
    let instruction = CleanupInstruction.sendable(appending: "Write everything in ALL CAPS.", mode: .fast)
    #expect(
      AssemblyAITranscriber.chooseTranscript(
        verbatim: "hi", rewrite: "HI.", enhanced: true, instruction: instruction) == "HI.")
  }
}

Expected

Done., Ten., Sent. and HI. are pasted. None of them is a read-back of the instruction.

Actual

All four expectations fail on main. Output from the same inputs:

mode=fast  verbatim=[Um, done.] rewrite=[Done.] pasted=[Um, done.]
mode=fast  verbatim=[Uh, ten.]  rewrite=[Ten.]  pasted=[Uh, ten.]
mode=fast  verbatim=[Um, sent.] rewrite=[Sent.] pasted=[Um, sent.]
mode=smart verbatim=[Um, done.] rewrite=[Done.] pasted=[Um, done.]
style=ALL CAPS, fast: rewrite=[HI.] pasted=[hi]
style=ALL CAPS, fast: rewrite=[DONE.] pasted=[done]

Suggested fix

Compare on word boundaries. Both strings are already normalized to single-spaced words, so padding them with spaces is enough:

if " \(rules) ".contains(" \(words) ") { return false }
return !sentences(of: instruction).contains { " \(words) ".contains(" \($0) ") }

With that change the four cases above pass, real echoes (a whole instruction sentence, or a phrase like "no" or "Hi" that the instruction actually lists) are still rejected, and the existing AssemblyAITranscriber, TextShortcut, InsertionSeparator, KeyInjector and STTPrompt suites (123 tests) still pass. A regression test with "Done." against the base instruction would pin it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions