Skip to content

Dictation: custom llm rewrite returns llm_error=truncated with empty llm_response by default #218

Open

Description

@YashasviThakur

Found while testing the llm request from the Dictation docs (Dictation API, not SDK-specific).

Request:

config={"llm":{"messages":[{"role":"system","content":"Rewrite as a formal email."},{"role":"user","content":"{{turn}}"}]}}
audio = 8 s WAV: "Um, so can we uh move the the meeting to Thursday at three p m? Also email Doctor Patel about lisinopril."

Result, six runs in a row: llm_response: null, llm_error: "truncated".

Controls (same audio):

  • System prompt "Rewrite the dictated note as a short, formal email." succeeds with a full email.
  • The failing request plus "max_tokens": 1024 succeeds.
  • "max_tokens": 64 returns truncated.

So the server-sized default max_tokens ("Dictation sizes it from the transcript") is too small for prompts that expand text, and a truncated rewrite yields nothing, not even the partial text produced before the cut-off.

Asks:

  1. Raise or floor the default max_tokens when llm.messages/prompt is used.
  2. Return the partial output in llm_response alongside llm_error: "truncated".
  3. Put the "set max_tokens when expanding text" warning in the quickstart, not only in a bullet further down the rewriting page.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions