You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Dictation: custom llm rewrite returns llm_error=truncated with empty llm_response by default #218
Found while testing the llm request from the Dictation docs (Dictation API, not SDK-specific).
Request:
config={"llm":{"messages":[{"role":"system","content":"Rewrite as a formal email."},{"role":"user","content":"{{turn}}"}]}}
audio = 8 s WAV: "Um, so can we uh move the the meeting to Thursday at three p m? Also email Doctor Patel about lisinopril."
Result, six runs in a row: llm_response: null, llm_error: "truncated".
Controls (same audio):
System prompt "Rewrite the dictated note as a short, formal email." succeeds with a full email.
The failing request plus "max_tokens": 1024 succeeds.
"max_tokens": 64 returns truncated.
So the server-sized default max_tokens ("Dictation sizes it from the transcript") is too small for prompts that expand text, and a truncated rewrite yields nothing, not even the partial text produced before the cut-off.
Asks:
Raise or floor the default max_tokens when llm.messages/prompt is used.
Return the partial output in llm_response alongside llm_error: "truncated".
Put the "set max_tokens when expanding text" warning in the quickstart, not only in a bullet further down the rewriting page.
Activity