Skip to content

DictationConfig cannot send the documented llm field (custom messages, aai_prompt, max_tokens) #217

Open

Description

@YashasviThakur

Docs ("Transcript rewriting → Sending your own LLM request") say config.llm accepts an LLM Gateway request: messages/prompt with {{turn}}, model, max_tokens, temperature, response_format, reasoning_effort, or a saved prompt via aai_prompt + prompt_replacements.

The API accepts it. Verified against dictation.assemblyai.com/v1/transcribe/live: {"llm":{"messages":[...]}} returns 200 with a rewrite, and llm + llm_instruction returns 400 "set either llm_instruction or llm, not both".

This SDK cannot send it. buildConfigJson in src/services/dictation/service.ts copies a fixed field list and drops everything else, and DictationConfig has no llm (its comment says "the dictation API accepts this exact set"). A TS user gets a type error, or with a cast silently nothing is sent.

Impact: saved prompts (aai_prompt), model choice, max_tokens, temperature and response_format are only usable by hand-rolling multipart, even though the docs steer people to the SDK.

Suggested fix: add llm?: {...} typed to the documented field set, enforce mutual exclusion with llm_instruction, and update the "exact set" comments. The Python SDK has the same gap (DictationConfig uses extra="forbid" with no llm field).

Activity

  1. YashasviThakur commented on Nov 19, 2026

    @YashasviThakur
    Author

    Adding context from the "Authenticate your app" docs page: it tells apps that run on users' devices to keep rewrite prompts off the device by saving them in the account and sending only {"llm": {"aai_prompt": "my-app-cleanup", "prompt_replacements": {...}}} in the dictation config. That flow is impossible through this SDK (and the other one), so the SDKs cannot follow the docs' own security recommendation for the audience (web, desktop, mobile) the page is written for. Raising the priority of this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions