Skip to content

fix(ai): accept content part arrays in OpenAI Chat streams - #53697

Open
kitlangton wants to merge 2 commits into
devfrom
fix/openai-compatible-array-content
Open

kitlangton wants to merge 2 commits into
devfrom
fix/openai-compatible-array-content

Conversation

@kitlangton

Copy link
Copy Markdown
Contributor

Why

Mistral-style models served through OpenAI-compatible /chat/completions gateways stream choices[0].delta.content as an array of typed content parts instead of a plain string:

{
  "id": "chunk_fixture",
  "object": "chat.completion.chunk",
  "created": 1791353545,
  "model": "example-model",
  "choices": [
    {
      "index": 0,
      "finish_reason": null,
      "logprobs": null,
      "delta": {
        "content": [
          {
            "type": "thinking",
            "thinking": [{ "type": "text", "text": "Let" }]
          }
        ]
      }
    }
  ]
}

Because OpenAIChatDelta.content only accepted string | null | undefined, every run against these models failed on the first chunk with InvalidProviderOutput.

What Changes

OpenAIChatDelta.content now accepts string | null | ReadonlyArray<OpenAIChatContentPart>, and contentDeltas flattens part arrays in order:

delta.content input Stream behavior
"Hello" Emits text-delta ("Hello") on text-0
[{ "type": "text", "text": "Hello" }] Emits text-delta ("Hello") on text-0
[{ "type": "thinking", "thinking": [{ "type": "text", "text": "Let" }] }] Emits reasoning-delta ("Let") on reasoning-0
[{ "type": "thinking", "thinking": "Planned." }] Emits reasoning-delta ("Planned.") on reasoning-0
Other part types (reference, image_url, …) Skipped and logged at debug level (openai-chat.content_parts_skipped)
[] after finish_reason Accepted as an empty trailing chunk
Non-empty text or thinking parts after finish_reason Rejected with InvalidProviderOutput (OpenAI Chat received content after the finish reason)

Verification

bun test packages/ai/test/provider/openai-compatible-chat.test.ts
bun run --cwd packages/ai test
bun run --cwd packages/ai typecheck
bunx oxlint packages/ai/src/protocols/openai-chat.ts packages/ai/test/provider/openai-compatible-chat.test.ts
  • openai-compatible-chat.test.ts: 25 pass, 0 fail (including 5 new tests under content part arrays covering thinking/text streaming and replay, intra-chunk ordering, skipped/empty parts, and post-finish validation).
  • @opencode/ai test suite: 1,782 pass, 0 fail.
  • @opencode/ai typecheck (tsgo --noEmit && tsgo --noEmit -p tsconfig.types.json) and workspace turbo typecheck: pass.
  • oxlint on changed files: 0 warnings, 0 errors.

Mistral-family models served through OpenAI-compatible gateways stream delta.content as typed parts ({type: "text"} and {type: "thinking", thinking: [...]}) rather than a string, which failed stream decoding on the first chunk. Map text parts to text deltas and thinking parts to reasoning deltas in order, and skip other part types.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant