Repository navigation
fix(ai): accept content part arrays in OpenAI Chat streams - #53697
Open
kitlangton wants to merge 2 commits into
Open
kitlangton wants to merge 2 commits into
kitlangton wants to merge 2 commits into
Conversation
Mistral-family models served through OpenAI-compatible gateways stream delta.content as typed parts ({type: "text"} and {type: "thinking", thinking: [...]}) rather than a string, which failed stream decoding on the first chunk. Map text parts to text deltas and thinking parts to reasoning deltas in order, and skip other part types.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Mistral-style models served through OpenAI-compatible
/chat/completionsgateways streamchoices[0].delta.contentas an array of typed content parts instead of a plain string:{ "id": "chunk_fixture", "object": "chat.completion.chunk", "created": 1791353545, "model": "example-model", "choices": [ { "index": 0, "finish_reason": null, "logprobs": null, "delta": { "content": [ { "type": "thinking", "thinking": [{ "type": "text", "text": "Let" }] } ] } } ] }Because
OpenAIChatDelta.contentonly acceptedstring | null | undefined, every run against these models failed on the first chunk withInvalidProviderOutput.What Changes
OpenAIChatDelta.contentnow acceptsstring | null | ReadonlyArray<OpenAIChatContentPart>, andcontentDeltasflattens part arrays in order:delta.contentinput"Hello"text-delta("Hello") ontext-0[{ "type": "text", "text": "Hello" }]text-delta("Hello") ontext-0[{ "type": "thinking", "thinking": [{ "type": "text", "text": "Let" }] }]reasoning-delta("Let") onreasoning-0[{ "type": "thinking", "thinking": "Planned." }]reasoning-delta("Planned.") onreasoning-0reference,image_url, …)openai-chat.content_parts_skipped)[]afterfinish_reasonfinish_reasonInvalidProviderOutput(OpenAI Chat received content after the finish reason)Verification
openai-compatible-chat.test.ts: 25 pass, 0 fail (including 5 new tests undercontent part arrayscovering thinking/text streaming and replay, intra-chunk ordering, skipped/empty parts, and post-finish validation).@opencode/aitest suite: 1,782 pass, 0 fail.@opencode/aitypecheck (tsgo --noEmit && tsgo --noEmit -p tsconfig.types.json) and workspaceturbo typecheck: pass.oxlinton changed files: 0 warnings, 0 errors.