Repository navigation
fix(ai): request inline image bytes by default - #59947
Conversation
There was a problem hiding this comment.
The change looks sound. I read the diff, traced that core's openai-compatible/images entry goes through configure().image, and ran the image and recorded tests and the full packages/ai suite (0 failures). typecheck:tsc is clean. I couldn't make live provider calls here.
Public API: the default request behaviour changes for three providers. xAI image requests now default to response_format: "b64_json" (previously unset, so xAI returned a URL). fal image requests now default to sync_mode: true (previously unset, so fal returned a hosted URL), and sync_mode?: boolean is added to FalImageOptions. OpenAI-compatible image requests now default to response_format: "b64_json", except for model IDs that contain gpt-image or chatgpt-image. This is backwards compatible for callers: they get inline bytes instead of URLs and can opt back into URLs through providerOptions.
One concern (inline): the GPT Image check depends on the model ID, and there's no way to switch the default off, so a proxy that serves GPT Image under another name would start failing.
There was a problem hiding this comment.
The new commit addresses my earlier point: passing response_format: null now removes the OpenAI-compatible default, and the field is left out of both JSON and multipart edit requests. I checked it with the updated unit test, the full packages/ai suite (0 failures) and typecheck:tsc (clean). Public API: callers can now pass response_format: null in OpenAI image provider options to leave the field out (backwards compatible). The rest of the API impact is as in my previous review. No new concerns.
Problem
A few image routes in
packages/aicame back with short-lived hosted URLs unless the caller knew to ask for base64. Core downloads them right away, but that's an extra round trip and a URL that can expire before anyone reads it.Change
I now ask for inline bytes wherever the provider has a switch for it. Callers can still opt back into URLs through
providerOptions.response_format: "b64_json"(was URL)responseFormat: "url"sync_mode: true, data URIs (was hosted URL)sync_mode: falseresponse_format: "b64_json", except GPT Image idsresponse_format: "url"responseFormat: "url"I didn't touch the shared OpenAI Images protocol. OpenAI now rejects
response_formatwithUnknown parameterfor every model it serves (DALL·E is gone from OpenAI and Azure), so the default only goes on the generic OpenAI-compatible provider, and I skip it there for ids containinggpt-imageorchatgpt-imageso proxies in front of OpenAI keep working.Out of scope
Verification
b64_jsonwith the field. OpenAI rejectsresponse_formatforgpt-image-1-mini. Meta, OpenRouter, Vercel (bfl/flux-2-klein-4b), DigitalOcean and DeepInfra already returnb64_json. DeepInfra acceptsresponse_format: "url".response_format: "url". xAI returns bytes by default.sync_mode: falseexplicitly, so it keeps covering the hosted-URL path, and I added that field to its fixture request body.bun typecheck,bun run typecheck:tscandbun testinpackages/aiall pass (1946 pass).Added by the opencode bot (AI-generated).
Public API: @opencode/ai image defaults: xAI and OpenAI-compatible hosts now send
response_format: b64_json(nullleaves it out), fal sendssync_mode: true,sync_modeis added toFalImageOptions(backwards compatible)