Skip to content

fix(ai): request inline image bytes by default - #59947

Merged
rekram1-node merged 2 commits into
devfrom
image-base64-default
Oct 28, 2026
Merged

rekram1-node merged 2 commits into
devfrom
image-base64-default

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A few image routes in packages/ai came back with short-lived hosted URLs unless the caller knew to ask for base64. Core downloads them right away, but that's an extra round trip and a URL that can expire before anyone reads it.

Change

I now ask for inline bytes wherever the provider has a switch for it. Callers can still opt back into URLs through providerOptions.

Provider Default now Opt into URLs
xAI response_format: "b64_json" (was URL) responseFormat: "url"
fal sync_mode: true, data URIs (was hosted URL) sync_mode: false
OpenAI-compatible hosts response_format: "b64_json", except GPT Image ids response_format: "url"
OpenAI, Azure, DigitalOcean, OpenRouter, Vercel AI Gateway already bytes, unchanged n/a
Meta already bytes, unchanged responseFormat: "url"
Google, Stability, Cloudflare Workers AI already bytes, unchanged n/a
BFL, Alibaba, Z.ai, Replicate URL only, unchanged n/a

I didn't touch the shared OpenAI Images protocol. OpenAI now rejects response_format with Unknown parameter for every model it serves (DALL·E is gone from OpenAI and Azure), so the default only goes on the generic OpenAI-compatible provider, and I skip it there for ids containing gpt-image or chatgpt-image so proxies in front of OpenAI keep working.

Out of scope

Verification

  • Live probes: xAI returns a URL by default and b64_json with the field. OpenAI rejects response_format for gpt-image-1-mini. Meta, OpenRouter, Vercel (bfl/flux-2-klein-4b), DigitalOcean and DeepInfra already return b64_json. DeepInfra accepts response_format: "url".
  • Live through the library: an OpenAI-compatible model on DeepInfra returns bytes by default and a URL with response_format: "url". xAI returns bytes by default.
  • I couldn't record fal without a key, so that change follows fal's docs ("If True, the media will be returned as a data URI"). The fal recorded test now passes sync_mode: false explicitly, so it keeps covering the hosted-URL path, and I added that field to its fixture request body.
  • New unit test covers the defaults and the URL overrides. bun typecheck, bun run typecheck:tsc and bun test in packages/ai all pass (1946 pass).

Added by the opencode bot (AI-generated).

Public API: @opencode/ai image defaults: xAI and OpenAI-compatible hosts now send response_format: b64_json (null leaves it out), fal sends sync_mode: true, sync_mode is added to FalImageOptions (backwards compatible)

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks sound. I read the diff, traced that core's openai-compatible/images entry goes through configure().image, and ran the image and recorded tests and the full packages/ai suite (0 failures). typecheck:tsc is clean. I couldn't make live provider calls here.

Public API: the default request behaviour changes for three providers. xAI image requests now default to response_format: "b64_json" (previously unset, so xAI returned a URL). fal image requests now default to sync_mode: true (previously unset, so fal returned a hosted URL), and sync_mode?: boolean is added to FalImageOptions. OpenAI-compatible image requests now default to response_format: "b64_json", except for model IDs that contain gpt-image or chatgpt-image. This is backwards compatible for callers: they get inline bytes instead of URLs and can opt back into URLs through providerOptions.

One concern (inline): the GPT Image check depends on the model ID, and there's no way to switch the default off, so a proxy that serves GPT Image under another name would start failing.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit addresses my earlier point: passing response_format: null now removes the OpenAI-compatible default, and the field is left out of both JSON and multipart edit requests. I checked it with the updated unit test, the full packages/ai suite (0 failures) and typecheck:tsc (clean). Public API: callers can now pass response_format: null in OpenAI image provider options to leave the field out (backwards compatible). The rest of the API impact is as in my previous review. No new concerns.

@rekram1-node
rekram1-node merged commit 506ac20 into dev Oct 28, 2026
22 checks passed
@rekram1-node
rekram1-node deleted the image-base64-default branch October 28, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant