Skip to content

feat(image): switch image generation to Meta Muse Image - #270

Merged
NickB03 merged 2 commits into
mainfrom
feat/muse-image-model
Sep 18, 2026
Merged

NickB03 merged 2 commits into
mainfrom
feat/muse-image-model

Conversation

@NickB03

@NickB03 NickB03 commented Sep 18, 2026

Copy link
Copy Markdown
Owner

Summary

Switches inline image generation from google/gemini-2.5-flash-image to Meta Muse Image (gateway:meta/muse-image-1.0) and reworks the tool to fit it.

Stacked on #269 (both PRs edit the same README and DECISIONS lines). GitHub retargets this to main once #269 merges.

$/image Generation Elo (rank) Editing Elo (rank)
Gemini 2.5 Flash Image (before) 0.039 985 (#50) 987 (#45)
Muse Image 1.0 (after) 0.010 1112 (#7) 1116 (#6)

Source: Artificial Analysis image arenas, 2026-09-18.

What changed

  • API: Muse is a Gateway image-type model, so the tool calls the AI SDK's generateImage instead of generateText + result.files. Added getImageModel to lib/utils/registry.ts.
  • Aspect ratios: Muse ignores aspectRatio (the Gateway warns "use size instead"). The tool maps each ratio to a size hint. Muse snaps output to about 2.4 MP at that ratio.
  • Editing: the source image is passed as an http(s) string, which the SDK forwards as a URL file for the Gateway to fetch. The ownership check and signed-URL flow are unchanged, and this server still never downloads model-supplied URLs.
  • Telemetry: generateImage has no telemetry option, so the tool annotates the enclosing tool span (image.model, image.is_edit, image.size), the same pattern the search tool uses.
  • No-image case: NoImageGeneratedError maps to the existing "No image was generated" tool error.
  • Output format: WebP. The storage bucket already allows image/webp; files are 60–330 KB against a 5 MB limit.

Verification

  • bun lint, bun typecheck, bun format, bun run test (193 files / 1650 tests), and graph freshness all pass.
  • Live against the Gateway with the same calls the tool makes:
Requested size sent Output
1:1 1024x1024 1600x1600
16:9 1792x1024 2016x1152
9:16 1024x1792 1152x2016
4:3 1536x1152 1792x1344
3:4 768x1024 1344x1792

Edit from bytes and edit from a public URL both succeeded; a visual check showed the edit recolored the subject and kept the scene. Latency was 9–19 s per image.

Not verified locally (check on the preview)

Local Supabase was not running, so the upload and signed-URL steps ran only under unit tests:

  • Generate an image in a signed-in chat and confirm it renders and downloads as .webp
  • Ask for an edit of that image (exercises Gateway fetch of a Supabase signed URL)
  • Generate as a guest

🤖 Generated with Claude Code

@vercel

vercel Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
polymorph Ready Ready Preview Sep 18, 2026 4:08pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: d2cde6dc-3a5a-4c96-8632-6d5a2ca8b6bd


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-18T16:08:28.640601Z d7f06e2 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

NickB03 and others added 2 commits September 18, 2026 11:06
…mage

Muse is a Gateway image-type model, so the tool now calls generateImage instead of generateText. Muse ignores aspectRatio, so ratios map to size hints (verified live for all five). Edit sources are passed as URL files the Gateway fetches; the ownership check and signed-URL flow are unchanged. Output is WebP, which the bucket already allows.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@NickB03
NickB03 force-pushed the feat/muse-image-model branch from d7f06e2 to ba8c972 Compare September 18, 2026 16:07
@NickB03
NickB03 changed the base branch from claude/musing-liskov-593f41 to main September 18, 2026 16:07

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7f06e2b24

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread graphify-out/graph.json
Comment on lines +46867 to +46871
"id": "generate_image_server_aspect_ratio_sizes",
"label": "ASPECT_RATIO_SIZES",
"file_type": "code",
"source_file": "lib/tools/generate-image/server.ts",
"source_location": "L27",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refresh shifted graph source locations

The graph refresh adds the new symbols but leaves existing lib/tools/generate-image/server.ts nodes at their pre-change positions: ImageToolContext, resolveSourceImageUrl, createGenerateImageTool, and serverTool still point to L19/L36/L61/L149 even though this commit moves them to L38/L55/L80/L168. Consequently, Graphify queries return incorrect source locations for this module; regenerate or repair all affected node metadata, not only the newly introduced nodes.

AGENTS.md reference: AGENTS.md:L122-L122

Useful? React with 👍 / 👎.

@NickB03 NickB03 closed this Sep 18, 2026
@NickB03 NickB03 reopened this Sep 18, 2026
@NickB03
NickB03 merged commit 72fb9f1 into main Sep 18, 2026
11 checks passed

This branch was successfully deployed

1 active deployment
Preview — ba8c9720 Deployed Sep 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant