Skip to content

synapse_vision: close the 3 documented gaps (URL ask, native runtime, in-result media) #235

Closed

Description

@Alter-Igor

Follow-up to #193 / #198. Three gaps in Synapse vision for text-only models, documented in AGENTS.md (fork notes, #193 paragraph) but not fixed:

  1. http(s) image URLs: synapse_vision passes an image URL to Synapse with no permission ask (packages/opencode/src/plugin/synapse-vision.ts, resolveVisionImage). Fix idea: ask webfetch for the URL first. ~10 lines + test.
  2. Native runtime: packages/opencode/src/session/llm/native-runtime.ts builds messages with ProviderTransform.message and skips the AI SDK middleware, so a text-only model on the native runtime never gets the vision note. Fix idea: run proxyUnsupportedImages before ProviderTransform.message there. Opt-in path. ~10 lines + test.
  3. Media kept inside tool results: supportsMediaInToolResult in packages/opencode/src/session/message-v2.ts keeps images inside tool results for some SDKs (anthropic, openai, bedrock, xai, google gemini-3), so the proxy never sees them. Only matters for non-Synapse providers. Fix idea: also require model.capabilities.input.image. Upstream file, so it may clash on upstream syncs.

Owner decision pending (2026-10-18): fix all 3 in one PR (~2 h incl. review rounds), or fix 1+2 now and leave 3 here (~1 h). Recommended: 1+2 now.

Activity

  1. self-assigned this
    on Oct 17, 2026
  2. Alter-Igor commented on Oct 17, 2026

    @Alter-Igor
    OwnerAuthor

    Project tracking (opencode Delivery #16):

    • Owner: Alter-Igor — defaulted (WT-PM-01)
    • Target Date: 2026-10-18 — review-by date, defaulted (WT-PM-02)
    • Priority: P2 — assessed by Synapse auto (served ornith-1.0-35b): "ordinary planned maintenance ... no stated business impact" (WT-PM-05)
    • Status: Planned, Stage: Backlog — waiting on the owner's scope pick (all 3, or 1+2 now)
  3. Alter-Igor commented on Nov 3, 2026

    @Alter-Igor
    OwnerAuthor

    ARCH-GATE-01: EXTEND packages/opencode/src/plugin/synapse.ts, packages/opencode/src/provider/vision-proxy.ts, packages/opencode/src/session/llm.ts class=Standard reviewer=antigravity/arch-review-subagent:5b8ceba4-c40c-4132-a1ab-203f396904f6 rules=ARCH-GATE-01,ARCH-GATE-03,ARCH-LAY-01,ARCH-EXT-01,SOLID,DIP

    Stage 1 Pre-Build Architecture Consult (WEBSTA-001-ARCHITECTURE-STANDARDS v2.4.1 / AIRUN-058)

    • (a) Existing path (ARCH-GATE-01(a)): EXTEND the existing fork vision-proxy and Synapse vision tool surfaces introduced in Text-only models use Synapse vision as their eyes (synapse_vision tool) #193 / synapse_vision: fix CodeRabbit follow-ups from #195 #198 (packages/opencode/src/plugin/synapse.ts, packages/opencode/src/plugin/synapse-vision.ts, packages/opencode/src/provider/vision-proxy.ts, and packages/opencode/src/session/llm.ts). No new service, plugin, dependency, or persistence store is needed.
    • (b) Disciplines & Classification (ARCH-GATE-01(b), ARCH-GATE-03):
      • Class: Standard (touches security permission boundary for external http(s) image URLs in synapse_vision and extends runtime message preparation across session/llm.ts and provider/vision-proxy.ts).
      • Gap 1 (http(s) image URL permission): Extend synapse_vision.execute in packages/opencode/src/plugin/synapse.ts to call context.ask({ permission: "webfetch", patterns: [url], always: [""], metadata: { url, format: "image" } }) before
        esolveVisionImage, matching packages/opencode/src/tool/webfetch.ts (�lways: ["
        "]) and rethrowing permission denials/cancellations (�skFailed = true).
      • Gap 2 (Native runtime vision swap): Extend LLM.run in packages/opencode/src/session/llm.ts when lags.experimentalNativeLlm is enabled and LLMNativeRuntime.status({ model: input.model, provider: item, auth: info }).type === "supported" to run proxyUnsupportedImages via Effect.tryPromise(...).pipe(Effect.orElseSucceed(() => prepared.messages)) before LLMNativeRuntime.stream(...). Gating on status.type === "supported" guarantees a paid �ision_proxy is never invoked twice when native falls back to �i-sdk.
      • Gap 3 (Tool-result media items): Extend proxyUnsupportedImages in packages/opencode/src/provider/vision-proxy.ts (swapToolResultImages) to inspect msg.role === "tool" parts ( ype === "tool-result" with output.type === "content") and replace media, image-data, and image/* ile-data items with the saved-file vision note or proxy description while leaving packages/opencode/src/session/message-v2.ts untouched to avoid upstream merge friction.
      • ADR check (§ 2.6): ARCH-ADR = NOT-REQUIRED (no standards deviation, no new external service or dependency).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions