Skip to content

Text-only models use Synapse vision as their eyes (synapse_vision tool) #193

Closed

Description

@Alter-Igor

Problem

A session or sub-agent pinned to a text-only model cannot see images. Only synapse/auto routes an image turn to an on-prem vision model.

Observed by a running session (2026-10-17):

Path Result
model: "auto" + image → on-prem vision model works
session pinned to a text-only model text-only
sub-agent on a text-only model text-only

Today the image part becomes ERROR: Cannot read image (this model does not support image input). The existing vision proxy only runs when a vision_proxy block is configured, defaults to OpenRouter, uses a plain API key (no Keystone sign-in), and only gives a generic description.

Want

A text-only agent uses Synapse vision as its eyes:

  1. A tool synapse_vision { image, question }: sends the image to Synapse (model: "auto", same sign-in and privacy tier as chat) with the question, returns the answer as text.
  2. When an image reaches a text-only model (pasted, or a screenshot from a tool), it is saved to a file and replaced by a note naming that file and the tool, so the agent can ask targeted questions about it.

Done when

  • Unit tests cover the tool request shape, credential choice, and the image-to-note swap.
  • Typecheck passes.

Activity

  1. self-assigned this
    on Oct 17, 2026
  2. Alter-Igor commented on Oct 17, 2026

    @Alter-Igor
    OwnerAuthor

    Project tracking (opencode Delivery #16):

    • Owner: Alter-Igor — defaulted (WT-PM-01)
    • Target Date: 2026-10-17 — review-by date, defaulted (WT-PM-02)
    • Priority: P2 — assessed by Synapse auto (served ornith-1.0-35b): "planned feature enhancement ... lacking a stated business consequence" (WT-PM-05)
    • Status: In Progress, Stage: Current — work started 2026-10-17
  3. Alter-Igor commented on Oct 17, 2026

    @Alter-Igor
    OwnerAuthor

    Checkpoint: PR #195 opened from branch synapse-vision-eyes (worktree X:\opencode---synapse-vision-eyes). Done: tool + proxy note + tests; live run with a text-only model answered correctly. Next: architect review, pr-loop, integrate. Blocker seen: Synapse currently has no model with the vision op (on-prem vision model left /v1/models), see Alterspective-Engine/Alterspective-Synapse#2061.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions