Repository navigation
feat(opencode-route): routing-only door — resolve the route, call the pool member directly - #2549
Conversation
… pool member directly (#2518) The opencode-route proxy no longer runs an OpenCode session per request. It resolves the requested route against the fleet config it shares with the OpenCode server (.opencode/opencode.jsonc: experimental.model_routes + provider blocks), picks the actual pool member (ordered/round-robin/weighted selection, #variant + variants remaps, disabled providers skipped, quota demotion from the route_attempt ledger), and makes ONE plain chat-completions call to that member with its credential from the OpenCode credential store. No session rows, no agent loop - the caller keeps its loop and gets its actual model by calling the route (Drew, 2026-10-29). Every attempt is recorded in route_attempt with session_id NULL so quota accounting and demotion keep working. Provider name opencode-route and port 18750 unchanged; no OpenClaw config change needed. Live proof (2026-10-29, spare port, production config+DB): workhorse non-stream and speed stream completions succeeded; session_v2+session_message count unchanged at 115051; route_attempt rows 19983/19984 written with session_id NULL. Unit tests rewritten for the routing shape: 7/7 pass (fake leaf providers, fixture config + fixture ledger).
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 59 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2d6e855ef7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const baseURL = providerBaseURL(cfg, leaf.providerId); | ||
| if (!baseURL) | ||
| throw new LeafError(`no endpoint configured for provider ${leaf.providerId}`, { |
There was a problem hiding this comment.
With the checked-in fleet config, openai supplies model overrides but no options.baseURL, and openai.alternate inherits that definition. Since the new resolver supplies a built-in endpoint only for opencode-go, both OpenAI leaves are rejected here before any request or credential lookup. The terra-plan route contains only these two leaves and therefore always returns 502; the OpenAI seats in sol-plan and luna-plan are also unusable. Preserve built-in provider endpoint defaults when overlaying the configured provider definitions.
Useful? React with 👍 / 👎.
| if (route.selection === "round-robin") targets = rotate(targets, rid); | ||
| else if (route.selection === "weighted") | ||
| targets = weightedOrder(targets, route.weights || {}); |
There was a problem hiding this comment.
For routes using selection: "round-robin", this branch ignores route.weights and assigns every target an equal share. The checked-in workhorse-primary route explicitly weights Gemini at 2 versus DeepSeek and Spark at 1 each, while speed and deepseek-flash-plan also specify unequal weights. OpenCode uses smooth weighted round-robin, as documented in the route-pools design document, so this door changes the fleet's intended quota allocation rather than sharing its routing semantics. Use weighted round-robin here instead of an unweighted rotation.
Useful? React with 👍 / 👎.
| const stats = await relayStream(leafRes, res); | ||
| if (!res.destroyed && !res.writableEnded) res.end(); | ||
| return stats; |
There was a problem hiding this comment.
If an upstream SSE connection breaks or the request deadline expires after headers arrive, relayStream catches the reader exception and returns it in stats.error. This callback then ends the downstream response normally without checking that error, and runCompletion only records it in the ledger. Clients therefore receive a clean HTTP 200 EOF with a truncated completion and no error event, preventing callers that rely on explicit errors from recognizing the failed turn. Surface the relay failure to the client before ending the response, or terminate the downstream connection as failed.
Useful? React with 👍 / 👎.
| if (t && typeof t.model === "string") | ||
| return { ...parseTargetRef(t.model), variantMap: t.variants || null }; |
There was a problem hiding this comment.
For the checked-in background route, object targets carry window gates referencing experimental.model_route_windows, but this normalization discards those gates and the resolver never evaluates them. Consequently, the first peak-only MiMo target is expanded even outside both peak windows, so background requests always prefer MiMo instead of the configured off-peak DeepSeek Flash primary. Resolving the actual fleet config confirms that all first six candidates are MiMo leaves regardless of the clock. Preserve and evaluate each target's named window, including inverted windows, before constructing the candidate order.
Useful? React with 👍 / 👎.
| ? AbortSignal.any([deadlineCtl.signal, clientSignal]) | ||
| : deadlineCtl.signal; | ||
| const sessionLabel = `ses_${randomBytes(12).toString("hex")}`; | ||
| const attempts = candidates.slice(0, MAX_ATTEMPTS); |
There was a problem hiding this comment.
When DeepSeek leaves are unavailable and there are no existing demotions, the checked-in workhorse route expands to seven DeepSeek leaves before reaching Meta and Antigravity. Truncating that flattened list to six therefore returns 502 after attempting only DeepSeek, without trying either healthy alternate primary leg or any configured relief member. This defeats the failover structure described in .opencode/opencode.jsonc, where the relief pool is intended to absorb primary failures. Preserve access to later route branches rather than allowing the first nested pool to consume the entire candidate budget.
Useful? React with 👍 / 👎.
Implements #2518.
What changed —
bin/opencode-route-proxy.mjsis rewritten. The door no longer creates an OpenCode session per request. It now:.opencode/opencode.jsonc:experimental.model_routes+ provider blocks,instanceOfchains,{env:}baseURL templates)#variantsuffixes and per-targetvariantsremaps,disabled_providersskipped, quota demotion from theroute_attemptledger (≥3 quota failures in 4h sinks a member to the back)opencode.db), forwarding the caller's messages as-is — no transcript flattening, no session, no looproute_attemptwithsession_idNULL, so quota accounting and demotion see door traffic exactly like session trafficProvider name
opencode-routeand port 18750 are unchanged — no OpenClaw config change; the internals swap behind the same endpoint when this deploys.Why — Drew's directive (2026-10-29): "opencode shouldn't do work besides routing it shouldn't do any loop etc." The caller (OpenClaw's embedded agent) gets its actual model by calling the opencode route. Dispatch paths (ACP, sessions_spawn — #2256, #2257) are where OpenCode doing the work is the point; they are untouched.
Evidence
workhorsenon-stream andspeedstreaming completions succeeded;session_v2+session_messagecount unchanged (115051 → 115051);route_attemptrows 19983/19984 written withsession_idNULLnode --test bin/opencode-route-proxy.test.mjs): 7/7 pass — fake leaf providers + fixture config/ledger covering ordered resolution, verbatim message forwarding, 429 failover with both attempts ledgered, variant→reasoning_effort remap, nested routes, disabled-provider skip, SSE relay, unknown-route/malformed rejectionNotes
x-opencode-sessionattribution header at their gateway; the door mints one per request (an attribution label only — no local session row exists).reasoning_effortis set when the leaf model defines the resolved variant in config; the intended variant is always recorded in the ledger.Need help on this PR? Tag
@codesmith-botwith what you need.