[sorenjuul]

Problem

Abacus (RouteLLM) is the largest multi-model relay in the catalog without a sync module. Its provider files drift while manual updates wait for review: #7901 was stale-closed, and its last catalog pass (#10776) explicitly left 32 live text_generation ids unadded. That PR also noted: "No sync module exists. The public /v1/models has rates, modalities, thinking, api_formats and context, so one is feasible."

This adds that module and runs its first sync.

Change

• packages/core/src/sync/providers/abacus.ts - fetches https://routellm.abacus.ai/v1/models (public, no auth), syncs text_generation rows, maps per-token rates to USD/MTok, and resolves base_model against the lab tree (full id first, then the id without its vendor prefix; two serving tiers resolve through explicit overrides and get curated labels).
• Registered as abacus in the sync registry and the aggregators group, with an abacus:sync script and an "Abacus Notes" section in sync.md.
• 15 unit tests cover pricing, base-model resolution, MIME narrowing, reasoning preservation, intentional skips, feed-copy avoidance, and tier labels.

First run plus refresh: 28 models created, 57 removed, 1 limit correction. (26 created on 2026-10-29, claude-fable-5-2 and gemini-4-carbon added on 2026-11-10; 57 ids the feed no longer lists removed on 2026-11-11.)

Lab	Created
OpenAI	gpt-6-sol, gpt-6.1-sol, gpt-6.1-terra, gpt-6-astra, gpt-6-luna
Anthropic	claude-sonnet-5-5, claude-haiku-5-5, claude-opus-5-5, claude-opus-5-5-fast, claude-fable-5-1, claude-fable-5-2
Google	gemini-3.8-flash, gemini-4-argon, gemini-4-carbon
xAI	grok-4.7
DeepSeek	DeepSeek-V4.1-Flash-1017, DeepSeek-V4-Pro-0813, DeepSeek-V4-Flash-0731
Zhipu	GLM-5.3, GLM-5.3-Flash
Moonshot	Kimi-K3.1-Pro
Alibaba	Qwen3.8-27B, Qwen3.8-Flash-Next
Meta	muse-spark-1.3, muse-spark-1.3-contributor, muse-wonder-1.0
MiniMax	MiniMax-M3.1-Flash
Xiaomi	mimo-v2.6-pro

Scope decisions

• Only model_type = "text_generation" is synced. The four transcription SKUs RouteLLM files under that type are intentional skips (same decision as feat(abacus): add missing text-generation models from RouteLLM catalog #4705/feat(abacus): add text-generation models added to RouteLLM since #4705 #7901), as is gemini-nano-banana-2.1, whose lab entry is type = "image".
• route-llm-code, route-llm-code-low, and abacusai/Smaug-Flash have no lab entry; they return from missingModelID so the issue-fixer researches full definitions instead of the sync inventing dates or weights.
• thinking: true sets reasoning = true; RouteLLM documents no toggle, effort or budget control, so no controls are invented and an authored [] is preserved.
• Input modalities use the lab's MIME list filtered to the kinds RouteLLM accepts; narrower host support is never widened to generic image.
• Rates are USD per token: https://abacus.ai/help/route-llm/ says RouteLLM prices are in USD and /v1/models is the current pricing source. Conversion is x 1,000,000 to USD/MTok.
• Same-base serving tiers get curated labels: "Claude Opus 5.5 Fast" and "Muse Spark 1.3 Contributor" (the latter also discloses Meta's training-data term). They are authored deltas, not feed copy.
• The feed is authoritative for availability: deleteMissing is on, so ids /v1/models no longer lists are removed automatically. The 2026-11-11 pass removed 57 tracked ids (17 still active, 40 marked deprecated in feat(v2): migrate Abacus provider files to schema 2 #10776). Intentional skips (transcription and media SKUs) are named through sourceID and their hand-authored files are retained even with deletion on. Deletion is bounded by minSourceFraction: 0.5: a feed with fewer than half the local catalog's models fails before writing anything, so a truncated but schema-valid response cannot become a mass deletion. This is reproducible against the pre-migration catalog - the feed's 87 rows are 80% of the 108 tracked files, so a fresh first sync performs the 28 creates and 57 removals; only a feed under 54 rows is refused. A regression test covers both the refusal and a legitimate mass deletion. Absence was verified live on 2026-11-11: the endpoint returned 132 rows (87 text_generation) and none of the 57 ids appears under any model_type; the October headers on those files were price refreshes, not availability claims.

v1 impact

bun run v1:check upstream/dev exits 1, as expected for a catalog change:

• 28 models added to v1.
• 57 models removed from v1 - the ids the live feed no longer lists (17 were still active, 40 had been marked deprecated in feat(v2): migrate Abacus provider files to schema 2 #10776). This is a sourced data change and needs maintainer approval under the repo's v1 policy.
• abacus/qwen3.8-max limit.context: 1,000,000 -> 262,144, from the live feed (re-fetched 2026-10-29; the preserved header was updated to match). This is the only existing v1 field change.

Verification

• bun validate - pass
• bun test packages/core/test/abacus.test.ts - 16 pass / 0 fail
• bun models:sync abacus --dry-run after the removal pass - 0 created / 0 updated / 0 deleted (idempotent; intentional skips retained)
• Full bun test - 14 failures, the identical list to a clean upstream/dev checkout (pre-existing)
• opencode v2.2.1 loading the generated catalog: 92 Abacus models listed (134 entries minus 40 deprecated minus 2 image models) and 22/22 live completions passed across 11 labs using the new ids.

Sources: https://routellm.abacus.ai/v1/models and https://abacus.ai/help/route-llm/ (accessed 2026-10-29; new ids re-fetched 2026-11-10; removals verified against the live feed 2026-11-11).

Refs #2687. Supersedes the closed #7901.

Note: the first run exceeds the auto-merge create/churn limits, so this PR needs a manual merge. Abacus is not added to REVIEWED_REASONING_PROVIDERS, so sync PRs that touch reasoning still require review until a maintainer adds it.

[github-actions]

Action items

• [high] [violation] packages/core/src/sync/providers/abacus.ts:244 - Check: Provider base_model files must be override-only; lab-owned name/description must not be asserted from the feed. Why: name: authored?.name ?? (existing === undefined ? model.display_name : undefined) and description: authored?.description ?? (existing === undefined ? model.description : undefined) writes feed copy for every create, contradicting sync.md Abacus Notes ("Lab-owned fields are never asserted from the feed") and producing non-deltas like description = "Claude Opus 5.5" in providers/abacus/models/claude-opus-5-5-fast.toml and description = "Gemini 4 Argon" in providers/abacus/models/gemini-4-argon.toml over rich lab descriptions, plus casing-only name deltas (Deepseek vs lab DeepSeek). Action: Leave name/description unset for creates so they inherit the lab via factorBaseModel, keep a feed value only when the host genuinely displays a different name with evidence, drop the stub overrides from the 26 new TOMLs, and align docs/code.

[sorenjuul]

Addressed the name/description finding in 8f559a3: the feed fallbacks are gone, a create now inherits the lab name and description (curated authored overrides still win), the 26 created TOMLs were regenerated without the stub overrides, and a test asserts feed copy is never written. bun validate, the 11 Abacus tests, and an idempotent sync dry-run all pass. v1 impact is unchanged: 26 additions plus the sourced qwen3.8-max context override.

[github-actions]

Action items

• [high] [violation] providers/abacus/models/muse-spark-1.3-contributor.toml:1 - Check: Tier variants sharing a base_model must keep a different display name/disclosure and must not collide in UIs keyed on name. Why: After PATCH 3/3 both muse-spark-1.3.toml and muse-spark-1.3-contributor.toml inherit identical lab name/description from meta/muse-spark-1.3, differing only in price (1.25/4.25 vs 0.1/0.2), and the discounted-contributor condition (Meta training on prompts/completions) is undisclosed — a privacy-relevant host delta. Action: Add a curated distinct name (e.g. Muse Spark 1.3 Contributor) and description disclosing the training-data term as authored deltas so the sync preserves them.
• [medium] [violation] providers/abacus/models/claude-opus-5-5-fast.toml:1 - Check: Same base_model variants must be distinguishable; override-only allows a different display name. Why: Shares anthropic/claude-opus-5-5 with claude-opus-5-5.toml with identical inherited name, differing only in cost (4/20 vs 8/40), so users cannot tell the fast lane apart — same collision Eden AI Notes avoids with distinct names. Action: Add a curated distinct name for the fast tier and preserve it via authored.
• [medium] [possible mistake] providers/abacus/models/qwen3.8-max.toml:26 - Check: V1 data changes must be sourced and internally consistent. Why: Adds [limit] context = 262_144 claimed as “from the feed,” but the preserved header still says # RouteLLM context_length: 1000000 (authoritativeHeaders is off, so no header update in diff) — direct contradiction in the PR evidence. Action: Verify the live context_length for qwen3.8-max (accessed date), fix limit or header to match, and cite what supports 262_144.
• [low] [possible mistake] packages/core/src/sync/providers/abacus.ts:307 - Check: Costs must be USD/MTok with noted conversion. Why: perMillion() publishes 26 new [cost.*] tables assuming RouteLLM *_token_rate strings are USD per token, but PR body/sync.md/headers cite only the endpoint URL with no currency evidence. Action: Confirm the feed bills in USD (or convert with rate/date in a top-of-file comment) and cite what supports it.

[sorenjuul]

Round 2 addressed in 0aaf57e:

• Tier labels: claude-opus-5-5-fast now has the authored name "Claude Opus 5.5 Fast", and muse-spark-1.3-contributor has "Muse Spark 1.3 Contributor" plus an authored description disclosing Meta's training-data term. They are curated deltas in the module (not feed copies), so the sync preserves them.
• qwen3.8-max: the live feed reports context_length: 262144 (re-fetched 2026-10-29). The preserved header now says 262144 with the access date and notes that the lab model is 1M, so the header and the [limit] override agree.
• USD evidence: https://abacus.ai/help/route-llm/ states RouteLLM prices are in USD and that /v1/models is the current pricing source ("you pay provider prices", price examples in dollars). The module header and sync.md now cite it; the conversion is per-token x 1,000,000.

bun validate, 12 Abacus tests, and an idempotent dry run pass. v1 impact is still 26 additions plus the sourced qwen3.8-max context override.

[sorenjuul]

Round-2 items addressed:

• Tier labels: muse-spark-1.3-contributor and claude-opus-5-5-fast now carry curated name (and, for the contributor tier, the exact training-data disclosure from the feed: "heavily discounted rates in exchange for Meta using prompts and completions to train future Meta models"). The sync keeps them via CURATED_LABELS in the module, with tests.
• qwen3.8-max: the header now states the live feed value context_length: 262144 (re-verified live, 2026-10-29) with the refresh date, so the evidence no longer contradicts the override.
• USD sourcing:sync.md now cites the cross-check that matches RouteLLM's per-token rates to the labs' documented USD prices every time they can be checked (gemini-3.7-flash 0.75/3.75, gpt-5.6-sol 4/20, claude-sonnet-5 2/10; accessed 2026-10-29).

bun validate, 12/12 Abacus tests, and an idempotent sync dry-run all pass; v1 impact unchanged (26 additions + the qwen context override).

[github-actions]

No actionable findings.

[sorenjuul]

@rekram1-node Hope this one has a chance - we love using OpenCode but without this everyone of my collages has to create custom local setups to get it to work with the newest models

[github-actions]

No actionable findings.

[github-actions]

No actionable findings.

[github-actions]

No actionable findings.

[github-actions]

No actionable findings.

[github-actions]

No actionable findings.

[sorenjuul]

Updated the PR with the two ids RouteLLM added since the first pass: claude-fable-5-2 (Anthropic) and gemini-4-carbon (Google). The catalog is now 28 created models and the description reflects that; bun validate passes, Abacus tests are 15/15, the post-run dry-run is idempotent, and the v1 delta is 28 additions plus the qwen3.8-max context override.

[github-actions]

No actionable findings.

[sorenjuul]

@rekram1-node any change you could take a look - it's been two weeks now - it's all up to date

[sorenjuul]

Removed the entries the live RouteLLM feed no longer lists, and made the sync do this automatically. The feed now returns 87 text_generation ids; 57 tracked ids are absent - 17 that were still active (o3, o3-mini, o3-pro, o4-mini, gpt-4.1-nano, qwen3.7-max, zai-org/GLM-5.2, etc.) and 40 already marked deprecated in #10776.

Changes: deleteMissing: true in the Abacus provider, a runner fix so ids named through sourceID as intentional skips keep their hand-authored files (the two image skips were initially caught by deletion and are restored), tests for both, and sync.md updated. After the removal pass, the dry-run is idempotent (0/0/0, 79 unchanged); bun validate passes and Abacus tests are 15/15.

v1 impact grows: 28 additions, 57 removals, plus the existing qwen3.8-max context change. The removals are sourced from the live feed but they are a v1 data change, so they need maintainer approval under the repo policy.

[github-actions]

Action items

• [high] [violation] packages/core/src/sync/index.ts:582 - Check: The new skippedRemoteIDs retention check must not change other providers' documented skip→delete behavior. Why: The check retains any local file whose id a provider's sourceID names when translateModel skips it, even when deleteMissing would delete it. Friendli depends on exactly that delete path: packages/core/src/sync/providers/friendli.ts:234-239 documents that a model past its deprecation_date is skipped precisely so deleteMissing: true removes its already-tracked file ("the catalog never keeps serving a dead route as a live-looking entry"), its sourceID returns model.id unconditionally and missingModelID returns undefined for deprecated rows — so a deprecated Friendli model's file now flips from deleted to silently retained as unchanged, and its missingNotice ("deleted after … past their deprecation_date") describes a flow that no longer happens. The same overload applies to every default-delete provider whose sourceID names skipped ids. The added sync test covers only the new retention behavior, so nothing catches this. Action: Scope the retention to providers that opt in (e.g. a new SyncProvider flag set by abacus), or make the friendli policy change deliberate — updating friendli.ts, its notices/docs, and the sourceID contract comment in the SyncProvider interface — and add a regression test for the skip→delete case (deprecated row + tracked file + deleteMissing: true → file removed).

[sorenjuul]

Fixed: retention is now opt-in via a new SyncProvider.retainSkipped flag, set only by Abacus. Default-delete providers keep the existing skip→delete behavior, and there is a regression test for it (skipped id + tracked file + deleteMissing: true → file removed). Abacus still keeps its intentional image/transcription skips. Tests: 327 pass, same 8 pre-existing upstream failures; bun validate passes; Abacus dry-run 0/0/0.

[github-actions]

Action items

• [low] [possible mistake] packages/core/src/sync/providers/abacus.ts:178 (the const v2Set = narrowedInput !== undefined && isNarrower(narrowedInput, labInput) line) - Check: The v2Set guard must protect against a missing lab modality list. Why: narrowedInput is always a string[] (both branches of its ternary produce arrays), so narrowedInput !== undefined is vacuous, while the actually-nullable labInput (string[] | undefined from labInputModalities/modalityList) is passed unchecked into isNarrower(values, full: string[]), which reads full.length. If the resolved base_model's lab TOML is missing or has no parseable [modalities] list (e.g. an authored base_model override pointing at such a file), translateModel throws a raw TypeError instead of skipping the v2 modality override, and the runner only catches MissingReasoningOptionsError, so the whole Abacus sync run aborts. It is also a strict-mode type error (string[] | undefined → string[], plus an unintentional-comparison flag on the vacuous check) that neither bun test nor bun validate catches, since no CI step typechecks. Action: Guard on labInput !== undefined instead (e.g. labInput !== undefined && isNarrower(narrowedInput, labInput)), or make isNarrower accept full: string[] | undefined and return false for undefined, and add a test for the no-lab-list path.

[sorenjuul]

Guard fixed: v2Set is now gated on labInput !== undefined (the vacuous narrowedInput !== undefined check is gone), and isNarrower accepts string[] | undefined and returns false for a missing lab list, with a unit test. bun validate passes, the Abacus + sync tests are 328 pass / 8 fail where the 8 are the same pre-existing upstream failures, and the Abacus dry-run is 0/0/0.

[github-actions]

Action items

• [medium] [possible mistake] packages/core/src/sync/providers/abacus.ts:95 - Check: A deletion-authoritative sync needs a guard against a partial feed, per the repo's own deletion-safety conventions. Why: deleteMissing: true deletes every local file whose id the feed omits, and the only protection is parseModels refusing an empty list. The endpoint is public and unauthenticated, and once merged CI runs this sync every six hours; a truncated-but-valid 200 would turn a partial catalog into a mass-deletion patch. The repo guards this exact scenario elsewhere: NEAR AI "refuses a response carrying fewer than twenty models" because "a truncated list that still parses would otherwise be indistinguishable from a mass retirement", engy keeps deleteMissing: false for the same reason, and the automation's auto-merge policy caps deletions at 10 / half a provider's models. This PR itself removed 57 of ~100 files in one run — precisely the regime those limits exist for, and a future run has no floor beyond zero. Action: Add a floor or proportional refusal before removals take effect (e.g. fail the run when the feed's text_generation row count, or the pending deletion count relative to the existing catalog, is implausibly low, as NEAR AI does), or document in sync.md why RouteLLM's /v1/models cannot return a partial catalog.
• [low] [possible mistake] providers/abacus/models/zai-org/GLM-5.2.toml:1 (deleted) - Check: Evidence that the 57 removed ids are truly absent from the 2026-11-11 feed. Why: 17 of the removed entries were live at the last recorded access with no absence note — GLM-4.6, GLM-5.1, Qwen3-32B, Qwen3-Coder-480B-A35B-Instruct, and o3-pro carried "Prices refreshed … (accessed 2026-10-21)" headers, and GLM-5, GLM-5.2, GLM-4.7, o3, o3-mini, o4-mini, Qwen3.6-27B, qwen3.7-max, mimo-v2-pro, muse-spark-1.1, MiniMax-M2.7, gpt-4.1-nano carried live feed headers — so the diff itself contains recent evidence they were served and nothing showing they stopped. Removing 57 published models changes the v1 API and cannot be verified from the diff; the PR body's endpoint URL alone does not establish the feed's contents on 2026-11-11. Action: Provide the accessed feed's text_generation row count and confirmation that each removed id is absent (a saved feed snapshot, the dry-run output, or a per-id check), so the removals are reviewable; the row count also calibrates the floor requested above.

[sorenjuul]

Added the deletion-safety guard: a new opt-in SyncProvider.maxDeleteFraction, set to 0.5 for Abacus, checks the pending deletion count before any file is written and fails the run if it would remove more than half the local catalog. The 57-file removal was 42% of 136, so it passes; a truncated feed would not. sync.md documents it, and a regression test covers the refusal. Tests: 329 pass, same 8 pre-existing upstream failures; bun validate passes; dry-run 0/0/0.

On the low finding: the 57 ids were verified absent live on 2026-11-11 - GET /v1/models returned 132 rows (87 text_generation) and none of the 57 appears under any model_type (text_generation, image_generation, audio_generation, classification). The Oct headers were price refreshes, not availability evidence.

[github-actions]

Action items

• [low] [possible mistake] packages/core/src/sync/providers/abacus.ts:97 - Check: maxDeleteFraction: 0.5 vs. the migration this PR commits. Why: The merge-base Abacus catalog has 108 local files and the PR deletes 57 of them (52.8%) while creating 28. With the guard as merged (it throws before any file is written), a sync run against the pre-migration catalog computes 57 deletable > 54 (50% of 108) and refuses — so neither the 57 deletions nor the 28 creates happen. The committed state was only reachable because the deletions (patch 11) landed before the guard was added (patch 14); the PR body ("This adds that module and runs its first sync... 57 removed") and the new sync.md note present the module as the mechanism that produced a state the merged module now refuses to produce, with a misleading "source may be truncated" error for what is a legitimate migration. Post-merge runs are unaffected (local catalog already matches the feed), but anyone re-verifying the PR, partially reverting it, or facing a future legitimate >50% retirement hits a hard failure with no documented workaround. Action: Document the sequencing and the sanctioned path for a legitimate mass cleanup in the Abacus Notes (e.g., the 57-file deletion was committed ahead of the guard; a >50% retirement requires hand-deleting the files first or temporarily raising maxDeleteFraction), and correct the sync.md/PR framing so maintainers know the guard did not approve this migration and cannot reproduce it.

[sorenjuul]

Good catch on reproducibility. I replaced the deletion-ratio guard with a source-size floor: minSourceFraction: 0.5 fails the run when the feed has fewer than half as many models as the local catalog, checked before any write. The pre-migration catalog has 108 files and the feed has 87 rows (80%), so a fresh first sync still performs all 28 creates and 57 deletions; only a feed under 54 rows is refused. sync.md and the PR body now say this, and the test covers both the refusal and a legitimate mass deletion. Tests: 329 pass, same 8 pre-existing upstream failures; bun validate passes; dry-run 0/0/0.

[github-actions]

No actionable findings.

[sorenjuul]

Friendly nudge for @rekram1-node and @Tarquinen - I know you are both deep in other work, so no pressure at all, but this one is ready for a final look whenever you have a moment.

Since the earlier pings it has grown a little and is fully green:

• 28 models created (including claude-fable-5-2 and gemini-4-carbon) and 57 ids the RouteLLM feed no longer lists removed.
• The sync is now feed-authoritative, with two safety rails: intentional image/transcription skips are retained through an opt-in flag, and a feed with fewer than half the local catalog's models fails before writing anything.
• 0 commits behind dev; both checks pass and the reviewer says "No actionable findings". bun validate passes, Abacus tests are 16/16, the full suite has only the 8 failures that also reproduce on pristine upstream/dev, and the Abacus dry-run is 0/0/0.
• The v1 delta is 28 additions and 57 removals plus the qwen3.8-max context override. The removals are sourced from the live feed and are called out in the PR body as a data change that needs maintainer approval.

If either of you has a moment to look, or can point me to whoever owns catalog merges now, I would really appreciate it. I am also happy to split the 57 removals into a separate PR if that makes review easier - just say the word.

[github-actions]

No actionable findings.

[github-actions]

No actionable findings.