Repository navigation
Evaluate OPENCODE_EXPERIMENTAL_NATIVE_LLM in detail — adopt if it has an advantage #194
Description
Question
Should we enable OPENCODE_EXPERIMENTAL_NATIVE_LLM (OpenCode's own request layer instead of the AI SDK provider packages)? Examine it in detail and adopt it if there is a real advantage.
Context
- Currently held OFF deliberately (decided 2026-10-20) while the rest of the experimental flag set was enabled.
- Concrete pain the AI SDK layer has already caused here: a custom provider wired with
@ai-sdk/openai-compatiblefailed at runtime (t.responses is not a function) where@ai-sdk/openaiworked — provider-package quirks are a real cost. - The fork's heavy use cases: Codex OAuth models with effort levels, OpenCode Go models, Cloudflare gateway models, custom OpenAI-compatible providers.
To examine
- Feature parity per provider: streaming, tool calls, reasoning/effort controls, structured output, image input, cache tokens, error surfacing.
- Where native is ahead/behind the SDK packages today (changelog + source), and how fast the gap is moving.
- Failure modes: does native surface errors better than the SDK (which has hidden a 402 as a silent multi-minute stream hang in our testing)?
Benchmark plan (required before a decision)
- Same prompt set (small/medium/agentic coding tasks) through 3+ providers, native on vs off: time-to-first-token, tokens/sec, total time, error rate, cost accounting accuracy.
- Note any behavioral differences in tool-call formatting or reasoning-effort mapping.
Decision criteria
Adopt if: parity on the providers above + measurably better latency, error surfacing, or one less class of SDK translation bugs. Otherwise record why not and re-check in a release cycle or two.
Acceptance criteria
- Feature-parity matrix for the fork's providers (Codex OAuth, OpenCode Go, Cloudflare gateway, custom OpenAI-compatible) documented in this issue — streaming, tool calls, reasoning/effort controls, structured output, image input, cache tokens, error surfacing — native on vs off.
- Benchmark results posted for the same prompt set through at least 3 providers, native on vs off: time-to-first-token, tokens/sec, total time, error rate, cost-accounting accuracy.
- An adopt / do-not-adopt decision recorded in this issue with the evidence; if adopted, the flag is enabled in config with a one-line rollback note.
Activity
dezren39 commentedon Oct 26, 2026
🤖 Intake review — fleet monitor, 2026-10-26 run
Still live: searches this run found no merged PR evaluating or adopting
OPENCODE_EXPERIMENTAL_NATIVE_LLM; the flag remains deliberately OFF per the 2026-10-20 decision recorded in the body.Related: the experimental flag set enabled 2026-10-20 (context in body); the
@ai-sdk/openai-compatibleruntime failure (t.responses is not a function) is the motivating SDK-quirk example.Notes / risks: benchmark validity is the main risk — keep the prompt set and providers identical across the on/off runs or the comparison is meaningless. Provider behavior drifts between releases, so date the parity matrix.
Suggested approach: parity matrix from source + changelog first (cheap), then the benchmark plan in the body, then the decision against the body's own criteria.
dezren39 commentedon Oct 28, 2026
🤖 Agent note · triage verdict: MOOT as framed — closing with evidence
OPENCODE_EXPERIMENTAL_NATIVE_LLMswitch left to evaluate or adopt in this fork's current code, so the required benchmark (native on vs off) cannot be run — the "off" state the 2026-10-20 decision held is the only state the variable has.b0633cc697, verified 2026-10-28):git grep NATIVE_LLMover the whole tree finds the name only in translated docs (packages/web/src/content/docs/<locale>/cli.mdx, which follow the locale-sync flow) — no reader in code, config, workflows, or scripts. Setting it silently does nothing.cli.mdxas one of twelve v1-era variables "the v2 code never reads — setting one silently did nothing (docs: OPENCODE_SERVER_USERNAME is promised but never read — basic-auth username is hardcoded to 'opencode' #103, fix(ci,desktop): filewatcher-disable sets a dead env name — OPENCODE_EXPERIMENTAL_DISABLE_FILEWATCHER is read nowhere #104, docs: OPENCODE_ENABLE_EXA / OPENCODE_ENABLE_PARALLEL / OPENCODE_EXPERIMENTAL_BASH_DEFAULT_TIMEOUT_MS / OPENCODE_EXPERIMENTAL_LSP_TOOL are promised but read nowhere #105, docs: eleven more OPENCODE_* variables in the cli.mdx env table are read nowhere — DISABLE_PRUNE, DISABLE_AUTOCOMPACT, PERMISSION, TUI_CONFIG and seven others #114, fix(desktop): OPENCODE_EXPERIMENTAL_ICON_DISCOVERY / OPENCODE_EXPERIMENTAL_FILEWATCHER are set on every launch but read nowhere #115, docs: eleven remaining OPENCODE_EXPERIMENTAL_* variables in the cli.mdx Experimental table are read nowhere #116)", and addedscript/check-doc-env-vars.ts, a CI guard that fails if the docs ever promise anOPENCODE_*variable nothing reads again.packages/core/src/session/model-request.tsroutes through@opencode/ai(packages/ai: ownprotocols/,providers/,route/) for the main path, whilepackages/corestill depends on a few AI SDK packages for specific providers (@ai-sdk/openai-compatible,@ai-sdk/cohere,@ai-sdk/perplexity,@ai-sdk/vercel, plusprovider/provider-utils—packages/core/package.json:78,99-103). So "native vs SDK" is no longer one flag; it is a per-provider wiring question insidepackages/ai/sdk-factory.@ai-sdk-backed provider (e.g. openai-compatible, the one whoset.responses is not a functionfailure motivated this issue) should migrate to apackages/aiprotocol — is a different, per-provider question; say the word and it gets filed as its own scoped issue with that provider's parity surface, rather than reviving a dead flag.