Custom OpenAI-compatible model: /thinking offers high, which breaks llama.cpp Qwen3.8 template, and the failure is retried 5 times #53202

Activity

1. github-actions

   assigned

   jlongster

   on Oct 4, 2026
2. github-actions

   added

   needs:complianceThis means the issue will auto-close after 2 hours.This means the issue will auto-close after 2 hours.

   and removed

   needs:complianceThis means the issue will auto-close after 2 hours.This means the issue will auto-close after 2 hours.

   on Oct 4, 2026
3. github-actions commented on Oct 4, 2026

   

   github-actionsBot

   on Oct 4, 2026 – with GitHub Actions

   Contributor

   More actions

   Thanks for updating the issue with the required description, reproduction steps, and system information!
4. opencode-agent commented on Oct 5, 2026

   

   opencode-agentBot

   on Oct 5, 2026 – with OpenCode Agent

   Contributor

   More actions

   Thanks for the detailed report. Until we improve the defaults, you can work around the first issue by declaring the variants yourself.

   When a custom model has no variants, opencode falls back to a generic low / medium / high effort list for openai-compatible providers. As soon as you set variants on the model, that fallback is skipped and /thinking shows only the variants you define. Each variant's settings.reasoningEffort is sent as reasoning_effort, so you can match exactly what the Qwen3.8 template accepts:

   "providers": {
     "llamaswap": {
       "package": "@opencode/ai/providers/openai-compatible",
       "settings": { "baseURL": "http://127.0.0.1:8080/v1" },
       "env": ["LLAMACPP_API_KEY"],
       "models": {
         "qwen-code": {
           "modelID": "qwen-code",
           "limit": { "context": 65536, "output": 4096 },
           "variants": [
             { "id": "low", "settings": { "reasoningEffort": "low" } },
             { "id": "medium", "settings": { "reasoningEffort": "medium" } },
             { "id": "xhigh", "settings": { "reasoningEffort": "xhigh" } }
           ]
         }
       }
     }
   }

   Default still sends no effort, so the template's own default applies. A variant can also set body (merged into the request JSON) or headers if a backend needs something other than reasoning_effort, for example "body": { "chat_template_kwargs": { ... } }.

   We'll also look at making the built-in variant logic less presumptuous for custom OpenAI-compatible models, and at the retry behavior for deterministic 5xx errors.
5. opencodejr commented on Oct 27, 2026

   

   opencodejrBot

   on Oct 27, 2026

   More actions

   Thanks for the detailed report and for offering a PR!

   As far as I can tell, both behaviours are how v2 currently works on purpose, so I'm treating this as an improvement request:

   1. Effort levels on custom models. If a custom OpenAI-compatible model sets neither variants nor capabilities.reasoning, OpenCode assumes it supports the generic low / medium / high efforts. You can replace that list with the efforts your template accepts, which also makes xhigh selectable:

      "models": {
        "qwen-code": {
          "modelID": "qwen-code",
          "limit": { "context": 65536, "output": 4096 },
          "capabilities": {
            "reasoning": { "supported": true, "effort": ["low", "medium", "xhigh"] }
          }
        }
      }

   2. Retrying the 500. In v2 the error is classified in packages/ai/src/provider-error.ts (packages/opencode/src/session/retry.ts isn't in the current repo). Every status >= 500 is treated as a temporary provider error and retried. Here llama.cpp sends a 500 for a request that can never succeed, so not retrying would mean recognising template errors in the response body. That's a reasonable idea, and maintainers will decide whether that kind of body check fits.

   Related issues:

   • Context overflow 400 is retried because the error message contains "500000" (substring match of "500") #52464: a different case of a deterministic error being retried
   • Infinite retry loop on HTTP 500 causes opencode to hang indefinitely #25041: earlier request to stop always retrying 5xx errors
   • core: flatten reasoningEffort settings for custom OpenAI-compatible providers #48183: reasoning settings for custom OpenAI-compatible providers (fixed in 2.0.0)
6. jokaperes commented on Nov 4, 2026

   

   jokaperes

   on Nov 4, 2026

   More actions

   Another affected provider: Refuseless, with OpenCode v2.2.5 on macOS. This also affects clients consuming model.list (observed in T3 Code).

   With a custom @opencode/ai/providers/openai-compatible provider at https://api.refuseless.com/v1, and neither variants nor capabilities.reasoning declared, opencode api model.list returns low, medium, high, each mapped to the same settings.reasoningEffort. Selecting Medium produced:

   reasoning_effort must be one of off (or none), low, high, or max.
   

   Affected model: deepseek-v4.1-flash-abliterated-free. Refuseless documents that Medium is rejected, and publishes per-model reasoning_efforts / reasoning_default: https://docs.refuseless.com/guides/reasoning and https://docs.refuseless.com/api/models.

   Confirmed the existing workaround on v2.2.5: adding this under the model's capabilities makes both a fresh standalone model.list and the active service after opencode reload return exactly Off/Low/High/Max:

   "reasoning": { "supported": true, "effort": ["off", "low", "high", "max"] }

   Verification covered catalog output, without a new inference request. The provider rejection above came from the original failed request. The fallback is still present in packages/core/src/config/plugin/provider.ts, with the generic effort list in packages/core/src/variant.ts. Please consider this provider when revisiting inferred reasoning support for custom models.