inference set --model passthrough — preserve client-supplied model in managed inference

[mogul]

Feature Request: inference set --model passthrough — allow client-supplied model names to be forwarded as-is

Problem Statement

When an agent (OpenCode, Claude Code, etc.) sends a chat completion request to the managed inference endpoint (inference.local), the Privacy Router unconditionally rewrites the model field to the value configured via openshell inference set --model <m>. This means the sandbox cannot dynamically select models — every inference request uses the same model, regardless of what the client requests.

For agents that need to use different models for different tasks (e.g., a cheap model for file summarization, a powerful model for code generation), the only current options are to:

• Reconfigure openshell inference set per session (not runtime-switchable)
• Use direct external inference (bypasses credential injection entirely)

This is a missing capability in the managed inference path that reduces the usefulness of the Privacy Router for multi-model agent workflows.

Proposed Design

Introduce a sentinel model value "passthrough" that, when set via openshell inference set --model passthrough, signals the Privacy Router to preserve the model field from the client's request body rather than overwriting it.

User-facing interface:

# Configure inference to forward the client's requested model as-is
openshell inference set --provider openai --model passthrough

Routing behavior change — in crates/openshell-router/src/backend.rs, the prepare_backend_request function currently does:

obj["model"] = route.model.clone().into();

With the passthrough feature, when route.model == "passthrough", the router would skip this assignment, leaving the client-supplied value intact (which may be obj["model"] from the original deserialized request, or some reasonable default if absent).

The credential injection (stripping caller API key, injecting backend API key) would continue to work as before — the feature is strictly about the model field passthrough.

Scope boundary: The passthrough only affects the model field. The provider selection (--provider) and API key injection remain fully managed.

Alternatives Considered

1. Per-request model override in the request body — e.g., a custom header like X-OpenShell-Model. This is more complex to implement, requires client-side changes, and introduces a non-standard protocol extension.

2. Multi-model provider configuration — allow openshell inference set to accept multiple model values. This is a larger feature (Discussion Provider Enhancements -- Declarative Profiles, Auto-Injected Policy, Multi-Provider Inference #865 touches on multi-provider support) and would require provider-level mapping logic, credential bundles per model, etc. The passthrough sentinel is a simpler first step.

3. Direct external inference — already works, but loses the credential-injection and audit-log benefits of the Privacy Router. Users who want managed inference with dynamic models have no path today.

The passthrough sentinel is the minimal, backward-compatible change that unblocks multi-model workflows without requiring a new protocol or a major provider architecture change.

Agent Investigation

Used create-spike to investigate the inference routing path. Key findings:

• Model rewrite lives in crates/openshell-router/src/backend.rs, function prepare_backend_request, which unconditionally sets obj["model"] = route.model for all standard routes (OpenAI, Anthropic, NVIDIA, etc.)
• The route.model value comes from openshell inference set --model via crates/openshell-server/src/inference.rs
• The Privacy Router always injects backend credentials and strips caller credentials — this behavior is independent of the model field
• Published docs at docs.nvidia.com/openshell/sandboxes/inference-routing explicitly state the rewrite behavior
• No existing sentinel or passthrough mechanism exists in any of the three routing paths (standard, Vertex rawPredict, AWS Bedrock)
• Searched GitHub issues, PRs, and discussions for keywords: model rewrite, model override, passthrough, any-model, model selection — no existing issue or PR requests this capability

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request

[johntmyers]

📋 triage-agent

Closing as superseded by the provider-native inference architecture merged in #3195. OpenShell no longer exposes openshell inference set, pins a route model, or rewrites the client model through the removed privacy router. Clients now send model selection directly to the attached provider native endpoint.