You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ship a self-contained, local-first WebUI that feels at home on macOS 27 while remaining a browser application on supported platforms. Balance convenience (discover → load → chat), control (explicit model lifecycle and scoped settings), and information (honest readiness, resource use, context, and request timings). This issue defines the product and integration contract; child issues define bounded ownership and executable acceptance criteria. No native wrapper is required.
Reviewed baseline and reuse decisions
Reviewed on 2026-09-12: lablup/mlxcel main 1f1077101151f454450a67962ae07f06362a68f3; reference lablup/continuum-router local checkout c79e54d0f9920953c417acd57b0a60421e485c93. Findings are source review, not a new runtime benchmark. Rebase the implementation survey when work starts; issue closure alone is not evidence that code landed.
Reuse the in-process RouterPool, cache, presets, load/unload/download/remove, and model SSE. Do not build a process supervisor or a second model registry. router_front.rs is a different, distributed tokenizer-bearing service and is NOT this foundation.
--models-dir scans immediate model children, cache uses owner/name, source precedence is cache < models-dir < preset. Downloaded, architecturally supported, complete, runnable on this backend, and loaded are different facts. mlxcel list is not a support catalog.
Lifecycle
begin_load, unload_entry, dispatch, SSE broadcast
Dropping the pool reference does not prove worker exit or memory release: requests retain clones. Introduce observed lifecycle completion, drain accounting, and race-safe operations. Existing SSE has no initial snapshot or durable replay and silently skips lag.
Reuse typed mutable/read-only settings and existing metrics. Settings PATCH can partially apply; a successful HTTP status is not all-fields success. #1800 is CLOSED as of this survey; verify its implementation before promising live occupancy. #1815 remains OPEN and is not a prerequisite: display actual resolved context rather than implement KV arithmetic here.
Reuse embedded, offline, same-origin assets; public login shell/private APIs; ETag and missing-file behavior; optional dependency feature isolation. Do not copy its admin backend, Alpine expressions, permissive CSP, or persistent bearer storage. Its debug-embed comment conflicts with the crate's documented embedding semantics: test the artifact instead of copying that comment.
Product and command contract
mlxcel-server --webui with no model/source/preset preload starts a model-free control plane using the existing model-store resolution. It does not fetch weights, load a tokenizer/provider, warm up a model, or enumerate arbitrary home/repository directories. An absent default store is an empty library, not a startup failure; create it only on an explicit download. An invalid explicit root is an actionable error.
mlxcel-server --webui --models-dir models/mlx --model-store-root <cache> discovers the explicitly selected directory and cache without silently treating either as the other. Respect explicit preset load-on-startup, and disclose that this is an opt-in exception to the empty-start guarantee. A preset-only invocation must work consistently in both binaries.
mlxcel-server --webui -m <checkpoint> preserves the existing single-model server semantics. Chat, observation, and enabled live settings work; switching, downloading and unloading are unavailable with a reason and instructions for model-free mode. Never create a duplicate pool around an already-loaded provider.
Runtime UI is opt-in. --ui is an alias, --no-ui/--no-webui explicitly disable it, and CLI precedence over LLAMA_ARG_UI plus last conflicting CLI occurrence is tested. A Cargo webui feature is included in ordinary shipped/default binaries; --no-default-features excludes UI assets/dependencies and rejects an enabling flag clearly. Keep unrelated flags such as tools, agent, proxy, arbitrary --path, and UI-config formats unsupported with precise diagnostics; enabling WebUI does not enable code execution.
Serve the shell at {api_prefix}/webui/, redirect only the missing trailing slash, and use hash navigation. Preserve / health behavior and existing API paths. api_prefix is the server's validated prefix, not a browser-supplied URL. Print the safe access URL after binding; do not auto-open a browser, put credentials in a URL, or change normal API-only defaults. Reject WebUI on UNIX-socket-only listeners with an actionable TCP/reverse-proxy instruction for v1.
Empty-start browsing and polling never autoload a model. UI inference, props, slots, metrics and settings explicitly use autoload=false in pool mode and carry a model ID. Model selection is browser state; loading requires an explicit action. Closing a browser tab never unloads a shared model.
Information architecture and interaction contract
Desktop: a 224–280 px sidebar, restrained top toolbar, content pane, optional 280–360 px inspector. Navigation contains Models, Chat, Activity, and Settings; the sidebar footer shows connection and backend identity. The top bar identifies the selected model and its actual state. No dashboard full of decorative cards and no fake macOS traffic lights.
Screen
Primary path
Progressive detail
Models
Search/filter local library; inspect; Load; Use in Chat; Unload
Source/root, exact architecture, input/output tasks, quantization, disk size versus memory estimate, completeness/support reason, active requests, load error and retry. Add Model accepts a HuggingFace repo ID; network only after explicit consent.
Chat
New conversation, model picker, stream, Stop, copy, edit/regenerate
System prompt and per-request sampling; distinct content/reasoning/tool-call views; actual token usage/finish reason; optional image attachment only when the selected backend/model supports it.
Activity
Current loads/downloads/drains and active request summary
Bounded operation history, observed TTFT/decode rate, per-slot context and aggregate pool metrics with units, timestamps, provenance and unknown states. No invented GPU utilization or completion percentages.
Settings
Appearance, privacy, enabled server controls
Separate browser preferences, request parameters, loaded-model live settings, next-load profile and restart-only server flags. Show effective values and scope before applying.
Every async view has empty/loading/ready/partial/stale/error/unauthorized/offline states and a recovery action. Busy and destructive controls have inline reasons. Changing selected models never reroutes an in-flight conversation. Deletion names the checkpoint and source, explains unload versus disk deletion, and is allowed only for managed cache entries.
Visual and accessibility contract
Target the macOS 27 design language, not a claim of AppKit rendering in CSS. Apple describes Liquid Glass as a navigation/control material; implement that hierarchy, concentric rounded geometry, neutral surfaces, clear selection and keyboard behavior. Apple Materials, WWDC26 AppKit updates, and macOS 27 design kits are dated references, not licenses to redistribute Apple assets.
Use system fonts (-apple-system, BlinkMacSystemFont, system fallback), 4 px spacing scale, shared semantic color/material/elevation/radius/motion tokens, and licensed bundled icons. Glass belongs on sidebar/toolbar/popovers; transcript, tables and forms get stable readable surfaces. Default light/dark follows the OS, with explicit override. Provide glass intensity (0–100, default 35), reduce-transparency and reduce-motion overrides and high-contrast support; browser detection of OS preferences is best-effort and must not be claimed as access to native macOS settings. Contrast settings override decorative intensity. With no backdrop-filter, use an opaque fallback. Motion is 120–180 ms, no shimmer or continuous backdrop animation, disabled with reduced motion. The exact color/radius table and screenshots become a shared artifact in the design-system child; page implementations cannot invent alternate tokens.
Keyboard-first navigation, visible focus, focus restoration, labeled controls, correct dialog/listbox behavior, IME-safe composition, WCAG 2.2 AA contrast, 200% zoom, screen-reader-friendly streaming summaries, and accessible error text are release gates. Compact below 960 px; at 390 px use an off-canvas sidebar and stacked inspector with no page-level horizontal overflow. Test light/dark, tinted/opaque, high contrast, long model names, CJK, empty and failure views. A browser cannot reproduce native behind-window refraction; do not spend the scope on a shader imitation.
Architecture and canonical boundaries
Frontend: webui/, React + strict TypeScript + Vite static build, pnpm lockfile, CSS tokens and shared accessible components. No SSR, Tauri/Electron, Node service, CDN fonts/scripts, remote telemetry or service worker. Prefer ordinary typed reducers/context and one query/cache layer over multiple state libraries. Rationale: testable typed workflows and deterministic shared components rather than copying the reference router's monolithic script model. Pin exact toolchain/package-manager versions when scaffolding; review current versions, do not use floating latest in CI.
Bundle: reproducible checked-in generated src/webui/assets/ plus a source/build manifest; rust-embed with embedded bytes in debug and release, behind webui. Cargo never executes pnpm or downloads frontend artifacts. UI developers rebuild the asset tree explicitly; CI rebuilds in a temporary output directory and fails on drift. The website under webpage/ remains unrelated. Vite relative base and hash routes support validated API prefixes without HTML string injection. Vite relative-base documentation and rust-embed embedding semantics inform these choices.
Rust serving: src/server/webui/ owns static routes/security and thin UI adapters. Existing RouterPool, downloader, config and metric owners remain the only source of runtime truth. No shell commands, arbitrary filesystem path APIs, foreign engine supervisor or duplicated inference handlers.
Data plane: existing OpenAI-compatible chat and model-scoped props/settings/slots/metrics routes. UI-specific control/status contracts live under {api_prefix}/ui-api/v1, opt-in with WebUI. Compatibility endpoints retain their shapes and status codes; both facades invoke shared operations. Extensions must not be bolted into /v1/models with fabricated OpenAI semantics.
Contract-first gate: the first child checks in an OpenAPI/JSON-schema contract, examples and machine-readable lifecycle fixtures under docs/webui/ and tests/fixtures/webui/; generate TypeScript DTOs and validate Rust responses against it. Breaking contract changes require fixture/client/server changes in one coordination PR before consumers continue. UI drafts can use those fixtures, but screenshots from mocks do not prove runtime acceptance.
Required UI adapter contract
Endpoint below /ui-api/v1
Required semantics
GET /bootstrap
Authenticated server instance ID, schema version, mode/backend/build, canonical relative API base, enabled features/actions and read-only reasons, effective roots with sensitive parts redacted. No secrets, weights or model initialization.
GET /catalog and GET /catalog/{id}
Paginated deterministic inventory (default 50, max 200), opaque stable entry ID, actual inference ID, source, support/completeness/task capabilities, lifecycle and nullable measured metadata. Filtering never downloads or loads.
POST /catalog/refresh
Bounded background rescan job; no mutation-on-GET in the UI.
POST /model-actions
`{model_id, action: load
POST /downloads
Validated {repo_id, revision?, idempotency_key} with fixed configured store; returns an operation. Repo metadata and revision are pinned before writing weights. No URL/path or code-execution field.
POST /model-removals
{model_id, expected_revision, idempotency_key}; cache-only deletion operation. Reject if busy/loading/downloading until separately drained/cancelled. UI confirmation is additional to server-side checks.
GET /operations, GET /operations/{id}, POST /operations/{id}/cancel
Bounded observed progress and terminal result. Cancellation is acknowledged only after the underlying worker stops; unsupported cancellation returns a typed reason.
GET /events
Authenticated SSE with instance ID, monotonic sequence/revision, event type and typed payload; snapshot/reconnect semantics are defined below.
GET /runtime?model_id=...
Typed observation snapshot projected from existing settings/slot/cache/worker counters, no hidden sampling work or autoload. Null plus reason for unsupported metrics, timestamp and measurement scope for every group.
Versioned errors are {error:{code,message,retryable,field_errors?,operation_id?},request_id}. Use 400 invalid syntax, 401 authentication, 403 forbidden, 404 unknown, 409 stale/conflicting action, 422 unsupported model/action/configuration, 429 resource/rate limit, 503 unavailable. Redact paths/tokens from client errors. IDs used in URLs are opaque; do not interpolate repo IDs into path segments. Shared operation states are queued/running/cancelling/succeeded/failed/cancelled; lifecycle states are unloaded/loading/ready/draining/unloading/failed. Download/install state is a separate axis (absent/downloading/complete/incomplete/failed) and must not overwrite inference readiness.
Canonical concurrency rules: serialize conflicting actions per model; deduplicate identical idempotency keys for the server session; reject stale revisions; count loading and draining resources against capacity until worker exit. Never hold registry/settings locks across network/GPU waits. An unload stops new admission, drains existing requests, awaits worker shutdown and only then marks unloaded; drain timeout reports a recoverable blocked/failure result and does not pretend to free memory. Loading cannot be forcibly cancelled unless the loader cooperates; report this honestly, with unload-after-load as a distinct requested action if implemented. A failed replacement does not promise rollback to a model already released. Existing compatibility actions participate in the same synchronization.
SSE uses a bounded ring (1,024 events, 10-minute retention) and operations retain the last 200 terminal records for one hour; active operations are never silently discarded. Unknown/gapped event IDs or changed server instance force a full authoritative resnapshot. Reconnect cannot re-POST an action. Polling fallback uses one shared loop, 2 seconds visible / 30 seconds hidden, with backoff and no overlap. Snapshot and stream subscription must have a tested sequence fence so no transition is lost between them. Limits and retention are part of the contract, not per-page defaults.
Security and privacy baseline
WebUI mode is an administrative surface. Its shell/assets may be public but every bootstrap, model read, control, settings and event route requires bearer authentication. With WebUI and no configured key on a loopback listener, generate a strong session-only key, protect all non-public APIs with it and present it once to the operator on the local terminal (never structured logs, query strings or asset HTML). A non-loopback WebUI requires an explicitly configured key and server TLS; alternatively use a documented loopback backend behind a TLS reverse proxy. UI-off behavior stays unchanged. Do not invent an implicit CORS wildcard or trust arbitrary forwarded headers. Credentials are memory-only in the browser; refresh requires re-authentication. All keys remain administrator-equivalent in v1; multi-user roles are not implied.
Apply exact Host/Origin/Fetch-Metadata checks to browser-accessible APIs, including compatibility model mutation routes and GET /models?reload=..., when UI mode is enabled; preserve bearer-authenticated non-browser clients without Origin. Loopback is not a CSRF/DNS-rebinding defense. CSP has no unsafe-eval or inline scripts; markdown is sanitized, raw HTML off, remote images blocked by default, no active SVG/HTML attachments. Metadata and model output are untrusted. No credentials/prompts in URLs, browser persistence, diagnostics or server event logs. Conversations are memory-only by default; explicit local history opt-in uses versioned IndexedDB with quota/error handling and Clear All; never store API keys there.
Scope boundaries
Required: local and managed-cache model inventory, explicit public-Hub download/cancel/cache delete, safe load/unload, text chat and capability-gated image input, live observation and existing mutable settings, explicit next-load request profiles, accessible glass design and offline single-binary delivery. Other model tasks (embeddings/rerank/audio/image generation) appear with correct capability descriptions and copyable API examples, not misleading chat buttons. No execution of tool calls; display/copy them only. Private/gated Hub authentication, Hub marketplace search, autonomous tools/MCP (#1457), multi-user accounts, arbitrary file browsing, distributed topology editing, native app packaging, remote engine management and hot mutation of worker-owned settings are deferred. The UI may explain these limits; it must not ship enabled placeholders.
Delivery and shared definition of done
Child issues below are a dependency DAG. The contract child lands first; only then may disjoint backend/frontend work use the same fixtures in parallel. Central shared files (Cargo.toml, lockfiles, server route registration, CLI plumbing and shared UI tokens/client) have explicit owners; do not independently rewrite them in page PRs. Each issue is one branch and one PR, each PR closes only its child and references this epic. Read the whole epic and child discussion before implementing, and preserve unrelated work.
All children completed and the final integration gate demonstrates the full no-model → discovery/download → load → chat → stop → drain/unload flow on an actual bundled binary.
Unit/contract/component tests, negative security cases, browser visual/accessibility tests, and real-checkpoint tests are attached to the implementing PRs; no skipped model test is counted as a pass.
Rust gate: cargo test --workspace --profile test-fast --features metal,accelerate, workspace/all-target clippy with the relevant features, fmt, compatibility/feature/asset checks; CUDA smoke on its host. Do not run Metal test binaries concurrently. Arithmetic is not in scope; if touched, the teacher-forced trace contract in docs/benchmarks.md applies.
Bundled UI works without network, node_modules, source checkout or external web assets (normal platform MLX runtime resources remain required). Plain UI-disabled server behavior has regression coverage.
Safari on macOS 27 plus Chromium and Firefox, 390/1024/1440 widths, keyboard/IME/screen-reader checks, default opaque fallback, and measured performance budgets pass the final child.
Documentation and compatibility manifest explain the new flags, security defaults, roots, mode limits and local history; NOTICE retains third-party licenses. No claim of byte-identical native AppKit rendering.
The design-system shell (#1843) must integrate the shared authentication provider from #1842 rather than ship a static login placeholder or duplicate credential state. Its final integration and acceptance therefore depend on #1842; independent component work already underway can continue, but the PR merges only after the client is on main and the real connection is verified. This changes execution order, not product scope. PR #1857 closes #1836 and PR #1860 closes #1839; both are merged.
Goal
Ship a self-contained, local-first WebUI that feels at home on macOS 27 while remaining a browser application on supported platforms. Balance convenience (discover → load → chat), control (explicit model lifecycle and scoped settings), and information (honest readiness, resource use, context, and request timings). This issue defines the product and integration contract; child issues define bounded ownership and executable acceptance criteria. No native wrapper is required.
Reviewed baseline and reuse decisions
Reviewed on 2026-09-12:
lablup/mlxcelmain1f1077101151f454450a67962ae07f06362a68f3; referencelablup/continuum-routerlocal checkoutc79e54d0f9920953c417acd57b0a60421e485c93. Findings are source review, not a new runtime benchmark. Rebase the implementation survey when work starts; issue closure alone is not evidence that code landed.src/bin/mlx_server.rs,src/main.rs,src/commands/serve.rs,src/server/cli_input.rs,startup.rsmlxcel-server --webui;mlxcel serve --webuiuses identical shared resolution, not a second server.src/cli/ui_compat_args.rs, closed #1435,compat/llama-server/b10621/--webuicurrently aliases hidden--uiand is intentionally rejected. Replace that classification, not add a conflicting Clap argument.router_server.rs,router_models.rs,router_cache.rs,router_presets.rsrouter_front.rsis a different, distributed tokenizer-bearing service and is NOT this foundation.discover_models, downloader store,src/models/registry.rs, detection, completeness helpers--models-dirscans immediate model children, cache uses owner/name, source precedence is cache < models-dir < preset. Downloaded, architecturally supported, complete, runnable on this backend, and loaded are different facts.mlxcel listis not a support catalog.begin_load,unload_entry,dispatch, SSE broadcast/props,/slots,/metrics,/settings,runtime_settings.rs, chat streamingProduct and command contract
mlxcel-server --webuiwith no model/source/preset preload starts a model-free control plane using the existing model-store resolution. It does not fetch weights, load a tokenizer/provider, warm up a model, or enumerate arbitrary home/repository directories. An absent default store is an empty library, not a startup failure; create it only on an explicit download. An invalid explicit root is an actionable error.mlxcel-server --webui --models-dir models/mlx --model-store-root <cache>discovers the explicitly selected directory and cache without silently treating either as the other. Respect explicit presetload-on-startup, and disclose that this is an opt-in exception to the empty-start guarantee. A preset-only invocation must work consistently in both binaries.mlxcel-server --webui -m <checkpoint>preserves the existing single-model server semantics. Chat, observation, and enabled live settings work; switching, downloading and unloading are unavailable with a reason and instructions for model-free mode. Never create a duplicate pool around an already-loaded provider.--uiis an alias,--no-ui/--no-webuiexplicitly disable it, and CLI precedence overLLAMA_ARG_UIplus last conflicting CLI occurrence is tested. A Cargowebuifeature is included in ordinary shipped/default binaries;--no-default-featuresexcludes UI assets/dependencies and rejects an enabling flag clearly. Keep unrelated flags such as tools, agent, proxy, arbitrary--path, and UI-config formats unsupported with precise diagnostics; enabling WebUI does not enable code execution.{api_prefix}/webui/, redirect only the missing trailing slash, and use hash navigation. Preserve/health behavior and existing API paths.api_prefixis the server's validated prefix, not a browser-supplied URL. Print the safe access URL after binding; do not auto-open a browser, put credentials in a URL, or change normal API-only defaults. Reject WebUI on UNIX-socket-only listeners with an actionable TCP/reverse-proxy instruction for v1.autoload=falsein pool mode and carry a model ID. Model selection is browser state; loading requires an explicit action. Closing a browser tab never unloads a shared model.Information architecture and interaction contract
Desktop: a 224–280 px sidebar, restrained top toolbar, content pane, optional 280–360 px inspector. Navigation contains Models, Chat, Activity, and Settings; the sidebar footer shows connection and backend identity. The top bar identifies the selected model and its actual state. No dashboard full of decorative cards and no fake macOS traffic lights.
Every async view has empty/loading/ready/partial/stale/error/unauthorized/offline states and a recovery action. Busy and destructive controls have inline reasons. Changing selected models never reroutes an in-flight conversation. Deletion names the checkpoint and source, explains unload versus disk deletion, and is allowed only for managed cache entries.
Visual and accessibility contract
Target the macOS 27 design language, not a claim of AppKit rendering in CSS. Apple describes Liquid Glass as a navigation/control material; implement that hierarchy, concentric rounded geometry, neutral surfaces, clear selection and keyboard behavior. Apple Materials, WWDC26 AppKit updates, and macOS 27 design kits are dated references, not licenses to redistribute Apple assets.
Use system fonts (
-apple-system,BlinkMacSystemFont, system fallback), 4 px spacing scale, shared semantic color/material/elevation/radius/motion tokens, and licensed bundled icons. Glass belongs on sidebar/toolbar/popovers; transcript, tables and forms get stable readable surfaces. Default light/dark follows the OS, with explicit override. Provide glass intensity (0–100, default 35), reduce-transparency and reduce-motion overrides and high-contrast support; browser detection of OS preferences is best-effort and must not be claimed as access to native macOS settings. Contrast settings override decorative intensity. With no backdrop-filter, use an opaque fallback. Motion is 120–180 ms, no shimmer or continuous backdrop animation, disabled with reduced motion. The exact color/radius table and screenshots become a shared artifact in the design-system child; page implementations cannot invent alternate tokens.Keyboard-first navigation, visible focus, focus restoration, labeled controls, correct dialog/listbox behavior, IME-safe composition, WCAG 2.2 AA contrast, 200% zoom, screen-reader-friendly streaming summaries, and accessible error text are release gates. Compact below 960 px; at 390 px use an off-canvas sidebar and stacked inspector with no page-level horizontal overflow. Test light/dark, tinted/opaque, high contrast, long model names, CJK, empty and failure views. A browser cannot reproduce native behind-window refraction; do not spend the scope on a shader imitation.
Architecture and canonical boundaries
webui/, React + strict TypeScript + Vite static build, pnpm lockfile, CSS tokens and shared accessible components. No SSR, Tauri/Electron, Node service, CDN fonts/scripts, remote telemetry or service worker. Prefer ordinary typed reducers/context and one query/cache layer over multiple state libraries. Rationale: testable typed workflows and deterministic shared components rather than copying the reference router's monolithic script model. Pin exact toolchain/package-manager versions when scaffolding; review current versions, do not use floating latest in CI.src/webui/assets/plus a source/build manifest;rust-embedwith embedded bytes in debug and release, behindwebui. Cargo never executes pnpm or downloads frontend artifacts. UI developers rebuild the asset tree explicitly; CI rebuilds in a temporary output directory and fails on drift. The website underwebpage/remains unrelated. Vite relative base and hash routes support validated API prefixes without HTML string injection. Vite relative-base documentation and rust-embed embedding semantics inform these choices.src/server/webui/owns static routes/security and thin UI adapters. ExistingRouterPool, downloader, config and metric owners remain the only source of runtime truth. No shell commands, arbitrary filesystem path APIs, foreign engine supervisor or duplicated inference handlers.{api_prefix}/ui-api/v1, opt-in with WebUI. Compatibility endpoints retain their shapes and status codes; both facades invoke shared operations. Extensions must not be bolted into/v1/modelswith fabricated OpenAI semantics.docs/webui/andtests/fixtures/webui/; generate TypeScript DTOs and validate Rust responses against it. Breaking contract changes require fixture/client/server changes in one coordination PR before consumers continue. UI drafts can use those fixtures, but screenshots from mocks do not prove runtime acceptance.Required UI adapter contract
/ui-api/v1GET /bootstrapGET /catalogandGET /catalog/{id}POST /catalog/refreshPOST /model-actionsPOST /downloads{repo_id, revision?, idempotency_key}with fixed configured store; returns an operation. Repo metadata and revision are pinned before writing weights. No URL/path or code-execution field.POST /model-removals{model_id, expected_revision, idempotency_key}; cache-only deletion operation. Reject if busy/loading/downloading until separately drained/cancelled. UI confirmation is additional to server-side checks.GET /operations,GET /operations/{id},POST /operations/{id}/cancelGET /eventsGET /runtime?model_id=...Versioned errors are
{error:{code,message,retryable,field_errors?,operation_id?},request_id}. Use 400 invalid syntax, 401 authentication, 403 forbidden, 404 unknown, 409 stale/conflicting action, 422 unsupported model/action/configuration, 429 resource/rate limit, 503 unavailable. Redact paths/tokens from client errors. IDs used in URLs are opaque; do not interpolate repo IDs into path segments. Shared operation states arequeued/running/cancelling/succeeded/failed/cancelled; lifecycle states areunloaded/loading/ready/draining/unloading/failed. Download/install state is a separate axis (absent/downloading/complete/incomplete/failed) and must not overwrite inference readiness.Canonical concurrency rules: serialize conflicting actions per model; deduplicate identical idempotency keys for the server session; reject stale revisions; count loading and draining resources against capacity until worker exit. Never hold registry/settings locks across network/GPU waits. An unload stops new admission, drains existing requests, awaits worker shutdown and only then marks unloaded; drain timeout reports a recoverable blocked/failure result and does not pretend to free memory. Loading cannot be forcibly cancelled unless the loader cooperates; report this honestly, with unload-after-load as a distinct requested action if implemented. A failed replacement does not promise rollback to a model already released. Existing compatibility actions participate in the same synchronization.
SSE uses a bounded ring (1,024 events, 10-minute retention) and operations retain the last 200 terminal records for one hour; active operations are never silently discarded. Unknown/gapped event IDs or changed server instance force a full authoritative resnapshot. Reconnect cannot re-POST an action. Polling fallback uses one shared loop, 2 seconds visible / 30 seconds hidden, with backoff and no overlap. Snapshot and stream subscription must have a tested sequence fence so no transition is lost between them. Limits and retention are part of the contract, not per-page defaults.
Security and privacy baseline
WebUI mode is an administrative surface. Its shell/assets may be public but every bootstrap, model read, control, settings and event route requires bearer authentication. With WebUI and no configured key on a loopback listener, generate a strong session-only key, protect all non-public APIs with it and present it once to the operator on the local terminal (never structured logs, query strings or asset HTML). A non-loopback WebUI requires an explicitly configured key and server TLS; alternatively use a documented loopback backend behind a TLS reverse proxy. UI-off behavior stays unchanged. Do not invent an implicit CORS wildcard or trust arbitrary forwarded headers. Credentials are memory-only in the browser; refresh requires re-authentication. All keys remain administrator-equivalent in v1; multi-user roles are not implied.
Apply exact Host/Origin/Fetch-Metadata checks to browser-accessible APIs, including compatibility model mutation routes and
GET /models?reload=..., when UI mode is enabled; preserve bearer-authenticated non-browser clients without Origin. Loopback is not a CSRF/DNS-rebinding defense. CSP has no unsafe-eval or inline scripts; markdown is sanitized, raw HTML off, remote images blocked by default, no active SVG/HTML attachments. Metadata and model output are untrusted. No credentials/prompts in URLs, browser persistence, diagnostics or server event logs. Conversations are memory-only by default; explicit local history opt-in uses versioned IndexedDB with quota/error handling and Clear All; never store API keys there.Scope boundaries
Required: local and managed-cache model inventory, explicit public-Hub download/cancel/cache delete, safe load/unload, text chat and capability-gated image input, live observation and existing mutable settings, explicit next-load request profiles, accessible glass design and offline single-binary delivery. Other model tasks (embeddings/rerank/audio/image generation) appear with correct capability descriptions and copyable API examples, not misleading chat buttons. No execution of tool calls; display/copy them only. Private/gated Hub authentication, Hub marketplace search, autonomous tools/MCP (#1457), multi-user accounts, arbitrary file browsing, distributed topology editing, native app packaging, remote engine management and hot mutation of worker-owned settings are deferred. The UI may explain these limits; it must not ship enabled placeholders.
Delivery and shared definition of done
Child issues below are a dependency DAG. The contract child lands first; only then may disjoint backend/frontend work use the same fixtures in parallel. Central shared files (
Cargo.toml, lockfiles, server route registration, CLI plumbing and shared UI tokens/client) have explicit owners; do not independently rewrite them in page PRs. Each issue is one branch and one PR, each PR closes only its child and references this epic. Read the whole epic and child discussion before implementing, and preserve unrelated work.cargo test --workspace --profile test-fast --features metal,accelerate, workspace/all-target clippy with the relevant features, fmt, compatibility/feature/asset checks; CUDA smoke on its host. Do not run Metal test binaries concurrently. Arithmetic is not in scope; if touched, the teacher-forced trace contract indocs/benchmarks.mdapplies.Sub-issues
Phase 1
Phase 2
Phase 3
Phase 4
Phase 5
Phase 6
Phase 7
Execution ownership and waves
Execution waves
File and review coordination
Related work
Execution refinement — 2026-09-12
The design-system shell (#1843) must integrate the shared authentication provider from #1842 rather than ship a static login placeholder or duplicate credential state. Its final integration and acceptance therefore depend on #1842; independent component work already underway can continue, but the PR merges only after the client is on main and the real connection is verified. This changes execution order, not product scope. PR #1857 closes #1836 and PR #1860 closes #1839; both are merged.