bug: SSH server accept loop exits permanently on transient errors (same class as #2337) #2372

Agent Diagnostic

Identified during review of PRs #2369 and #2370 (fix for #2337). Three independent review agents flagged the same vulnerability in the SSH server accept loop.

Finding: crates/openshell-supervisor-process/src/ssh.rs:141 uses ? on listener.accept().await, so any transient error (EMFILE, ENFILE, ECONNABORTED) propagates out of run_ssh_server() and permanently kills the SSH accept loop. The spawned task in crates/openshell-supervisor-process/src/run.rs:238 is fire-and-forget — the JoinHandle is dropped, so the sandbox has no way to detect that the SSH server has exited.

This is the same class of bug as #2337, but in the exec relay path instead of the proxy path.

Description

What happens: When the SSH server's TCP listener hits a transient error like EMFILE (too many open file descriptors), the accept loop exits permanently. The sandbox continues to report Ready even though all subsequent exec relays will fail. The OCSF event is emitted at Critical severity, but no recovery or sandbox termination occurs.

What should happen: The SSH server should either:

1. Retry with bounded backoff when FD pressure clears (matching the proxy fix from fix(proxy): retry with backoff on transient accept errors instead of exiting #2369), AND
2. Notify the sandbox main loop if the accept task exits unexpectedly, so the sandbox terminates rather than staying misleadingly Ready (matching the proxy fix from fix(sandbox): terminate sandbox when proxy accept loop exits unexpectedly #2370)

Why this matters: The proxy and SSH server share the supervisor's FD table. FD exhaustion triggered by proxy connections (the scenario in #2337) can simultaneously kill the SSH server. PR #2369 keeps the proxy alive through backoff, but during the backoff period the SSH accept can still fail and exit permanently. The proxy recovers, but exec relays are dead.

This means the invariant from #2337 — "Ready must imply the sandbox's required policy proxy and exec relay are still operational" — is only enforced for the proxy path, not the SSH/exec relay path.

Reproduction Steps

1. Start a sandbox with both proxy and SSH server enabled
2. Exhaust file descriptors in the supervisor process (e.g., via sustained MCP/tunnel traffic as described in bug(proxy): EMFILE stops the sandbox proxy while phase remains Ready #2337)
3. The SSH server's listener.accept().await returns EMFILE
4. The ? operator propagates the error, exiting run_ssh_server()
5. The OCSF log is emitted but the sandbox stays Ready
6. All subsequent openshell exec commands fail because the SSH server is gone

Environment

• Affects all compute drivers (Docker, Podman, Kubernetes, VM)
• Same FD pressure scenario as bug(proxy): EMFILE stops the sandbox proxy while phase remains Ready #2337
• Relevant code: crates/openshell-supervisor-process/src/ssh.rs:141, crates/openshell-supervisor-process/src/run.rs:238-262

Suggested Fix

Apply the same two-layer defense pattern from PRs #2369 and #2370:

1. Retry with backoff (ssh.rs): Replace listener.accept().await.into_diagnostic()? with a sleep-and-continue pattern matching proxy.rs. Reuse or mirror is_fd_exhaustion_error() and accept_backoff().

2. Exit notification (ssh.rs + lib.rs): Add a oneshot drop-guard channel (matching the _proxy_exit_guard pattern from PR fix(sandbox): terminate sandbox when proxy accept loop exits unexpectedly #2370). Store the JoinHandle or exit receiver and race it in the sandbox's tokio::select! wait paths.

Related

• bug(proxy): EMFILE stops the sandbox proxy while phase remains Ready #2337 — Original EMFILE proxy bug (parent issue)
• fix(proxy): retry with backoff on transient accept errors instead of exiting #2369 — Step 1 fix: proxy accept loop retry with backoff
• fix(sandbox): terminate sandbox when proxy accept loop exits unexpectedly #2370 — Step 2 fix: sandbox termination on proxy death