feat: sandbox ingress — expose services from inside the network namespace #994

Problem Statement

The sandbox network namespace is egress-only. A process running inside the sandbox (an HTTP API, a webhook receiver, an MCP tool server, a development web server) is unreachable from outside the namespace. There is no mechanism — at the kernel, Kubernetes, or API level — for external traffic to reach a listening socket inside the sandbox.

This matters because agents increasingly need to serve, not just consume:

• Webhook-driven workflows. An agent registers a GitHub webhook to react to push events. The webhook payload must reach the agent's HTTP server inside the sandbox. Today this is impossible without running the server outside the sandbox, defeating the purpose of sandboxed execution.
• MCP tool servers. An agent exposes tools via the Model Context Protocol. Other agents or orchestrators in the cluster need to reach the MCP endpoint. The sandbox blocks all inbound paths.
• Development servers. An agent stands up a preview server (Next.js, Flask, etc.) for a human to review. The human cannot reach it without SSH port forwarding via openshell forward, which only works for local access — other pods, load balancers, and CI runners cannot use the SSH tunnel.
• Inter-agent communication. In multi-agent setups, agents need to call each other's APIs. With each agent in its own sandbox, there is no cluster-reachable endpoint.
• Database/cache endpoints. An agent runs a local SQLite HTTP interface or Redis instance that a companion service needs to query.

Current barriers

Five independent barriers prevent ingress into the sandbox:

1. No IP forwarding. net.ipv4.ip_forward is never set on the host side of the veth pair. The kernel drops forwarded packets.
2. No NAT rules. No PREROUTING, DNAT, or MASQUERADE rules exist. Even with forwarding enabled, there is no address translation to route packets from the pod IP to 10.200.0.2 inside the namespace.
3. No Kubernetes Service. The compute driver creates a Sandbox CRD but never creates a corresponding Service. The pod spec declares no containerPort entries. Standard Kubernetes service discovery does not work.
4. Kubernetes NetworkPolicy. The Helm chart only permits ingress from the gateway pod on port 2222 (SSH). All other in-cluster ingress is denied.
5. No API surface. No protobuf field, no policy schema, and no CLI flag exists for declaring exposed ports.

Why SSH port forwarding is insufficient

openshell forward tunnels traffic from the operator's local machine through the gateway's SSH connection. This solves local developer access but has fundamental limitations:

• Cluster-internal access. Other pods cannot use the SSH tunnel. A frontend pod that needs to call a sandbox API has no path.
• Load balancer / Ingress access. Cloud load balancers operate at the Kubernetes Service level. No Service means no external exposure.
• Latency and overhead. Every connection traverses SSH through the gateway.
• No service discovery. No DNS name or stable IP exists for a sandbox service.

Interaction with the split-pod proposal (#981)

Issue #981 proposes splitting the supervisor and agent into separate pods. In that architecture, the agent pod has its own CNI-assigned IP, so ingress becomes a standard Kubernetes Service pointing at the agent pod directly — no DNAT needed. However, the InPod (single-pod) architecture remains the default and will coexist with the split-pod model. Ingress support for the InPod mode requires solving the DNAT/forwarding problem described here.

The design should be structured so that the API surface (CLI flags, policy schema, proto fields) is shared across both modes, with only the enforcement mechanism differing (InPod uses DNAT; split-pod uses native pod networking).

Proposed Design

The design should address two questions: what ports to expose (the API surface) and how to forward traffic (the enforcement mechanism). This section focuses on problem constraints and design decisions rather than implementation details.

Where should ingress ports be declared?

There are three reasonable options:

Option A: In the sandbox spec (e.g., --publish 8080 on create). The sandbox creator decides what to expose. Simple, Docker-like UX. But it means the sandbox creator can expose arbitrary ports without policy approval.

Option B: In the policy (e.g., ingress.ports in the policy YAML). The policy author controls what can be exposed. Consistent with how egress is policy-controlled today. The sandbox creator cannot circumvent this.

Option C: Both, via the provider profile layer model. Discussion #865 (tracked in #896) introduces a 3-layer policy composition model where provider profiles auto-inject endpoints into sandbox policy. Ingress ports could follow the same pattern:

• Layer 1 (base policy): No ingress by default.
• Layer 2 (provider profiles): A provider profile declares the ports its agent needs to expose (e.g., an MCP tool server provider declares port 8080). These are auto-injected when the provider is attached.
• Layer 3 (user policy): The user can add additional ingress ports via openshell policy update --add-ingress 9090.

This aligns ingress with the direction the project is already heading for egress policy. The provider profile already declares endpoints, binaries, and deny rules — adding an ingress_ports field is a natural extension.

The decision between these options has implications for the proto schema (field on SandboxTemplate vs. SandboxPolicy), the CLI surface, and whether ingress can be modified at runtime (policy supports hot reload; the sandbox spec does not).

Kernel-level forwarding (InPod mode)

Regardless of where ports are declared, the InPod mode needs iptables DNAT rules in the pod root namespace to forward traffic across the veth pair. The key constraints:

• IP forwarding must be enabled. In Kubernetes, /proc/sys is mounted read-only. The supervisor needs a fallback that temporarily remounts /proc/sys rw (requires CAP_SYS_ADMIN, which the sandbox already has), writes the sysctl, and remounts back to ro. The remount is scoped to the pod's mount namespace, and net.ipv4.ip_forward is scoped to the pod's network namespace — neither affects the node or other pods.
• Per-port DNAT rules in the pod root namespace (PREROUTING, FORWARD, POSTROUTING). These must cover both external inbound traffic (PREROUTING) and locally-originated traffic from other containers in the pod (OUTPUT).
• No new capabilities required. CAP_SYS_ADMIN and CAP_NET_ADMIN are already present.
• Failure must be fatal. If the user declared a port, it must be reachable. Unlike bypass detection (where LOG rule failure is non-fatal), DNAT failure should prevent sandbox startup.

Kubernetes integration

• containerPort declarations on the pod spec so the CRD controller includes them in the per-sandbox Kubernetes Service.
• NetworkPolicy management. The default Helm-deployed NetworkPolicy only allows ingress on port 2222. Published ports need either (a) operator-managed NetworkPolicy updates, or (b) auto-generated per-sandbox NetworkPolicy with configurable source selectors (namespace/pod labels controlling who can reach the ports).
• Service discovery. Published ports should be reachable via the sandbox's Kubernetes Service DNS name (sandbox-name.namespace.svc.cluster.local:port).

Interaction with existing sandbox features

• Bypass detection operates in the sandbox inner namespace's OUTPUT chain. Ingress DNAT operates in the pod root namespace's PREROUTING/FORWARD/POSTROUTING chains. They are independent.
• HTTP CONNECT proxy is not in the ingress path. Inbound traffic is forwarded at the kernel level; the proxy only handles outbound egress.
• Egress isolation is not weakened. Exposing a port for inbound traffic does not create an egress path. Outbound connections still hit the OUTPUT chain's bypass detection and must go through the proxy.

Alternatives Considered

1. HostPort mapping

Using Kubernetes hostPort to expose the sandbox port on the node. Rejected: requires knowing the node IP (breaks service discovery), causes port conflicts across sandboxes on the same node, does not integrate with Kubernetes Services.

2. Sidecar reverse proxy

Running envoy/socat as a sidecar that forwards into the sandbox namespace. Rejected: adds container overhead and image complexity, the sidecar runs outside sandbox policy enforcement, and the DNAT approach is simpler using existing veth infrastructure.

3. Modifying inner namespace routing for inbound

Adjusting the sandbox inner namespace to accept connections natively instead of DNAT in the root namespace. Rejected: the inner namespace intentionally has no default route back to the cluster (all egress goes through proxy), adding inbound routing would weaken the isolation model.

4. Defer entirely to the split-pod model (#981)

Wait for the split-pod architecture where ingress is trivial (agent pod has its own CNI IP). Rejected: the InPod mode is the default and will remain so for Docker Desktop, local development, K3s clusters, and environments without gVisor. These deployments still need ingress support.

Agent Investigation

Explored the codebase to assess feasibility:

• Network namespace (crates/openshell-sandbox/src/sandbox/linux/netns.rs): Already has the veth pair, IP assignment, bypass detection iptables rules, and find_iptables() helper. Ingress DNAT follows the same pattern as install_bypass_detection_rules().
• K8s driver (crates/openshell-driver-kubernetes/src/driver.rs): sandbox_template_to_k8s() already has env var injection and platform_config_struct() for extracting typed config.
• Proto (proto/openshell.proto): SandboxTemplate uses google.protobuf.Struct for resources and volume_claim_templates. Ingress could use the same pattern, or typed proto messages if the design goes with policy-level declaration.
• Gateway (crates/openshell-server/src/compute/mod.rs): build_platform_config() already assembles a Struct from template fields.
• Supervisor startup (crates/openshell-sandbox/src/lib.rs): Port publication must happen after netns creation but before seccomp hardening. The insertion point is between the existing netns block and the seccomp prelude.
• Policy engine (crates/openshell-policy/): The policy language is entirely egress-oriented today. Adding an ingress section is structurally straightforward but would be the first inbound policy primitive.
• Provider profiles (#865 / Enhanced Provider Management #896): The 3-layer policy composition model is designed for extensibility. An ingress_ports field on provider profiles would let providers declare required inbound ports alongside their egress endpoints.

Open questions for the maintainers

1. Spec vs. policy vs. provider layer? Should ingress ports live on SandboxTemplate (creation-time, immutable), in the policy (hot-reloadable, policy-author-controlled), or auto-injected via provider profiles (Provider Enhancements -- Declarative Profiles, Auto-Injected Policy, Multi-Provider Inference #865)? This is the most consequential design decision.
2. NetworkPolicy management. Should the platform auto-generate per-sandbox NetworkPolicy with source selectors, or leave this to the operator? Auto-generation is more complete but adds Kubernetes resource lifecycle management.
3. Scope for first pass. TCP only? ClusterIP only? Creation-time only (no hot reload)? Narrowing the first pass reduces risk.
4. Port validation. Should control-plane ports (2222, 2379, 6443, 10250) be rejected at validation time?