Repository navigation
Conversation
A sandbox policy can be written in Cedar instead of YAML. The format is chosen once per sandbox: YAML policies keep using the Rego engine unchanged, and a .cedar policy makes Cedar the sole authority for network connections, per-request L7 checks, and Landlock filesystem grants. Policy model and validation: - New crates openshell-policy-cedar-schema (canonical Cedar schema) and openshell-policy-cedar (CedarEngine). Policies are validated in strict mode, and shapes that cannot be enforced exactly as written are rejected at submission in the CLI, the gateway, and the supervisor. - Load-time analysis derives Landlock grants, the endpoints and paths inspected per request (@protocol rest, json-rpc, mcp, graphql, websocket, or websocket-graphql, scoped by @path), native TCP endpoints (@transport("tcp")), and policy DNS eligibility, including delimited host globs. - Delimited like gives YAML-style segment globs for hosts, paths, and binaries. It comes from ai-openshell/cedar, pinned by commit, with a matching deny.toml allow-git exception; its Lean specification is ai-openshell/cedar-spec. - The schema declares only what the supervisor populates: NetworkConnect carries no request fields, Process has no workdir, and a has test on an undeclared attribute is rejected. A filesystem permit must list ReadFile with WriteFile, and resource.host_port names an endpoint for policy DNS. - @enforcement("audit") provides audit endpoints and staged audit policies; decisions name policies by their @id annotation. - Request policies match query parameters through context.query tags; repeated keys are evaluated per combination of values, as YAML matches every value. - context.destination_ip restricts the resolved addresses a connection may reach, the Cedar form of allowed_ips. Permits read it only as fixed isInRange lists; connections are evaluated before DNS and again per resolved address, private addresses need an address-constrained permit or an exact host, and always-blocked destinations stay blocked. - context.binary_aliases matches symlinked binaries, resolved inside the container as YAML does, and context.binary_globs holds the glob patterns that match the calling binary or an ancestor, so YAML binary globs translate exactly. - @enforcement("staged") marks a policy that is only compared against the enforced decision; @enforcement("audit") has YAML's audit-endpoint semantics. An HttpRequest policy may name its host with a delimited glob, inspecting every matching host as a YAML wildcard endpoint does. A GraphQL path is authoritative for the requests it parses. A delimited like whose ** touches other characters, and a GraphQL-over-WebSocket message permit without a GraphQL condition, are rejected at load. Integration: - SandboxPolicy gains cedar_policy_source, the gateway-managed provider_credential_rules, and endpoint_settings. Attached providers supply credentials and token grants on requests Cedar allows but never grant access; their allowed_ips only narrow addresses. Provider rules are compiled into Cedar policies tagged with their token-grant owner, so owner selection is decided by Cedar. - The CLI accepts .cedar policies and a separate --middleware file that also carries endpoint settings (TLS skip, body limits, MCP versions, persisted queries) and the process identity; policy get renders the Cedar text, middleware, endpoint settings, process identity, and provider rules. - The gateway rejects format switches, Cedar global policies, and removal of Cedar-derived Landlock grants on live sandboxes. - Cedar sandboxes run without an OPA engine or Rego. The supervisor builds the Cedar engine at startup admission, which owns middleware state and its own generation, commits Cedar, credentials, and middleware together on reload, and applies the policy validation failure mode to Cedar. Every network decision point, including the Windows host proxy, routes through the PolicyEngine enum; Cedar tunnels pin Cedar's generation. Middleware selection is shared Rust, the data both engines hand to the proxy is typed Rust with the Rego output converted at one boundary, and engine-neutral types move out of opa.rs. - The supervisor maps JSON-RPC, MCP, GraphQL, and WebSocket requests into Cedar requests the way the YAML engine sees them: uppercase methods, calls and response frames only in a POST, and one evaluation per call, GraphQL operation, or WebSocket message. - Cedar decisions report the policy DNS records that cover a connection, so transparent TCP correlation works as for YAML. - Cedar support is the default cedar feature of openshell-policy; crates that never handle Cedar policies disable it. The Go SDK carries the new fields. - The RPM vendor archive captures the git source replacement tables cargo vendor prints, so the offline RPM build resolves the Cedar dependency. Testing and documentation: - Unit tests for the Cedar crates and supervisor integration, and an e2e:cedar suite run by mise run e2e and the Docker E2E matrix. - A parity harness runs paired YAML and Cedar policies through both engines. It ports 258 of the 260 existing YAML decision tests; the remaining one that is valid YAML needs a Cedar language change. On Linux, 2,904 of 2,909 cases agree; the 5 differences are deliberate and recorded with their reasons, an evaluation error on either side fails a case, and load rejections are asserted. Token-grant owners match YAML for every shipped provider profile. - Every example and e2e directory with a YAML policy has a Cedar version beside it, and its scripts accept POLICY_FORMAT=cedar; a parity test runs each pair through both engines. - User documentation for Cedar policies, and a crate README. Signed-off-by: Vishwanath Raman <vishwanathr@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Keep WebSocket assembly admission shared across tunnels and policy reloads. Derive configured MCP descriptors once for routing and passive endpoint reporting, preserving wildcard selectors, provider ports, and scoped paths. Correct wildcard L7 parity coverage and assert allow/deny behavior on a second host through the relay. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Bring the branch up to date with main (9a85667), which hosts sandboxes on a shared supervisor and isolates networking state per sandbox instance. Conflict resolution: - openshell-supervisor: the policy poll loop keys on the per-instance `gateway` from main and the `PolicyEngine` from this branch. - openshell-supervisor-network: `run_networking` keeps main's per-instance token-grant resolver and denial log; the OPA import is gone because the engine is reached through `PolicyEngine`. - openshell-server: keep main's rewritten authorization test together with the assertion that `Health` stays callable by ordinary users for Cedar capability discovery. - Cargo.lock: main's rustls-webpki and webpki-roots versions are kept; Cedar's regex requirement is retained. Adapt tests that main added since the merge base: the passthrough relay test selects middleware through `PolicyEngine::passthrough_middleware`, the Cedar mixed-version test builds a `GatewayEndpoint`, and the testkit sandbox configuration fetch advertises Cedar support. Refs #4474 Signed-off-by: Vishwanath Raman <vishwanathr@nvidia.com>
Bring the branch up to date with main (c4688aa). The Windows test repairs on main (92434dc) cover the same tests this branch had stabilized, so the conflicts resolve to main's versions: the mutation guard stress test yields until its deadline instead of counting yields, and the startup write-refusal scenarios gate baseline enrichment on Unix with main's wording. The `PolicySource` import that resolution needs is restored. Refs #4474 Signed-off-by: Vishwanath Raman <vishwanathr@nvidia.com>
Cedar sandbox policies now ship only in builds that enable the gateway and supervisor `cedar` Cargo feature. Release Dev and branch E2E enable it; prerelease and stable builds exclude the Cedar engine. - openshell-policy has no default features. Cedar validation, load warnings, filesystem-grant derivation, and provider-rule conversion live in its gated `cedar` module; clients carry Cedar documents as text. - openshell-supervisor-network gates its Cedar engine behind `cedar`; builds without it use an uninhabited stub that rejects Cedar policies. - The gateway and supervisor derive supports_cedar_policy from the compiled feature. A gateway without Cedar rejects Cedar on create, update, start, and stored-policy reads, and skips such sandboxes in store-wide checks. - The gateway returns policy load warnings (PolicyWarning) from CreateSandbox and sandbox UpdateConfig, including on replay; the CLI prints them and no longer links Cedar. - Compute drivers report driver_enforces_policy, and the gateway rejects Cedar for drivers that enforce policy themselves, such as MXC. - CI lints and tests both configurations, verifies stable dependency graphs exclude Cedar, and release builds fail if a binary embeds a Cedar crate. - The Cedar docs page is published only on the dev docs site. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Address the review findings on the Cedar policy support. Engine robustness: - Bound policy text before parsing (size, bracket nesting, expression depth) with a non-recursive scan, and run loading and evaluation on a stack sized for those bounds, so no policy within the gateway limit or request the proxy delivers can exhaust a worker stack. - Deny requests whose inspected strings exceed 16 KiB, bounding the delimited `like` recursion. - Decide GraphQL paths by ignoring excluded permits in one evaluation instead of building per-path policy set copies. Analysis: - Bucket and budget host-overlap checks. - Warn on audit paths beside enforced policies of overlapping entries. - Keep staged policies from creating inspection, paths, or protocols, and warn when they would never be evaluated. - Reject forbids that read `destination_ip` without a guard, and policies that read `principal.user` or `principal.group`. Runtime and control plane: - Close open Cedar tunnels when authored middleware configs change. - Reject `tls: skip` endpoint settings on endpoints Cedar inspects, and keep inspecting them when a provider rule sets `tls: skip`. - Report Cedar support in `SupervisorHello` and deliver a not-admitted configuration instead of a Cedar policy to supervisors without it. - Reject incremental policy updates on Cedar sandboxes in the gateway and CLI. - Re-resolve binary aliases when a policy staged before startup commits. - Keep OPA result conversion outside the engine lock and project only consumed fields; document the policy-DNS identity contract. - Test the provider-rule JSON contract end to end. Docs: Cedar limits, `tls: skip`, staged and audit semantics, tested drivers, and the Cedar dependency pin. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Adapt main's suspend and resume support to the Cedar changes: resume applies the start path's Cedar support and driver checks, new SandboxResponse constructions carry policy_warnings, tests match the boxed sandbox subcommand, and the reviewed schema fingerprints cover both sides. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Engine: - Reject Cedar policies that test which process the principal is. Every request is evaluated as one placeholder process, so such a test never matches, and a forbid written that way never applied. - Deny hosts with an empty DNS label before evaluating policies. Host globs matched them, but inspection did not select them, so glob HttpRequest forbids were skipped. - Bound the work of one request's query-value combinations by the size of the evaluated policy set (16,384 policy evaluations, at most 1,024 combinations). - Summarize CedarEngine in Debug instead of walking every policy's expression tree outside the grown stack. Build and CI: - Reject a second stacker version in cargo-deny; Cedar's stack checks share its thread-local. - Require the Cedar crates in Cedar-enabled gateway and supervisor binaries, a positive control for the stable forbidden-crates check. CLI: print Cedar policy text without YAML highlighting. Docs: `tls: skip` for clients that pin certificates or present client certificates, and the new principal, host, and query limits. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Keep the per-policy evaluation budget, which bounds the work of one request for large policy sets, but restore 4,096 combinations as the ceiling RFC 0020 specifies. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
- Compile correlated_endpoint only where it is called: on Linux, and in Cedar-enabled tests. - Expect needless_pass_by_ref_mut on the non-Unix SuspendSignal::recv, which keeps the Unix receiver's signature. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
- Load a Cedar policy once per supervisor startup and reload: validation hands its engine to the Cedar network engine, and startup reads filesystem grants from that engine. - Convert Cedar inspection protocols to proxy protocols with an exhaustive match instead of a string label. - Share path specificity with the relay, endpoint port selection, the endpoint selector match, and the `tls: skip` overlap check between the gateway and supervisor. - Document why Cedar's exact host overlap and core's conservative overlap both exist, and test that core's covers Cedar's. - Walk Cedar expressions with one visitor, and split policy analysis into per-action collectors. - Split cedar_only.rs into engine, routes, L7, destination, and test modules. - Name the Cedar policy check (`SandboxPolicy::is_cedar`) and the gateway and supervisor Cedar support checks. - Group argument lists in Cedar evaluation and provider-rule compilation into structs. - Import shared types from policy_types and policy_engine; keep the opa re-exports as transitional. - Document the policy engines in the supervisor-network README. Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
|
🌿 Preview your docs: https://nvidia-preview-pr-5006.docs.buildwithfern.com/openshell |
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
vishwa-raman
left a comment
There was a problem hiding this comment.
Drew, I went through #5006 against vishwa/cedar-openshell and the goals we set for bringing Cedar in. The direction looks right to me: the runtime interface, the separate adapter and runtime crates, and composing the backend in the supervisor are cleaner than what we had. One issue blocks the merge and a few others need answers.
What I verified
- Parity is intact. All 147 parity tests moved to
tests/policy-conformanceunder the same names. On macOS they pass on both branches (149 passed, the same 3 Linux-only symlink tests ignored).expected.jsonrecords the same 7 YAML/Cedar differences the old harness documented, with no new ones. I like that each backend is now checked against its own expected outcome. - The stable dependency graph is clean.
cargo tree -e normal -i cedar-policyfinds nothing foropenshell-gateway,-supervisor,-cli,-sandboxor-tuiwith default features. With--features experimental, the gateway and supervisor resolve theai-openshell/cedarfork. - Supervisor negotiation is right.
POLICY_CEDAR_V1is only advertised undercfg!(feature = "experimental").
Blocker
The stable RPM build fails offline resolution. 3a4a04506 deletes deploy/rpm/create-vendor-archive.sh and returns .packit.yaml and openshell.spec to main's versions. But Cargo.lock lists the ai-openshell/cedar git source in every build, with or without experimental. %cargo_prep only redirects crates.io to the vendored copy, so offline resolution fails on the git source even when nothing compiles Cedar:
cargo vendor --locked vendor > /dev/null
CARGO_HOME=$(mktemp -d) cargo tree --offline --locked -e normal -p openshell-gateway \
--config 'source.crates-io.replace-with="vendored-sources"' \
--config 'source.vendored-sources.directory="vendor"'
# error: can't checkout from 'https://github.com/ai-openshell/cedar.git': you are in the offline mode(If mise shims cargo, use the toolchain's cargo binary directly. Overriding CARGO_HOME breaks the shim.)
openshell-cli and openshell-supervisor fail the same way. With the [source."git+…"] mapping the old script wrote, resolution succeeds. The vendoring script, or something equivalent, needs to stay as long as the lockfile carries a git source.
Needs an answer before merge
driver_enforces_policyremoval. The field and its admission check are gone fromcompute_driver.protoand all six drivers. Does MXC on thewindowsbranch rely on it to keep out policies it can't enforce? If so, the removal needs a replacement there.- Older gateways silently dropping Cedar. Without the health flag and the client-side check, a gateway that predates
CedarPolicydrops tag 7 and accepts the request as an empty YAML policy (which passes validation on main). The docs cover it, but I couldn't establish what an empty policy enforces on the filesystem. Could we confirm it fails closed, ideally with an e2e against a current stable gateway? - Policy warnings.
policy_warningsandPolicyWarningare removed entirely, so load-time analysis (for example, audit-only rules next to enforced rules on the same endpoint) is computed but never shown to the author. Was dropping them intended, or should a small additive field come back later?
Follow-ups (fine after merge)
- A CI check that stable builds exclude Cedar. The boundary is clean today, but with
verify-cedar-off.shgone nothing would catch a regression. Acargo tree -i cedar-policycheck that must fail, following the webpki/telemetry pattern intasks/rust.toml, would cover it. - Configuring Cedar sandboxes through the CLI. With
--middlewaregone, the CLI can't set middleware,endpoint_settingsor process identity on a Cedar policy. That matches the plan, but it should have a tracking issue. - Rollout paragraph in the RFC (#4564): Cedar is opt-in and dev-only, with criteria for promoting it.
- Commit messages and review size. The 21 refactor commits have no message bodies, and the API changes (
cedar_policyas a message, capability strings, the warning and driver-field removals) are mixed into a 222-file diff. Short bodies on3a4a04506,73419b00cand7e66b3491would help other reviewers.
Summary
Support Cedar-authored sandbox policies in development builds. The gateway and supervisor enable Cedar through the single
experimentalfeature; crate boundaries forward that bundle without separatecedarswitches. Default, prerelease, and stable product builds exclude Cedar.Cedar parsing and evaluation stay in dedicated engine/schema crates. A Cedar message adapter owns analysis shared by gateway and runtime composition, and a separate runtime adapter implements neutral policy contracts. Supervisor composition selects the backend; networking enforces baseline destination restrictions before consulting opaque policy authorizers.
Related Issue
Stacked on #5005. Closes #4411. Builds on #4565. Cedar example policies and demos are reviewed separately in #4991.
Behavior
CedarPolicyat protobuf tag 7 selects Cedar, including empty source. Existing YAML fields retain their meanings.--policyoption for Cedar source and adds no middleware flag. Middleware, endpoint settings, and custom process identity are not configurable through the CLI for Cedar policies; existing YAML middleware is unchanged.Validation
experimental, 283 default policy tests, and 18 default gateway/supervisor Cedar rejection tests. Gateway/supervisor compile and dependency-graph checks passed with the bundle enabled and disabled.libgcc_s; production image defaults are unchanged.Release Notes
Development builds can create and enforce sandboxes with Cedar policies. YAML policies keep their existing behavior.