Skip to content

feat(policy): support Cedar in development builds - #5006

Draft
drew wants to merge 55 commits into
drew/dev-buildsfrom
drew/cedar/dev-builds
Draft

drew wants to merge 55 commits into
drew/dev-buildsfrom
drew/cedar/dev-builds

Conversation

@drew

@drew drew commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Support Cedar-authored sandbox policies in development builds. The gateway and supervisor enable Cedar through the single experimental feature; crate boundaries forward that bundle without separate cedar switches. Default, prerelease, and stable product builds exclude Cedar.

Cedar parsing and evaluation stay in dedicated engine/schema crates. A Cedar message adapter owns analysis shared by gateway and runtime composition, and a separate runtime adapter implements neutral policy contracts. Supervisor composition selects the backend; networking enforces baseline destination restrictions before consulting opaque policy authorizers.

Related Issue

Stacked on #5005. Closes #4411. Builds on #4565. Cedar example policies and demos are reviewed separately in #4991.

Behavior

  • Presence of CedarPolicy at protobuf tag 7 selects Cedar, including empty source. Existing YAML fields retain their meanings.
  • Cedar policies support gateway validation, filesystem grants, network/L7 enforcement, provider credentials, and staged configuration reloads. Load-time warning analysis stays in policy crates without adding warning fields to public responses.
  • The CLI uses the existing --policy option for Cedar source and adds no middleware flag. Middleware, endpoint settings, and custom process identity are not configurable through the CLI for Cedar policies; existing YAML middleware is unchanged.
  • The CLI, Rust SDK, and TypeScript SDK send Cedar policies directly and display gateway rejection errors without a Cedar capability preflight. Core policy sync follows the same path; the health API remains unchanged. The CLI, TUI, and TypeScript SDK do not advertise runtime enforcement capabilities. Supervisor capability negotiation remains in place. Gateways predating the Cedar protobuf field may ignore it; Cedar requires compatible development gateway and supervisor builds.
  • Endpoint settings and provider configuration retain their existing API shapes. Providers supply credentials without granting Cedar access.

Validation

  • 2,595 policy, networking, and CLI tests passed for the CLI cleanup; four tests remain ignored. After removing CLI capability checks, all 559 CLI tests and the gateway rejection test passed again.
  • Without Cedar: default product dependency graphs and gateway/supervisor compile checks passed, along with YAML policy and gateway/supervisor rejection tests.
  • Feature-bundle cleanup passed 207 Cedar-focused tests with experimental, 283 default policy tests, and 18 default gateway/supervisor Cedar rejection tests. Gateway/supervisor compile and dependency-graph checks passed with the bundle enabled and disabled.
  • The core boundary cleanup passed 612 core tests, 15 server Cedar compatibility tests, and 3 CLI Cedar regression tests.
  • Server scope cleanup passed 2,475 server tests plus the updated schema inventory check; 33 tests remain ignored. Strict Clippy passed across server, SDK, and CLI targets.
  • Strict Clippy across affected crates passed.
  • All 37 Cedar Docker E2E checks passed with the rebuilt CLI and existing Cedar gateway/supervisor/sandbox artifacts. The local debug supervisor image used the distroless C++ base for libgcc_s; production image defaults are unchanged.
  • Rust SDK and TUI tests passed, and strict Clippy passed for both crates. The Rust SDK and its tests have no remaining diff.
  • TypeScript SDK CI passed locally, including 197 tests.
  • Rust formatting, Markdown lint, shell syntax, strict protobuf compatibility, and whitespace checks passed.

Release Notes

Development builds can create and enforce sandboxes with Cedar policies. YAML policies keep their existing behavior.

vishwa-raman and others added 30 commits October 16, 2026 10:53
A sandbox policy can be written in Cedar instead of YAML. The format is
chosen once per sandbox: YAML policies keep using the Rego engine
unchanged, and a .cedar policy makes Cedar the sole authority for network
connections, per-request L7 checks, and Landlock filesystem grants.

Policy model and validation:
- New crates openshell-policy-cedar-schema (canonical Cedar schema) and
  openshell-policy-cedar (CedarEngine). Policies are validated in strict
  mode, and shapes that cannot be enforced exactly as written are
  rejected at submission in the CLI, the gateway, and the supervisor.
- Load-time analysis derives Landlock grants, the endpoints and paths
  inspected per request (@protocol rest, json-rpc, mcp, graphql,
  websocket, or websocket-graphql, scoped by @path), native TCP endpoints
  (@transport("tcp")), and policy DNS eligibility, including delimited
  host globs.
- Delimited like gives YAML-style segment globs for hosts, paths, and
  binaries. It comes from ai-openshell/cedar, pinned by commit, with a
  matching deny.toml allow-git exception; its Lean specification is
  ai-openshell/cedar-spec.
- The schema declares only what the supervisor populates: NetworkConnect
  carries no request fields, Process has no workdir, and a has test on an
  undeclared attribute is rejected. A filesystem permit must list ReadFile
  with WriteFile, and resource.host_port names an endpoint for policy DNS.
- @enforcement("audit") provides audit endpoints and staged audit
  policies; decisions name policies by their @id annotation.
- Request policies match query parameters through context.query tags;
  repeated keys are evaluated per combination of values, as YAML matches
  every value.
- context.destination_ip restricts the resolved addresses a connection
  may reach, the Cedar form of allowed_ips. Permits read it only as fixed
  isInRange lists; connections are evaluated before DNS and again per
  resolved address, private addresses need an address-constrained permit
  or an exact host, and always-blocked destinations stay blocked.
- context.binary_aliases matches symlinked binaries, resolved inside the
  container as YAML does, and context.binary_globs holds the glob patterns
  that match the calling binary or an ancestor, so YAML binary globs
  translate exactly.
- @enforcement("staged") marks a policy that is only compared against the
  enforced decision; @enforcement("audit") has YAML's audit-endpoint
  semantics. An HttpRequest policy may name its host with a delimited glob,
  inspecting every matching host as a YAML wildcard endpoint does. A
  GraphQL path is authoritative for the requests it parses. A delimited
  like whose ** touches other characters, and a GraphQL-over-WebSocket
  message permit without a GraphQL condition, are rejected at load.

Integration:
- SandboxPolicy gains cedar_policy_source, the gateway-managed
  provider_credential_rules, and endpoint_settings. Attached providers
  supply credentials and token grants on requests Cedar allows but never
  grant access; their allowed_ips only narrow addresses. Provider rules
  are compiled into Cedar policies tagged with their token-grant owner,
  so owner selection is decided by Cedar.
- The CLI accepts .cedar policies and a separate --middleware file that
  also carries endpoint settings (TLS skip, body limits, MCP versions,
  persisted queries) and the process identity; policy get renders the
  Cedar text, middleware, endpoint settings, process identity, and
  provider rules.
- The gateway rejects format switches, Cedar global policies, and removal
  of Cedar-derived Landlock grants on live sandboxes.
- Cedar sandboxes run without an OPA engine or Rego. The supervisor
  builds the Cedar engine at startup admission, which owns middleware
  state and its own generation, commits Cedar, credentials, and
  middleware together on reload, and applies the policy validation
  failure mode to Cedar. Every network decision point, including the
  Windows host proxy, routes through the PolicyEngine enum; Cedar tunnels
  pin Cedar's generation. Middleware selection is shared Rust, the data
  both engines hand to the proxy is typed Rust with the Rego output
  converted at one boundary, and engine-neutral types move out of opa.rs.
- The supervisor maps JSON-RPC, MCP, GraphQL, and WebSocket requests into
  Cedar requests the way the YAML engine sees them: uppercase methods,
  calls and response frames only in a POST, and one evaluation per call,
  GraphQL operation, or WebSocket message.
- Cedar decisions report the policy DNS records that cover a connection,
  so transparent TCP correlation works as for YAML.
- Cedar support is the default cedar feature of openshell-policy; crates
  that never handle Cedar policies disable it. The Go SDK carries the new
  fields.
- The RPM vendor archive captures the git source replacement tables cargo
  vendor prints, so the offline RPM build resolves the Cedar dependency.

Testing and documentation:
- Unit tests for the Cedar crates and supervisor integration, and an
  e2e:cedar suite run by mise run e2e and the Docker E2E matrix.
- A parity harness runs paired YAML and Cedar policies through both
  engines. It ports 258 of the 260 existing YAML decision tests; the
  remaining one that is valid YAML needs a Cedar language change. On
  Linux, 2,904 of 2,909 cases agree; the 5 differences are deliberate and
  recorded with their reasons, an evaluation error on either side fails a
  case, and load rejections are asserted. Token-grant owners match YAML for every shipped provider
  profile.
- Every example and e2e directory with a YAML policy has a Cedar version
  beside it, and its scripts accept POLICY_FORMAT=cedar; a parity test
  runs each pair through both engines.
- User documentation for Cedar policies, and a crate README.

Signed-off-by: Vishwanath Raman <vishwanathr@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Keep WebSocket assembly admission shared across tunnels and policy reloads. Derive configured MCP descriptors once for routing and passive endpoint reporting, preserving wildcard selectors, provider ports, and scoped paths.

Correct wildcard L7 parity coverage and assert allow/deny behavior on a second host through the relay.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Bring the branch up to date with main (9a85667), which hosts sandboxes
on a shared supervisor and isolates networking state per sandbox
instance.

Conflict resolution:
- openshell-supervisor: the policy poll loop keys on the per-instance
  `gateway` from main and the `PolicyEngine` from this branch.
- openshell-supervisor-network: `run_networking` keeps main's
  per-instance token-grant resolver and denial log; the OPA import is
  gone because the engine is reached through `PolicyEngine`.
- openshell-server: keep main's rewritten authorization test together
  with the assertion that `Health` stays callable by ordinary users for
  Cedar capability discovery.
- Cargo.lock: main's rustls-webpki and webpki-roots versions are kept;
  Cedar's regex requirement is retained.

Adapt tests that main added since the merge base: the passthrough relay
test selects middleware through `PolicyEngine::passthrough_middleware`,
the Cedar mixed-version test builds a `GatewayEndpoint`, and the testkit
sandbox configuration fetch advertises Cedar support.

Refs #4474

Signed-off-by: Vishwanath Raman <vishwanathr@nvidia.com>
Bring the branch up to date with main (c4688aa). The Windows test
repairs on main (92434dc) cover the same tests this branch had
stabilized, so the conflicts resolve to main's versions: the mutation
guard stress test yields until its deadline instead of counting yields,
and the startup write-refusal scenarios gate baseline enrichment on Unix
with main's wording. The `PolicySource` import that resolution needs is
restored.

Refs #4474

Signed-off-by: Vishwanath Raman <vishwanathr@nvidia.com>
Cedar sandbox policies now ship only in builds that enable the gateway and
supervisor `cedar` Cargo feature. Release Dev and branch E2E enable it;
prerelease and stable builds exclude the Cedar engine.

- openshell-policy has no default features. Cedar validation, load
  warnings, filesystem-grant derivation, and provider-rule conversion live
  in its gated `cedar` module; clients carry Cedar documents as text.
- openshell-supervisor-network gates its Cedar engine behind `cedar`; builds
  without it use an uninhabited stub that rejects Cedar policies.
- The gateway and supervisor derive supports_cedar_policy from the compiled
  feature. A gateway without Cedar rejects Cedar on create, update, start,
  and stored-policy reads, and skips such sandboxes in store-wide checks.
- The gateway returns policy load warnings (PolicyWarning) from
  CreateSandbox and sandbox UpdateConfig, including on replay; the CLI
  prints them and no longer links Cedar.
- Compute drivers report driver_enforces_policy, and the gateway rejects
  Cedar for drivers that enforce policy themselves, such as MXC.
- CI lints and tests both configurations, verifies stable dependency graphs
  exclude Cedar, and release builds fail if a binary embeds a Cedar crate.
- The Cedar docs page is published only on the dev docs site.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Address the review findings on the Cedar policy support.

Engine robustness:
- Bound policy text before parsing (size, bracket nesting, expression
  depth) with a non-recursive scan, and run loading and evaluation on a
  stack sized for those bounds, so no policy within the gateway limit or
  request the proxy delivers can exhaust a worker stack.
- Deny requests whose inspected strings exceed 16 KiB, bounding the
  delimited `like` recursion.
- Decide GraphQL paths by ignoring excluded permits in one evaluation
  instead of building per-path policy set copies.

Analysis:
- Bucket and budget host-overlap checks.
- Warn on audit paths beside enforced policies of overlapping entries.
- Keep staged policies from creating inspection, paths, or protocols,
  and warn when they would never be evaluated.
- Reject forbids that read `destination_ip` without a guard, and
  policies that read `principal.user` or `principal.group`.

Runtime and control plane:
- Close open Cedar tunnels when authored middleware configs change.
- Reject `tls: skip` endpoint settings on endpoints Cedar inspects, and
  keep inspecting them when a provider rule sets `tls: skip`.
- Report Cedar support in `SupervisorHello` and deliver a not-admitted
  configuration instead of a Cedar policy to supervisors without it.
- Reject incremental policy updates on Cedar sandboxes in the gateway
  and CLI.
- Re-resolve binary aliases when a policy staged before startup commits.
- Keep OPA result conversion outside the engine lock and project only
  consumed fields; document the policy-DNS identity contract.
- Test the provider-rule JSON contract end to end.

Docs: Cedar limits, `tls: skip`, staged and audit semantics, tested
drivers, and the Cedar dependency pin.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Adapt main's suspend and resume support to the Cedar changes: resume
applies the start path's Cedar support and driver checks, new
SandboxResponse constructions carry policy_warnings, tests match the
boxed sandbox subcommand, and the reviewed schema fingerprints cover
both sides.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Engine:
- Reject Cedar policies that test which process the principal is.
  Every request is evaluated as one placeholder process, so such a test
  never matches, and a forbid written that way never applied.
- Deny hosts with an empty DNS label before evaluating policies. Host
  globs matched them, but inspection did not select them, so glob
  HttpRequest forbids were skipped.
- Bound the work of one request's query-value combinations by the size
  of the evaluated policy set (16,384 policy evaluations, at most 1,024
  combinations).
- Summarize CedarEngine in Debug instead of walking every policy's
  expression tree outside the grown stack.

Build and CI:
- Reject a second stacker version in cargo-deny; Cedar's stack checks
  share its thread-local.
- Require the Cedar crates in Cedar-enabled gateway and supervisor
  binaries, a positive control for the stable forbidden-crates check.

CLI: print Cedar policy text without YAML highlighting.

Docs: `tls: skip` for clients that pin certificates or present client
certificates, and the new principal, host, and query limits.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
Keep the per-policy evaluation budget, which bounds the work of one
request for large policy sets, but restore 4,096 combinations as the
ceiling RFC 0020 specifies.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
- Compile correlated_endpoint only where it is called: on Linux, and in
  Cedar-enabled tests.
- Expect needless_pass_by_ref_mut on the non-Unix SuspendSignal::recv,
  which keeps the Unix receiver's signature.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
- Load a Cedar policy once per supervisor startup and reload:
  validation hands its engine to the Cedar network engine, and startup
  reads filesystem grants from that engine.
- Convert Cedar inspection protocols to proxy protocols with an
  exhaustive match instead of a string label.
- Share path specificity with the relay, endpoint port selection, the
  endpoint selector match, and the `tls: skip` overlap check between the
  gateway and supervisor.
- Document why Cedar's exact host overlap and core's conservative
  overlap both exist, and test that core's covers Cedar's.
- Walk Cedar expressions with one visitor, and split policy analysis
  into per-action collectors.
- Split cedar_only.rs into engine, routes, L7, destination, and test
  modules.
- Name the Cedar policy check (`SandboxPolicy::is_cedar`) and the
  gateway and supervisor Cedar support checks.
- Group argument lists in Cedar evaluation and provider-rule
  compilation into structs.
- Import shared types from policy_types and policy_engine; keep the opa
  re-exports as transitional.
- Document the policy engines in the supervisor-network README.

Signed-off-by: Johnny Greco <jogreco@nvidia.com>
drew added 4 commits October 27, 2026 21:42
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 28, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@drew
drew added this pull request to stack #5007 October 28, 2026 15:43
@github-actions

Copy link
Copy Markdown

drew added 3 commits October 28, 2026 16:09
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
drew added 2 commits November 16, 2026 08:00
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
drew added 12 commits November 18, 2026 14:15
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
Signed-off-by: Drew Newberry <anewberry@nvidia.com>

@vishwa-raman vishwa-raman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drew, I went through #5006 against vishwa/cedar-openshell and the goals we set for bringing Cedar in. The direction looks right to me: the runtime interface, the separate adapter and runtime crates, and composing the backend in the supervisor are cleaner than what we had. One issue blocks the merge and a few others need answers.

What I verified

  • Parity is intact. All 147 parity tests moved to tests/policy-conformance under the same names. On macOS they pass on both branches (149 passed, the same 3 Linux-only symlink tests ignored). expected.json records the same 7 YAML/Cedar differences the old harness documented, with no new ones. I like that each backend is now checked against its own expected outcome.
  • The stable dependency graph is clean. cargo tree -e normal -i cedar-policy finds nothing for openshell-gateway, -supervisor, -cli, -sandbox or -tui with default features. With --features experimental, the gateway and supervisor resolve the ai-openshell/cedar fork.
  • Supervisor negotiation is right. POLICY_CEDAR_V1 is only advertised under cfg!(feature = "experimental").

Blocker

The stable RPM build fails offline resolution. 3a4a04506 deletes deploy/rpm/create-vendor-archive.sh and returns .packit.yaml and openshell.spec to main's versions. But Cargo.lock lists the ai-openshell/cedar git source in every build, with or without experimental. %cargo_prep only redirects crates.io to the vendored copy, so offline resolution fails on the git source even when nothing compiles Cedar:

cargo vendor --locked vendor > /dev/null
CARGO_HOME=$(mktemp -d) cargo tree --offline --locked -e normal -p openshell-gateway \
  --config 'source.crates-io.replace-with="vendored-sources"' \
  --config 'source.vendored-sources.directory="vendor"'
# error: can't checkout from 'https://github.com/ai-openshell/cedar.git': you are in the offline mode

(If mise shims cargo, use the toolchain's cargo binary directly. Overriding CARGO_HOME breaks the shim.)

openshell-cli and openshell-supervisor fail the same way. With the [source."git+…"] mapping the old script wrote, resolution succeeds. The vendoring script, or something equivalent, needs to stay as long as the lockfile carries a git source.

Needs an answer before merge

  1. driver_enforces_policy removal. The field and its admission check are gone from compute_driver.proto and all six drivers. Does MXC on the windows branch rely on it to keep out policies it can't enforce? If so, the removal needs a replacement there.
  2. Older gateways silently dropping Cedar. Without the health flag and the client-side check, a gateway that predates CedarPolicy drops tag 7 and accepts the request as an empty YAML policy (which passes validation on main). The docs cover it, but I couldn't establish what an empty policy enforces on the filesystem. Could we confirm it fails closed, ideally with an e2e against a current stable gateway?
  3. Policy warnings. policy_warnings and PolicyWarning are removed entirely, so load-time analysis (for example, audit-only rules next to enforced rules on the same endpoint) is computed but never shown to the author. Was dropping them intended, or should a small additive field come back later?

Follow-ups (fine after merge)

  • A CI check that stable builds exclude Cedar. The boundary is clean today, but with verify-cedar-off.sh gone nothing would catch a regression. A cargo tree -i cedar-policy check that must fail, following the webpki/telemetry pattern in tasks/rust.toml, would cover it.
  • Configuring Cedar sandboxes through the CLI. With --middleware gone, the CLI can't set middleware, endpoint_settings or process identity on a Cedar policy. That matches the plan, but it should have a tracking issue.
  • Rollout paragraph in the RFC (#4564): Cedar is opt-in and dev-only, with criteria for promoting it.
  • Commit messages and review size. The 21 refactor commits have no message bodies, and the API changes (cedar_policy as a message, capability strings, the warning and driver-field removals) are mixed into a 222-file diff. Short bodies on 3a4a04506, 73419b00c and 7e66b3491 would help other reviewers.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(policy): Cedar as a policy language in OpenShell

3 participants