Repository navigation
feat: add a command to check a policy file before creating a sandbox #4035
Description
User Story
As someone writing a sandbox policy,
I want a command that tells me whether the file is valid,
so that I can fix mistakes before I try to create a sandbox with it.
Problem Statement
There's no command that checks a policy file on its own. openshell policy has set, update, get, list, and delete, and sandbox create has no dry run.
From reading the code, the CLI only parses the file, and the gateway does the remaining checks, such as rejecting a root run_as_user. So full validation needs a reachable gateway. sandbox create also connects before it reads the file, so with the gateway down, openshell sandbox create --policy bad.yaml fails with a connection error and never mentions the file.
Reproduction Steps
- Write
policy.yamlwith noversion:process: run_as_user: "1000" run_as_group: "1000"
openshell --gateway-endpoint http://127.0.0.1:1 sandbox create --policy policy.yaml, where nothing is listening on that port. It fails withfailed to connect to gatewayand doesn't mention the file.openshell-prover check --boundary policy.yaml policy.yamlcorrectly fails withmissing field: version.- Add
version: 1and change both IDs to"0". The same prover command printsresult: within_boundaryand exits 0.
Impact / Why This Matters
My first policy was missing version: 1. I couldn't tell whether the file was right without a running gateway and a sandbox create, which also pulls the image. And yes I should have read the documentation!
The workaround is openshell-prover check --boundary policy.yaml policy.yaml, with the same file as both arguments. The prover is meant for comparing two policies, not for validating one. On 0.1.2 it catches a missing version and unknown fields, but it accepts run_as_user: "0", which the schema reference says OpenShell rejects. It's also a separate binary that nothing points you to for this.
Proposed Design
A CLI command that takes a policy file, for example:
openshell policy check policy.yamlIt runs locally and doesn't need a gateway. It applies the same validation to the file as sandbox create --policy. A valid file exits 0. An invalid one exits non-zero and prints the same errors that creating a sandbox with it would. Alternative names for the subcommand: lint or validate. This is the same idea as terraform validate, helm lint, or opa check.
A pass means the file is valid, not that a sandbox will start. Checks that depend on the gateway or the image stay where they are, such as registered middleware, attached providers, or the image's USER. The command could also ask a running gateway to run its checks, the way kubectl has --dry-run=client and --dry-run=server. That could be done in follow-up work.
Acceptance Criteria
- The command works with no gateway configured or running.
- It rejects everything
sandbox create --policyrejects in the file itself, including a missingversion, unknown fields, and a rootrun_as_user. - When the file parses, it reports every validation problem in one run, not just the first.
- The exit code is 0 for a valid file and non-zero otherwise.
- The policy docs mention it where they describe writing a policy.
Alternatives Considered
- The prover, as above. It doesn't apply all the validation and isn't meant for this.
policy update --dry-run. It previews a merge into a live sandbox's policy. It doesn't take a file, and it needs a sandbox.- A gateway-side check only, such as
sandbox create --dry-run. The gateway can see more, such as registered middleware and attached providers. But it doesn't work before a gateway is up, or in CI or an editor, and it ties a file check to create's other flags.
Agent Investigation
Used the create-spike skill to check feasibility against main at 021400be8.
Feasible, medium complexity. Most of the file-only validation is already a pure function in openshell-policy, which the CLI already depends on. The work is a subcommand that skips gateway resolution, plus moving a few pure checks out of the gateway so both sides share them.
What validates a policy file today:
| Check | Where | Needs gateway state | Reports every problem |
|---|---|---|---|
File read, YAML limits, unknown fields, version |
openshell-policy-schema/src/lib.rs |
No | No, stops at the first |
| Endpoint TLS, enforcement, and access modes | to_proto in openshell-policy/src/lib.rs |
No | No, stops at the first |
| Semantic rules: identity, paths, endpoints, MCP, middleware structure | validate_and_canonicalize_sandbox_policy, openshell-policy/src/lib.rs:1873 |
No | Yes |
Reserved _provider_ rule names |
openshell-server/src/grpc/validation.rs:1018 |
No | No, stops at the first |
| Encoded size over 256 KiB | grpc/validation.rs:267, MAX_POLICY_SIZE in grpc/mod.rs:160 |
No | Single check |
credential_binding structure |
grpc/policy.rs:1891 |
No | No, stops at the first |
| Ambiguous endpoints | openshell-policy/src/ambiguity.rs, called from grpc/policy.rs:1864 |
No, for the file's own rules | Yes |
| Built-in middleware config | openshell-supervisor-middleware-builtins/src/lib.rs:29 |
No | No, stops at the first |
| External middleware, attached providers, global policy, compute driver | grpc/sandbox.rs, grpc/policy.rs |
Yes | Out of scope |
The CLI runs only the first two rows. The rest run in the gateway on CreateSandbox (grpc/sandbox.rs:548-565 and 620).
What would change:
openshell-cli: a newPolicyCommandsvariant (main.rs:2100) with its own dispatch arm. The shared policy arm callsresolve_gatewayandapply_authatmain.rs:3021before it matches the subcommand.doctor check(main.rs:2715) is the existing pattern for a command that runs without a gateway.openshell-policy: one shared function for file-only validation. None exists today. The gateway chains the checks itself on create and repeats the chain on the policy update paths (grpc/policy.rs:3606and3997).openshell-server: the reserved-name,credential_binding, and size checks move intoopenshell-policy, and the gateway calls the shared function. Otherwise the command and the gateway only match by convention.- Docs and skills:
docs/how-it-works/policies/manage-policies.mdx, a cross-reference fromprover.mdx, and theopenshell-cliandgenerate-sandbox-policyskills.
That's roughly 9 to 13 files. It doesn't touch gateway configuration or anything sensitive to SELinux or AppArmor.
Decisions for a maintainer:
- Name.
doctor checkruns locally andprofile lintgoes through the gateway, solintwould end up meaning two things. - Exit codes and output. Plain exit 1, or the prover's split of 1 for an invalid policy and 2 for usage or I/O errors. Also whether to offer
--output json. - Reporting every problem. The checks that stop at the first failure would need to return lists. The parse stage stopping early is consistent with the acceptance criterion.
- Built-in middleware config. Checking it locally adds a CLI dependency on the builtins crate. External middleware can only be skipped with a note.
- Reserved
_provider_names. If they're an error locally, the output ofpolicy get --fullfails the check and the output of--basepasses. - No path argument. Require it, or fall back to
OPENSHELL_SANDBOX_POLICYassandbox createdoes. - Version skew. The result reflects the CLI's rules, which can differ from an older or newer gateway.
Tests:
- Unit tests on the shared function, including a file with several violations.
- CLI tests with no gateway configured, using
run_isolatedincrates/openshell-cli/tests/common/mod.rs. - A parity test against the gateway is practical.
policy_only_process_identity_integration.rsalready runs an in-process gateway.
Checklist
- I've reviewed existing issues and the published docs
- This is a design proposal, not a "please build this" request
Activity
shiju-nv commentedon Oct 1, 2026
I'd call this
openshell policy lint policy.yaml. It checks a file and reports errors without applying the policy, and it follows the existingprofile lintname. Sinceprofile lintuses the gateway today, the new command should say clearly that it works offline.We can reuse
openshell-policy-schemafor the policy types and bounded parser, then use the semantic checks inopenshell-policyand the existing middleware validators. The schema parser alone doesn't cover everything though.elezar commentedon Oct 2, 2026
Classification: validated-feature
The reporter describes first-hand difficulty checking an authored policy before a gateway is available. The proposed offline file-validation workflow is technically coherent and feasible, with sufficient impact, alternatives, and observable acceptance criteria.
Independent source review on October 1 covered checkout
021400be8af471f8669369e679de3e18cf0bd672.PolicyCommands(crates/openshell-cli/src/main.rs:2100) has no standalone file validator, and dispatch resolves gateway/authentication before selecting the policy operation (main.rs:3021). Sandbox creation connects atrun.rs:552before loading the policy atrun.rs:609, confirming why a connection error masks file errors. A read of remote-main CLI source that day confirmed the command remained absent.The bounded authored-policy parser and conversion can run locally.
openshell-policy::validate_and_canonicalize_sandbox_policy(src/lib.rs:1873) already collects semantic violations, including unsafe explicit identities (src/lib.rs:1422). Gateway creation adds reserved provider rule-name, canonicalized size, safety, middleware, and credential-policy checks (openshell-server/src/grpc/sandbox.rs:550-568). Exposing only the schema parser would not satisfy the requested validation contract. Existing middleware/extension validation can supply relevant checks, but gateway-dependent services cannot by themselves satisfy the offline use case.The key design boundary is clear: validate everything ascertainable from the authored file, while retaining gateway registrations, provider attachments, image identity, and runtime capabilities as contextual checks. Collecting every post-parse problem requires accounting for validators that currently stop at the first error. Shared validation should keep local and gateway rejection behavior consistent; output should describe CLI/gateway version skew and any checks requiring live context. The suggested
openshell policy lint policy.yamlinteraction is reasonable and follows the maintainer's naming feedback; naming and detailed output remain design decisions for accepted work.The prover checks containment rather than equivalent file validity. Closed #4032 explicitly names #4035 as its replacement, so this is the canonical report. The release/duplicate searches performed during investigation found no implementing fix or canonical competing issue; the reported v0.1.2 remains the latest stable release. One impact qualification: rejecting an invalid policy need not pull an image, because gateway validation precedes compute creation; the requirement for a reachable gateway is nevertheless established.
Impact: Policy authors and potential editor/CI consumers cannot obtain complete local validation through the current CLI. A reachable gateway or self-comparison with the separate prover supplies only an unsuitable or incomplete workaround. This is a feature gap, with no removed command identified. Evidence is strong for the absent workflow and reusable architecture. Binary reproductions were not rerun: the local development CLI wrapper attempted compilation and failed under sandbox restrictions in sccache.
Next human decision: Decide whether to accept this feature, its ownership and sequencing. If accepted, settle the local validation coverage and output contract before implementation, and verify no-gateway operation, multiple simultaneous violations, and parity for file-only checks. Technical validation does not record acceptance; planning or implementation can be requested separately.
letv1nnn commentedon Oct 5, 2026
hey @mpereira, I'd like to take this one unless you are already working on it.
mpereira commentedon Oct 5, 2026
Hey @letv1nnn,
I will work on this one today.