Repository navigation
feat(observability): export an atomic sandbox governance evidence bundle #2745
Description
Activity
The digest chain gives integrity — a consumer can confirm the files match
manifest.json— but the manifest itself is unsigned, so it gives no authenticity. Nothing binds the manifest to the gateway that produced it. For the file digests that's fine, since a consumer can re-derive them from the bytes.complete: trueis the exception: it's the one claim in the bundle a consumer can't reconstruct from the bytes, and it's the one compliance leans on hardest. As a plaintext field it's only as trustworthy as whoever handed over the tarball.I'd put the vendor-neutral line one step further out than the proposal does. A gateway-authority signature over the manifest — or a key-identified digest-of-manifest — isn't TRACE-specific conformance signing; it's what makes "OpenShell established this association and this completeness" actually attributable to OpenShell rather than asserted by the exporter. Consumer-specific signing still layers on top for their own formats. Without it, the strongest guarantee in the design is the one part a consumer has to take on faith.
@lbelyaev, you are right. The file hashes prove that the files match the manifest, but they do not prove that OpenShell produced the manifest. That matters most for
complete: true, because a consumer cannot work that out from the exported files.I would change the proposal to include a detached gateway signature:
manifest.jsoncontains the export facts and file hashes.manifest.sigsigns the exact bytes ofmanifest.json.- The signature identifies the gateway key and signing algorithm.
- The verifier checks that key against the OpenShell deployment authority it trusts.
Consumer formats such as TRACE can still add their own signatures later. This first signature has a narrower job: prove that this OpenShell gateway made the policy, sandbox, event-range, and completeness claims in the manifest.
This also adds a needed acceptance test: changing
complete, a file hash, the sandbox ID, or the capture interval must make signature verification fail.Thank you for catching the gap.
@lbelyaev, you are right. The file hashes prove that the files match the manifest, but they do not prove that OpenShell produced the manifest. That matters most for
complete: true, because a consumer cannot work that out from the exported files.I would change the proposal to include a detached gateway signature:
manifest.jsoncontains the export facts and file hashes.manifest.sigsigns the exact bytes ofmanifest.json.- The signature identifies the gateway key and signing algorithm.
- The verifier checks that key against the OpenShell deployment authority it trusts.
Consumer formats such as TRACE can still add their own signatures later. This first signature has a narrower job: prove that this OpenShell gateway made the policy, sandbox, event-range, and completeness claims in the manifest.
This also adds a needed acceptance test: changing
complete, a file hash, the sandbox ID, or the capture interval must make signature verification fail.Thank you for catching the gap.
That's exactly the shape — detached
manifest.sigover the canonical bytes, key-identified, verified against the deployment authority. The tamper test (mutatingcomplete, a hash, the sandbox ID, or the interval must fail verification) is the right acceptance bar.One thing the signature leaves open: invariant 4 makes
complete: truea promise OpenShell makes about the interval, and the signature proves OpenShell made it — but a consumer still can't independently check it. They're trusting the gateway's completeness logic rather than verifying it from the bundle. OCSF already definesmetadata.sequencefor exactly this, and OpenShell doesn't populate it today — if each exported event carried a per-sandbox sequence, a missing number would be a provable gap, and a verifier could confirm the interval has no holes itself. That turns invariant 4 from a producer-side assertion into something the consumer can check. Happy to help with that acceptance test.@lbelyaev accepted, and this is the second time you have moved the proposal somewhere better than where I had it.
The distinction you are drawing is the one that matters and I had blurred it. A detached
manifest.sigestablishes that OpenShell assertedcomplete: true. It does nothing about whether the assertion is correct, because the consumer is still trusting the gateway's completeness logic rather than checking it. Attribution is not verification, and I made that argument elsewhere this month while leaving it unmade here.Folding it in. Revised shape:
manifest.jsoncarries the export facts and file hashes.manifest.sigsigns its exact bytes, key-identified, verified against the OpenShell deployment authority the consumer trusts.- Each exported event carries a per-sandbox
metadata.sequence. - The verifier checks the sequence range itself for holes, rather than reading
complete: trueand stopping.
I checked the field rather than taking it on trust:
metadata.sequenceis present on the OCSFmetadataobject at v1.9.0, typedinteger_t, described as making the exact ordering of events unambiguous regardless of event-time precision. It isoptionalthere, so populating it is a decision OpenShell makes rather than something the schema already obliges, which I think strengthens the case rather than weakening it: the field exists, the semantics are already defined, and nothing has to be invented.The acceptance bar then has two halves. The tamper half stands as before: mutating
complete, a file hash, the sandbox ID, or the capture interval must fail signature verification. The completeness half is new and is the one worth your offer of help: removing an event from the export must be detectable from the bundle alone, without asking the gateway, because the sequence has a hole.One boundary I want stated in the proposal rather than discovered by a consumer later. Sequence numbers prove nothing was removed after a number was assigned. An event suppressed before assignment leaves no hole and no trace, and a clean interval must not be read as proof that everything which happened was exported. So the verifier's output should distinguish three results rather than two: verified complete over the covered range, a provable gap, and not established. The third is not a failure and it is certainly not a pass, and if the bundle format cannot express it, the first consumer to hit it will record it as a pass.
That last part is the same shape as
complete: trueitself, one level down, which is what makes me confident it is worth building in now.@lbelyaev accepted, and this is the second time you have moved the proposal somewhere better than where I had it.
The distinction you are drawing is the one that matters and I had blurred it. A detached
manifest.sigestablishes that OpenShell assertedcomplete: true. It does nothing about whether the assertion is correct, because the consumer is still trusting the gateway's completeness logic rather than checking it. Attribution is not verification, and I made that argument elsewhere this month while leaving it unmade here.Folding it in. Revised shape:
manifest.jsoncarries the export facts and file hashes.manifest.sigsigns its exact bytes, key-identified, verified against the OpenShell deployment authority the consumer trusts.- Each exported event carries a per-sandbox
metadata.sequence. - The verifier checks the sequence range itself for holes, rather than reading
complete: trueand stopping.
I checked the field rather than taking it on trust:
metadata.sequenceis present on the OCSFmetadataobject at v1.9.0, typedinteger_t, described as making the exact ordering of events unambiguous regardless of event-time precision. It isoptionalthere, so populating it is a decision OpenShell makes rather than something the schema already obliges, which I think strengthens the case rather than weakening it: the field exists, the semantics are already defined, and nothing has to be invented.The acceptance bar then has two halves. The tamper half stands as before: mutating
complete, a file hash, the sandbox ID, or the capture interval must fail signature verification. The completeness half is new and is the one worth your offer of help: removing an event from the export must be detectable from the bundle alone, without asking the gateway, because the sequence has a hole.One boundary I want stated in the proposal rather than discovered by a consumer later. Sequence numbers prove nothing was removed after a number was assigned. An event suppressed before assignment leaves no hole and no trace, and a clean interval must not be read as proof that everything which happened was exported. So the verifier's output should distinguish three results rather than two: verified complete over the covered range, a provable gap, and not established. The third is not a failure and it is certainly not a pass, and if the bundle format cannot express it, the first consumer to hit it will record it as a pass.
That last part is the same shape as
complete: trueitself, one level down, which is what makes me confident it is worth building in now.The three-valued result is the right call, and "not-established" being first-class is the part I'd least want dropped — a format that can only say pass/fail records the third case as a pass exactly when it matters most. Same shape as
complete: true, one level down.One design input from your pre-assignment boundary: where the sequence is assigned sets how wide "not-established" is. OpenShell already has a single emit point — events pass through
ocsf_emit!/emit_ocsf_eventbefore the JSONL sink, and the sink assigns no ordering of its own. Stamping the per-sandbox sequence at emit rather than at serialization shrinks "not-established" to the smallest it can be: events suppressed beforeocsf_emit!was ever called. Anything reaching emit gets a number, so any drop between emit and export leaves a hole the verifier catches; assigning later only pulls more of the pipeline into the untrustable window.On the completeness half — the case that matters is that a removed event must show as a provable gap from the bundle alone, and an empty interval must land as not-established rather than complete. Worth pinning both as explicit acceptance cases so the three-valued output is testable, not just described.
Taking the emit-point input, and it is the better placement. Stamping the per-sandbox sequence at
ocsf_emit!rather than at serialization makes the untrustable window as narrow as the architecture allows: anything that reaches emit gets a number, so every drop between emit and export leaves a hole, and "not established" collapses to exactly one case, an event suppressed beforeocsf_emit!was ever called. Assigning at serialization would pull the whole sink path into that window for no benefit.So the shape is settled from my side:
manifest.jsoncarries export facts and file hashes;manifest.sigsigns its exact bytes, key-identified, verified against the deployment authority the consumer trusts.- Per-sandbox
metadata.sequencestamped atocsf_emit!, using the existing OCSFinteger_tfield at v1.9.0. - The verifier checks the sequence range for holes rather than reading
complete: true. - Three-valued output: verified complete over the covered range, provable gap, not established.
Two acceptance cases to pin now, so the third value is testable rather than described:
- Remove one event from a valid export. The verifier must return a provable gap, from the bundle alone, without querying the gateway.
- Export an interval in which nothing was emitted. The verifier must return not established, not complete. A clean empty interval is the exact case a two-valued format records as a pass.
Add a third if you want the tamper half covered in the same suite: mutate
complete, a file hash, the sandbox ID or the capture interval, and signature verification must fail.What I would like to agree: who implements, and by when. I can write the verifier side and the acceptance cases against a bundle you produce. If that split works, name a target and I will work to it. Six weeks of good design on an issue is worth less than a rough implementation that runs.
Taking the emit-point input, and it is the better placement. Stamping the per-sandbox sequence at
ocsf_emit!rather than at serialization makes the untrustable window as narrow as the architecture allows: anything that reaches emit gets a number, so every drop between emit and export leaves a hole, and "not established" collapses to exactly one case, an event suppressed beforeocsf_emit!was ever called. Assigning at serialization would pull the whole sink path into that window for no benefit.So the shape is settled from my side:
manifest.jsoncarries export facts and file hashes;manifest.sigsigns its exact bytes, key-identified, verified against the deployment authority the consumer trusts.- Per-sandbox
metadata.sequencestamped atocsf_emit!, using the existing OCSFinteger_tfield at v1.9.0. - The verifier checks the sequence range for holes rather than reading
complete: true. - Three-valued output: verified complete over the covered range, provable gap, not established.
Two acceptance cases to pin now, so the third value is testable rather than described:
- Remove one event from a valid export. The verifier must return a provable gap, from the bundle alone, without querying the gateway.
- Export an interval in which nothing was emitted. The verifier must return not established, not complete. A clean empty interval is the exact case a two-valued format records as a pass.
Add a third if you want the tamper half covered in the same suite: mutate
complete, a file hash, the sandbox ID or the capture interval, and signature verification must fail.What I would like to agree: who implements, and by when. I can write the verifier side and the acceptance cases against a bundle you produce. If that split works, name a target and I will work to it. Six weeks of good design on an issue is worth less than a rough implementation that runs.
Agreed on the split — you take the verifier and the acceptance cases, I take the producer side, and I'm ready to work to a target on it. The only blocker is commit access: the producer half lands as a commit here, and I'm a Contributor without write access yet — my vouch request is #2377. So I can't put my name on a merge date until that resolves.
What I can do in the meantime: keep the producer design and the two completeness cases precise enough that your verifier has a stable contract to build against, and validate a produced bundle against your test vectors as they land. The verifier side isn't blocked either way — the shape is settled (per-sandbox metadata.sequence at
ocsf_emit!, detachedmanifest.sig, three-valued output), so it can move now regardless of who implements the producer.@lbelyaev the verifier and synthetic acceptance cases are ready in agentrust-io/integrations#204. The proposed manifest includes signed first/last sequence numbers so missing edge events are detectable. Could you review the producer contract before wiring up an export? No live OpenShell compatibility is claimed yet.
Problem Statement
OpenShell already exposes the ingredients needed by external governance and compliance consumers: the effective sandbox policy, sandbox configuration and revision state, and OCSF audit events. Those ingredients are currently retrieved independently, however, so a consumer cannot establish that they describe the same sandbox state or determine whether the event interval is complete.
This is complementary to #1922 (portable, durable sandbox log collection) and #2640 (trace/span correlation). Durable logs and correlation fields are valuable inputs, but they do not bind the exact effective policy, sandbox/workload identity, event bytes, and completeness state into one authoritative export.
External adapters can hash and sign bytes they receive, but they should not invent authoritative associations or claim completeness that OpenShell itself has not established.
Proposed Design
Add an atomic export command along these lines:
openshell sandbox evidence export <sandbox> --since <timestamp> --output <directory>The export would contain:
manifest.jsoneffective-policy.yamlevents.ocsf.jsonlmanifest.jsonshould include:effective-policy.yamlbytes;completeboolean and, when false, a machine-readable reason;Required invariants:
effective-policy.yamlis the policy OpenShell actually enforced, not the originally submitted input.metadata.uididentifies the exported sandbox.complete: trueis emitted only when OpenShell can account for the entire requested interval. Rotation, truncation, gateway restart, or unavailable history must producecomplete: falsewith a reason.Acceptance tests should cover:
This primitive would support SIEM ingestion, incident response, audit archives, and third-party governance formats without coupling OpenShell to any one consumer.
Alternatives Considered
Agent Investigation
Investigation was performed against OpenShell
v0.0.105(0f8fad23c4712afc1d4a7b07a06d635b030e9521):proto/sandbox.protoexposesGetSandboxConfigResponse.config_revision.proto/openshell.protoexposes sandbox policy status/revision APIs.openshell policy get <name> --fulland JSON sandbox retrieval.ocsf_json_enabledis enabled.crates/openshell-ocsfsupplies product identity and sandbox identity throughmetadata.uid.A released external consumer demonstrates the current integration boundary and the reason the authoritative association belongs in OpenShell:
Checklist