fix(sandbox): acknowledge initial policy load and expose SDK labels

[KyleZheng1284]

Summary

Two OpenShell lifecycle/API gaps block the policy-bound per-job sandbox integration in NVIDIA-AI-Blueprints/aiq#298 for AI-Q 2.2:

1. A newly created sandbox can reach Ready and expose an effective policy while the same revision remains Pending indefinitely.
2. The public Python SDK cannot send gateway-object labels or list sandboxes by label selector, although the protobuf supports both.

AI-Q intentionally fails closed instead of exposing execution without authoritative policy attestation.

Confirmed policy-state behavior

Reproduced on macOS/Apple Silicon with Docker Desktop:

OpenShell	Creation path	Result
0.0.72	clean gateway diagnostic	Ready; effective policy present; matching revision remains Pending; current_policy_version=0
0.0.77	CLI directly, without AI-Q	Ready; revision 2 is Effective via policy get; revision 2 remains Pending via policy list; revision 1 is Superseded

The state remained stable rather than transitioning to LOADED. The policy itself was successfully constructed and applied. landlock.compatibility: best_effort is therefore not an attestation bypass or explanation; it controls filesystem-enforcement compatibility.

Linux has not yet been reproduced. The observed behavior is confirmed on the macOS/Docker Desktop path; source inspection suggests shared lifecycle logic, but a Linux run is required to establish platform scope.

Minimal reproduction

1. Create a sandbox with an explicit policy.
2. Wait until the sandbox reaches Ready.
3. Compare policy get <sandbox> -o json, policy list <sandbox>, and the sandbox's current_policy_version.
4. Observe an effective positive revision while that same revision remains Pending.
5. Delete the sandbox and verify it is absent.

Expected contract

After initial policy construction:

• report the exact sandbox-scoped revision as LOADED;
• report FAILED with a load error when construction fails;
• never leave a successfully applied initial revision permanently Pending;
• continue reporting later hot-reload revisions normally.

Suspected source path

Inspection of v0.0.72 and v0.0.74 found:

• load_policy() constructs the initial OpaEngine;
• the first run_policy_poll_loop() iteration records config_revision and policy_hash;
• ReportPolicyStatus(LOADED) is sent only from the later policy_changed branch;
• an unchanged initial hash never enters that branch.

The same runtime symptom is present in 0.0.77; its exact source path should be confirmed against that tag.

Python SDK label/selector gap

CreateSandboxRequest supports request-level labels and ListSandboxes supports label_selector, but the public Python lifecycle API does not expose them:

• SandboxClient.create(), create_session(), and Sandbox(...) cannot forward gateway labels;
• SandboxClient.list() cannot forward a selector;
• SandboxRef does not retain gateway metadata labels.

Template labels reach Docker/runtime resources, but gateway selectors inspect gateway-object metadata. Consequently, a Python-created AI-Q sandbox is not returned by:

openshell sandbox list --selector aiq=deep-research

Proposed fix

Initial policy acknowledgement

• Preserve the fetched sandbox policy version/source through initial construction.
• Report initial success or failure exactly once.
• Track the acknowledged version so later polling does not duplicate it.
• Preserve existing hot-reload behavior.

Python SDK

Add optional, source-compatible parameters:

• name and labels for creation/session/Sandbox;
• label_selector for listing;
• immutable gateway labels on SandboxRef.

The lifecycle and SDK fixes may land separately, but AI-Q requires both in a tagged release.

Acceptance criteria

• Initial success and failure acknowledgement tests.
• Later policy-revision/hot-reload regression tests.
• Python request-label and selector-forwarding tests.
• Docker E2E: create → Ready → LOADED → selector membership → execute → delete.
• Direct lookup and selector both confirm deletion.
• Tagged release with compatible CLI, Python SDK, and gateway versions.
• AI-Q macOS demo validation with explicit best_effort.
• AI-Q Linux production validation with hard_requirement.

Downstream impact

This issue blocks AI-Q #298. It blocks only the OpenShell-specific live acceptance for stacked artifact-lifecycle PR AI-Q #305; #305's provider-independent artifact-storage implementation can be restacked onto develop and proceed separately. AI-Q will not treat Pending as success, weaken attestation, or pin an unreleased fork.

A focused implementation PR can be prepared after maintainer triage and the repository's human state:agent-ready gate.

[KyleZheng1284]

Cross-project impact: blocks AI-Q 2.2 OpenShell integration

This issue currently blocks NVIDIA-AI-Blueprints/aiq#298, which integrates policy-bound, per-job OpenShell sandboxes for the AI-Q 2.2 release. The AI-Q PR has been marked BLOCKED and will continue to fail closed rather than expose execution before authoritative policy attestation succeeds.

There is now an additional reproduction using the OpenShell 0.0.77 CLI directly, without AI-Q, on macOS/Docker Desktop:

• the sandbox reaches Ready;
• openshell policy get reports revision 2 as Effective;
• openshell policy list leaves revision 2 Pending, with revision 1 Superseded;
• AI-Q's subsequent timeout is therefore a consequence of the unresolved authoritative status, not its sandbox-creation implementation.

This also shows that upgrading from the earlier 0.0.72 reproduction to 0.0.77 does not by itself resolve the policy-state contradiction. best_effort is not being treated as an attestation bypass; it only changes Landlock compatibility.

For AI-Q to unblock, we need:

1. confirmation of the intended authoritative success contract for Effective + Pending;
2. initial policy construction to report LOADED or FAILED exactly once if Pending is not intended;
3. public Python SDK support for request-level labels and selector-based listing;
4. a tagged release containing the fixes so AI-Q can pin matching CLI/SDK/gateway versions;
5. successful macOS demo and Linux hard_requirement live acceptance.

A focused upstream implementation PR can be prepared after maintainer triage and the repository's human state:agent-ready gate. A Linux CLI-only reproduction is still needed to establish whether the observed runtime manifestation is limited to macOS/Docker Desktop or is platform-independent.

[johntmyers]

@KyleZheng1284 I'm unable to reproduce. I see a different gap where current version was 1 vs 0 between the policy get and sandbox list commands but I'm not seeing the Pending state. Will post a more thorough analysis shortly.

[johntmyers]

📋 triage-agent

Triage Assessment

Classification: bug-confirmed

Summary

I reproduced the initial policy acknowledgement problem on local docker-dev gateway 0.0.78-dev.6+ga7271169. The exact revision 2 Pending state reproduces when the initial policy is sparse enough for sandbox-side baseline enrichment to sync an updated policy back to the gateway during startup. The Python SDK label/selector API gap also reproduces: public SDK methods reject labels, name, and label_selector, and template labels do not satisfy gateway object selectors.

Reproduction

Using examples/policy-advisor/sandbox-policy.yaml:

openshell sandbox create --name issue-2159-enrich-repro \
  --policy examples/policy-advisor/sandbox-policy.yaml \
  --label enrich-2159=true \
  --no-tty -- true

Observed after the sandbox reached Ready:

openshell policy get issue-2159-enrich-repro -o json

returned version: 2, active_version: 2, status: effective.

openshell policy list issue-2159-enrich-repro --limit 50

returned revision 2 Pending and revision 1 Superseded.

openshell policy get issue-2159-enrich-repro --rev 2 -o json

returned status: pending, active_version: 0.

openshell sandbox list --selector enrich-2159=true -o json

returned current_policy_version: 0.

Sandbox logs showed the policy was actually used successfully: CONFIG:ENRICHED, Landlock applied, and the workload process launched.

Root Cause

current_policy_version lives on SandboxStatus, not in policy history. Hot reload updates it only through ReportPolicyStatus(Loaded).

Relevant paths:

crates/openshell-server/src/grpc/policy.rs:1133-1179: GetSandboxConfig lazily backfills spec.policy to revision 1 and marks it loaded, but does not update Sandbox.status.current_policy_version.

crates/openshell-sandbox/src/lib.rs:1456-1465: sandbox startup may enrich the fetched policy and sync it back to the gateway.

crates/openshell-server/src/grpc/policy.rs:1879-1891: that sync uses UpdateConfig, which inserts revision 2 as pending and supersedes older revisions.

crates/openshell-sandbox/src/lib.rs:1634-1668: the poll loop seeds current_policy_hash from the first poll, so the already-current enriched revision is not treated as a policy change.

crates/openshell-sandbox/src/lib.rs:1775-1782: ReportPolicyStatus is only called inside the later policy_changed branch.

crates/openshell-server/src/grpc/policy.rs:2051-2069: ReportPolicyStatus(Loaded) is the path that updates Sandbox.status.current_policy_version.

Why Commands Disagree

openshell sandbox get prints Revision from GetSandboxConfig.version (crates/openshell-cli/src/run.rs:2635-2711).

openshell sandbox list -o json prints Sandbox.status.current_policy_version (crates/openshell-cli/src/run.rs:3427-3438).

openshell policy get without --rev labels the current config as effective and sets active_version equal to that config version (crates/openshell-cli/src/run.rs:7062-7123).

openshell policy get --rev uses GetSandboxPolicyStatus, whose active_version comes from sandbox.current_policy_version() (crates/openshell-server/src/grpc/policy.rs:1927-1936).

Recommendation

Fix in the sandbox runtime rather than only in server backfill. After initial policy construction succeeds, the sandbox should report the actual initial sandbox-scoped policy version exactly once via ReportPolicyStatus(Loaded). If initial construction fails after a persisted revision exists, report ReportPolicyStatus(Failed) with the load error. Then seed the poll loop with the acknowledged version/hash so existing hot-reload behavior does not duplicate reports.