Skip to content

test: run behavior suites and enforce local push checks (#194 Part 1) - #224

Merged
andersalm merged 18 commits into
developfrom
fix/194-behaviour-ci
Oct 4, 2026
Merged

andersalm merged 18 commits into
developfrom
fix/194-behaviour-ci

Conversation

@andersalm

@andersalm andersalm commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

CI runs the previously omitted capsule JavaScript unit tests and signed installer tests. Three source-text checkers are replaced by data validation and executable behavior checks; copy and symbol-presence assertions are removed. Runtime and UI behavior use one Node test command, with every refused case retained.

The local pre-push script and git hook enforce Anders's checks on the actual pushed ref: current develop, clean source, formatting, workspace all-target compile, touched-crate Clippy and changed-module unit tests with positive counts. The hook is enabled in this task's worktree.

Published candidate 149c39a normally integrates develop e8d9564. Its full local gate and actual-ref push hook pass. Current CI37192908134 is green. Current elastos-pr-review PASS follows matched API/fetched heads; required human approval precedes merge.

The draft starts from develop fe033fe. It removes 16,157 lines from check-wci-alignment.sh, home-entropy-check.mjs and browser-entropy-check.mjs. Standing command references use the data checker; historical audit records stay intact. The canary workflows on fix/89-canary-inputs remain release-owned.

The data suite has 92 tests, including malformed manifests, unsafe paths, missing entrypoints and icons, protected method approval metadata, and catalogue integrity refusals. The initial fast Node suite passed 1,247 tests in 4.34s; 28 ESP projection tests and two acceptance-refusal tests are now included. The repaired Browser focus fixture passes locally with explicit bundled Playwright/Chromium inputs in 1.43s; hosted pins remain the candidate proof. Four retained Browser suites receive pinned Playwright/Chromium inputs in the hosted behavior job. Fixture extraction and current startup inputs were repaired in existing tests.

The current signed installer suite discovers 55 tests: 51 pass locally, with four optional published-fixture tests skipped by their existing input gate. CI runs the key-free installer unit/refusal suite. Four optional published-byte cases keep their existing --publisher-fixtures input gate; the separate native CI journey and #89 operator acceptance provide installed proof. The same Node command and data validator are available through just test-behaviour and just product-data. Release retains the Mac heavy build slot.

Coverage retained after deleting the text checkers:

  • Capsule data and packaging: check-product-data.test.mjs mutates manifests, profiles, icons, method approval contracts, release paths, catalogue integrity, declared publication sets and confined icon paths; each refused case executes the new validator. Linux extraction paths stay mandatory for first-party assets.
  • Home isolation: browser_capsule_documents_allow_isolated_home_embedding and the symlink escape refusals exercise served capsule CSP and artifact paths. gateway_tests/home_system.rs retains Home authentication and cookie authority checks.
  • Runtime provider ownership: provider_runtime_contract_covers_exact_active_provider_set still checks cross-file authority scopes in components.json; stream bridge networking refusal tests remain in Rust.
  • Signed publication and installation: release_publication.rs retains pin, signer/domain, hash and tampered-byte refusals; release-platform-input-test.py and install-bootstrap-test.py retain signing-input and installer refusals.
  • Home, Browser and wallet behavior: the selected Node suites retain close ownership, wrong-token refusals, signing sessions, frame selection, stale/reconnect and conflict behavior; elastos/esp/projections.test.mjs now joins the same command.
  • Native networking: the existing supervisor/proxy smoke now requires a distinct kernel network namespace, real loopback-only listeners, direct TCP/DNS refusal and Runtime relay success. Hosted Linux CI builds both helpers, runs the prebuilt fixture with explicit isolation, and reaps its owned children. This is fixture proof; installed Browser media acceptance keeps its separate gate.
  • Copy, source symbol presence and deprecated spelling checks are removed. Installed media, Home GUI isolation and Runtime projection acceptance continue to require their own candidate/manual evidence; two new fixtures prove valid data alone cannot close those gates.

Historical verification of candidate 35b636d, tree 25c007557815816016f86ef24bd22167707ecece, has green CI37136105515, historical elastos-pr-review PASS and independent gpt-6.1-sol high development PASS. The API and fetched heads matched that historical candidate. Formatting, diff and CI policy checks passed on it. All repair commits and prior review evidence stay in history. Release retains the local Mac heavy build slot. Existing Browser objective/runbook smokes retain their doc-copy failures; installed product acceptance continues to use its artifact, media and manual proof gates.

Hosted before/after proof uses cargo test -- --list in the same jobs. Job time includes checkout, tools and other checks; the test step includes compilation and execution. These observed runs vary with runner/cache state and source base. Capsule jobs took longer than baseline, so this evidence establishes coverage and test inventory and keeps an overall speed claim open.

Candidate Workspace list Capsule list Total test-elastos job / test step test-capsules job / test step
Baseline 3,792 898 4,690 9m33s / 6m45s 9m59s / 9m39s
Part 1 3,792 898 4,690 12m19s / 9m13s 15m37s / 14m02s
Part 2 3,680 876 4,556 9m13s / 5m58s 15m24s / 13m47s
Current Part 1, develop e8d9564f 3,796 905 4,701 12m40s / 9m36s 15m25s / 13m54s

Historical Part1 behavior CI passed all 1,334 Node tests in 11.13s with zero skips, including the 95 previously omitted capsule tests. That signed installer CI discovered 55 tests: 51 passed in 41.64s and four retain their existing published-byte fixture gate. The actual native guard passes in 13s, bound to source cad4975/tree 7c2cb779fa6e711b25ac67d94be5c2ccb372c996. Supervisor SHA-256 8f6aab192bd7918920a08b0a56c14129e1ff8129ca5f0d1b42fffe231212b16b; proxy SHA-256 8a061b70cf4a316dc55f6910bc614210521f280d037bbedfcdd1ae6c223dedb0. It proves a distinct kernel namespace, loopback-only listeners, direct TCP/DNS refusals, relay success and stopped children.

The independently inspected Mac receipt 11279040958 passes frozen install, plain Carrier check/apply/repeat, all 12 refusals, config/data/frozen-support preservation, four empty successful stderr streams, 17 clean holder outputs and empty owned cleanup. Install/check/apply/repeat take 1.544s/2.193s/1.400s/0.026s; 53 observer and 3 build processes stop, with zero M2 HTTP fallback. This disposable regression proof is separate from #89 operator acceptance. Source cad4975/tree 7c2cb779fa6e711b25ac67d94be5c2ccb372c996 matches its tested PR merge against develop ad85c2e. Receipt JSON SHA-256 01ed20a65f143fc2c4130b1bd897f2d5f1453523ce8c5417307db1c08d732944; ZIP SHA-256 ccdc058e9f90d11de58ad22e77b2837c873ffabc5f40750dd8986b3ca909a13d. Runtime N 0.7.1-dev SHA-256 1184917a6ed9aa0151c907e675b76760391876b37ee3c33f5ce6fca146f75276; compiled N+1 0.7.2 SHA-256 1a8ce98f016f183342bf5e2bf72e8298f87fa0be476fd28a5ea6c26fcb6b8186.

Anders reports #89 M2 passed and lifts the #194 hold. The branch normally integrates current develop e8d9564. The new pre-push script and committed worktree hook require the actual pushed ref to match clean HEAD and current develop. They run formatting, separate workspace all-target compile, touched-crate Clippy and changed-module unit tests with positive executed counts. Runtime inputs select their consumers and reverse standalone dependencies; uncertain module mapping widens the unit scope. Hook fixtures clear Git's repository-local environment before mutating disposable repositories. Command ownership and the stable heavy-build lock keep interruption cleanup bounded.

The gate keeps shared Cargo intermediates and workspace-owned final outputs. An isolated pinned-Cargo A/B/A experiment proved older-source reuse across worktrees even with separate output directories. The repaired gate cleans the resolved repository path-package closure before compile and each provider release closure before its build. External package/dependency-library collisions refuse before cleaning; registry sources/downloads and distinct-name artifacts stay preserved. Tracked locks resolve with --locked, and ignored generated locks keep their recorded resolution policy. Equivalent did-provider formatting and Option mapping satisfy the mandated all-target checks.

Current published candidate 149c39a, tree 117dd3b7f63c296b9f841df3ff3ca85c946dd6fe, passes the full native pre-push gate in 914.517s: 59 gate fixtures, 92 product-data cases, 17 workspace checks, 22-package Clippy, 3 fresh provider builds and 24 unit targets with 3,816 parent-target passes, 27 ignored and zero failures, plus one separate custody subprocess. Final current-develop and clean-source checks pass. Independent high integration/proof review and bounded local Opus integration review pass; all 326 observed PID/groups ended and the lease released. The actual-ref hook also passes in 897.569s and the normal push publishes that exact head. Current elastos-pr-review PASS applies to this published head; current CI is green. The PR is ready and irzhywau has the required review request. His approval precedes the normal Part1 merge; Part2 follows through normal integration preserving source and refusal history.

Current CI passes 1,346 Node tests in 11.812s with zero failures/skips, 59 gate fixtures, and the installer suite (55 discovered, 51 passed, four published-byte fixture cases skipped by their existing input gate, 40.454s). All 10 active jobs and 127 executed steps pass; the dispatch-only Mac fixture and release publication skip as intended. The current Mac rehearsal receipt11300397800 binds checkout55ab661bd6561ab9e0fe07e2accb5c7bb2d9ec6d to candidate tree117dd3b7f63c296b9f841df3ff3ca85c946dd6fe. Its 6 install and 10 discovery/update checks pass, refusal cases preserve prior state, 53 observer and 3 build processes end, remaining PIDs are empty, and M2 HTTP fallback is zero. Result JSON SHA-2569378b317702b1a533ee80406e9f774850f7af18da55d0f21e9bcac6075c3132a; N binary0377370b3b10e3be90237d10c899564b024a6ab3be78000a90498dff612a9b75; N+1 binary9a17beff16707cc33aa29d886af50ae693119f383bec5633adb12db1bf85d7d4; fixture manifest27b6f390606887fdb562832d6b17409fd5d00377e77137c888fcc867a5102c10. This is disposable CI rehearsal; the operator Home/account/update acceptance keeps its own gate.

Refs #194.

@andersalm

andersalm commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Independent gpt-6.1-sol high development round PASS on 35b636d / tree 25c007557815816016f86ef24bd22167707ecece; origin divergence 0/0.
Publication data parity, extraction and icon confinement retain executable refusals. Native guard script SHA-256 4f6c5bccc4a7424ca446fb12e3d4859b81f7092cefc57930c91bc35ace762349 retains namespace, loopback, direct-network refusal and owned-child cleanup.
The current two-line repair removes the strict request's unsupported field and sets the relay's actual 27-byte Content-Length; syntax, field/byte checks, formatting, CI policy 19 tests and diff checks pass.
Current CI37136105515 and renewed current-head Opus are running. The prior behavior job passed 1,334 tests in 12.66s with zero skips; the previous native input failure ended cancelled and its source/review history remains.
Merge waits for #89 real-Mac M2, current checks and required reviews; installed Browser/media acceptance keeps its own gates.

@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review, 2026-10-03), head a32e3e2.
VERDICT: PASS

Blocking findings: none. Most of the deleted checks were source-text pins. I spot-checked the authority and provenance ones, and Rust or Python behaviour tests still cover each of them:

  • Capsule CSP sandbox without allow-same-origin: browser_capsules.rs:987,1093,1148.
  • Home routes do not mint session cookies: gateway_tests/home_system.rs:384,518.
  • The stream bridge refuses direct networking: browser-stream-bridge/src/main.rs:272.
  • Publication refuses a wrong pin or tampered bytes: release_publication.rs:1239-1370.
  • Providers stay Runtime-only: setup.rs:4179.
  • Signing-input admission: release-platform-input-test.py.

The new validator's refused-case tests mutate valid data and would fail if their checks were removed. Install refusals at install-bootstrap-test.py:748,929 run without the optional fixture gate.

Non-blocking notes:

  1. Issue CI and tests: warm caches, fail fast, keep only what catches real bugs #194's coverage gate asks each deleting PR to list what the deleted checks covered and which tests still cover it. This PR deletes about 16k lines without that list. Please add it before merge.
  2. Two objective audits now treat productDataValid as proof of unrelated things: Home DOM isolation, projection contracts and native media. See home-shell-objective-audit.mjs:228,351 and browser-objective-audit.mjs:492. This overclaims.
  3. elastos/esp/projections.test.mjs is still not in the node --test set. The PR description says CI uses "the retained release artifact", but CI calls install-bootstrap-test.py without --publisher-fixtures, so the 4 tests on real published bytes still skip.

Not checked:

  • Hosted CI results, including whether test-behaviour stays under 60 s and passes with Playwright/Chromium. CI is still pending.
  • Whether branch protection marks test-behaviour as a required check.
  • Every one of the roughly 1,500 deleted assertions; I sampled them.
  • Installed-Home app launch. This PR changes only CSS and HTML comments in shipped capsules.
  • The G1 Install and update: Signed update #89 Mac acceptance.

@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review, 2026-10-03), head 98661f9.
VERDICT: FAIL

Blocking findings:

  • scripts/check-product-data.mjs:193-265 (validateComponents): this PR deletes the only check that kept the published capsule sets in line with the components.json home profile (deleted check-wci-alignment.sh, pr.diff:5060-5097). The sets are DEFAULT_CAPSULES and REQUIRED_SUPPORTED_CAPSULES in scripts/publish-release.sh:34-100, and HOME_PUBLISH_CAPSULES, REQUIRED_SUPPORTED_PUBLISH_CAPSULES and DEMO_PUBLISH_CAPSULES in elastos/crates/elastos-server/src/publish.rs:10-47. I searched the whole repo and nothing else compares them. The publish.rs:2115-2141 tests only compare the constants with themselves. If a capsule is added to the home profile but not to the publish lists, a signed release can ship without that app's artifact, so the app is missing or fails to install in the installed Home. Issue CI and tests: warm caches, fail fast, keep only what catches real bugs #194 Part 1 says the checks of real data move into the lint. This one did not move, and the PR's coverage list does not mention it. Fix: add the parity check to validateComponents (read both the shell arrays and the Rust consts) and add a refusal test.

Non-blocking notes:

  1. The Browser engine's network isolation (CLONE_NEWNET, browser-engine-supervisor/src/main.rs:503) and the loopback-only proxy bind lose their only CI guard. browser-native-supervisor-smoke.sh tests this behavior but runs in neither CI nor just. The coverage list should say so.
  2. The icon containment check (check-product-data.mjs icon block) uses resolve, not realpath, so a symlinked icon directory pointing outside the capsule passes. The test fixture itself uses symlinks.
  3. The old check required extract_path for home, system, documents, library, marketplace and inbox on every Linux platform. The new check requires it only when release_path ends in .tar.gz.

Not checked: hosted CI results, including the test-behaviour runtime, 0 failures and whether the Playwright 1.60.0/1.59.1 installs work. I also did not run install-bootstrap-test.py on Linux, the Node suites, or the "1,247 tests" count.

@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review, 2026-10-03), head e8fc25a.
VERDICT: PASS

Blocking findings: none.

  • Runtime protections stay in place. Three removed text checks matter for security, and the Runtime still enforces each one with tests:
    • The ban on allow-same-origin in Home iframes: browser_capsules.rs:362-372 puts a sandbox without allow-same-origin on every non-Home capsule document, and :1148 tests it.
    • The ban on direct fetches in app source: connect-src restricts it.
    • The wallet-price approval: gateway_tests/wallet/prices.rs covers it.
  • The manifest authority checks are kept and slightly widened. These cover role, host_process/guest_network, raw backends, provider overrides, microVM ingress and external dependencies, in check-product-data.mjs:36-110. For non-provider roles, the else branch now also covers shell.
  • Refused cases are tested. The refusal tests change one field at a time, and each would fail if its assert were removed. The CLI exit code is checked against a broken fixture, as CI and tests: warm caches, fail fast, keep only what catches real bugs #194 asks.
  • Installer refusal tests run unskipped in CI. install-bootstrap-test.py runs in source-gate, and the corrupt-artifact and signature/binding failure tests (:748, :929) need no fixture flag.
  • The native smoke now fails closed. With --require-isolation, it fails instead of skipping when isolation is unavailable. It also checks the netns inode and that the proxy listens on loopback only.

Non-blocking notes:

  1. ci.yml:101-103 runs npm install playwright@… with no lockfile or integrity pin, and release now depends on that job. Registry or CDN trouble can therefore hold a release tag.
  2. The 60-second time limit for the JS step in CI and tests: warm caches, fail fast, keep only what catches real bugs #194 has no proof yet. Four of the suites are Playwright, with --test-timeout=60000 for each test.
  3. Local just test-behaviour fails on the Browser suites unless NODE_PATH and BROWSER_OPERATOR_PLAYWRIGHT_CORE are set. The AGENTS.md "no npm install" line holds only for build, lint and test.

Not checked: I ran nothing; this review is static. I did not confirm that CI37134868577 is green, the test counts or timings, or the result of the root-run netns smoke on the hosted runner. I spot-checked about 12k deleted lines of home-entropy-check.mjs and did not review all of them.

@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review, 2026-10-03), head 35b636d.
VERDICT: PASS

Blocking findings: none

Non-blocking notes:

  1. .github/workflows/ci.yml (test-behaviour) installs playwright@1.60.0 and playwright-core@1.59.1 from npm without a lockfile or integrity pin, then downloads Chromium from the CDN. The release job now needs this job, so a tag build depends on npm and the CDN being available. No release artifact uses these packages, so this does not create a supply-chain path into shipped bytes.
  2. The browser-native-supervisor-proxy-smoke.sh --require-isolation step and the test-native-browser-isolation recipe in justfile run the supervisor under sudo -n. The new checks are real behavior checks, and stronger than the deleted source-text check for TcpListener::bind("127.0.0.1:0"): a separate network namespace from /proc/self/ns/net, /proc/net/tcp showing only 0100007F, direct TCP and DNS refused, and a reaped child tree. But they prove isolation only when run as root. They do not show that the unprivileged installed launch gets the same namespace. The PR correctly calls this fixture proof.
  3. The deleted static scans looked for allow-same-origin, raw provider routes and http://127.0.0.1 in ordinary capsule source. That protection now depends on enforcement in the Runtime. The CSP builder in browser_capsules.rs gives every non-Home document a sandbox without allow-same-origin and limits connect-src to the Home origin. The tests in browser_capsules.rs around lines 987, 1093 and 1148–1149 and gateway-content-sandbox-browser-smoke.mjs cover this. No test asserts connect-src for an ordinary app document directly; the shell-document test covers it only because both use the same builder. A focused app-document assertion would close that gap.

Other checks: check-product-data.mjs fails closed on every refused case in check-product-data.test.mjs, including the CLI broken-manifest test. Both objective audits now keep their criteria open instead of passing on source text. The edited Browser startup test matches browser.js:2732-2744. Only historical docs and the CHANGELOG still mention the deleted checkers.

Not checked: the results of CI37136105515, including the claimed 1,334 Node tests with zero failures and the namespace fixture as root on hosted Linux. I also did not check whether vendor-ui-tokens.sh --check runs anywhere in CI, the four skipped published-fixture installer tests, or #89 M2 Mac acceptance.

@andersalm andersalm mentioned this pull request Oct 3, 2026
10 of 27 tasks
@andersalm andersalm changed the title test: run behavior and signed installer suites (#194 Part 1) test: run behavior suites and enforce local push checks (#194 Part 1) Oct 4, 2026
@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review, 2026-10-04), head 149c39a.
VERDICT: PASS

Blocking findings: none. The did-provider change is formatting only. All new CI jobs run on GitHub-hosted runners with contents: read. The deleted text checkers protected several authority boundaries, and I sampled those. Each one I traced is still enforced at run time and covered by a test:

  • Capsule sandbox and network limits: the opaque sandbox and the connect-src limit are tested in browser_capsules.rs:1148-1169.
  • Session cookies: the Home static route sets no session cookie, tested in gateway_tests/home_system.rs:384-518.
  • Provider binaries: unstamped binaries and checksum mismatches are refused, tested in binaries.rs:245-361.
  • Installer: tampered and wrong-signer releases are refused in install-bootstrap-test.py:929,1355,1361.
  • New product-data checker: check-product-data.test.mjs changes the data and checks that the checker refuses it, so these tests would fail without the checker.
  • Pre-push gate: it fails closed on an extra ref, a changed object or a stale base (ci-local-prepush.py:105-120), and refuses a run that executes zero tests (:467-491).
  • Browser tests: the Node behaviour suites contain no conditional skips, so a missing Playwright input fails the job.

Non-blocking notes:

  1. The pre-push gate always fetches from origin (ci-local-prepush.py:116). In a worktree whose upstream remote has another name, such as elacity, it checks the wrong base or fails.
  2. The native isolation smoke runs as root through sudo -n (ci.yml test-elastos step, justfile test-native-browser-isolation). It proves the browser gets its own network namespace, but it does not prove the engine runs without root. Do not cite it for the "no root at run time" promise.
  3. These removed source guards have no test that replaces them: window.ethereum/personal_sign in System, and navigator.credentials in Inbox, System and Wallet. CSP and the sandbox do not stop a wallet extension from injecting into a capsule frame. Separately, the behaviour job installs Playwright with no lockfile, so its transitive dependencies are not pinned.

Not checked: I ran no tests and did not check the CI result. I also did not check whether test-behaviour is a required check in branch protection. I only sampled the 12k-line home-entropy-check.mjs deletion. The run-time figures (under 60 s, 1,334 tests) come from the PR description and I did not verify them.

@andersalm
andersalm merged commit 3ce5f36 into develop Oct 4, 2026
12 checks passed
SashaMIT added a commit that referenced this pull request Oct 4, 2026
develop (#224) removed scripts/home-entropy-check.mjs, so nothing pins the phone media list to PHONE_MAX_WIDTH any more.

Co-authored-by: Cursor <cursoragent@cursor.com>
@andersalm andersalm mentioned this pull request Oct 7, 2026
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant