Skip to content

ci: prove signed SmolLM2 Get and Assistant reply in fresh Homes - #153

Open
andersalm wants to merge 43 commits into
developfrom
feat/0.7.2-ci-core-journeys
Open

andersalm wants to merge 43 commits into
developfrom
feat/0.7.2-ci-core-journeys

Conversation

@andersalm

@andersalm andersalm commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

CI installs a fresh source Home, Gets pinned SmolLM2 through a signed disposable catalogue and the normal Marketplace control, then displays a retained local reply in Assistant. The journey uses the visible model picker, Prompt output setting and Send control. It checks the actual 16-token request, completed typed output and desktop/phone screenshots.

Source setup uses the current licensed engine and Kubo recipes before writing its installation receipt. A separate fresh Home proves ordinary use with the optional engine absent. Runtime owns package admission through Content. The fixture uses local Kubo blocks; signed-release engine acquisition and physical Jetson acceptance keep their recorded gates.

All three CI rows require a real reply. Mac runs three fresh Homes and records engine readiness, generation, durable delta/terminal Applied timing and spread. Linux keeps provider stderr confined and records the real reply, exact artifacts, disk reserve and owned-process cleanup. The existing 120-second provider deadline and current signed Mac update journey stay in place. Browser setup failures now write safe receipts; the picker waits for offer readiness, and phone captures wait for the closed sidebar.

Local installed proof passes at 064c5729ad7e116cf1cc59f1a5b0c086ece7d28d (tree 64b030052a6f6aa15e4376e3da1845e4a96eb480): an engine-absent Home and three signed Get-to-reply journeys, with inspected desktop/phone media and all final owned-process counts zero. Built and installed Runtime SHA-256 match c941daf3ab453a41a645966c471ea445923b65acec81c6f98658b7aa83637b8f. The recorded journey cost is 334.48 seconds (5m 34s), including fixture preparation, after engine/browser/model-input setup. CI will record the total added time on each platform in #150.

Validation: 52 CI policy tests, 16 fixture/privacy tests, product-data, canonical vendor/shell, Marketplace behavior, syntax and current development/adversarial source reviews pass. The full push gate passes, including 2,803 Runtime and 51 selected model-provider tests. Exact published-head review, three-platform CI artifacts and required human approval are recorded in #150 before merge.

PR #208 supplies canonical model-contract parity and is included through a normal merge. The branch refresh preserves existing PR and develop history. #92 owns the confirmed reduced-motion phone layout finding.

Refs #150, #84, #89, #136, #208, #167.

@andersalm

Copy link
Copy Markdown
Contributor Author

Measured added CI time for candidate 8837e66a in completed run 36856389961: prepare/journey time was ARM64 44/66 seconds, Mac 17/65 seconds, and x86-64 26/64 seconds.
The added block, including Node handoffs, summaries and artifact upload, totaled 119/87/92 seconds respectively: 298 seconds, or 4.97 runner-minutes. The observed critical x86-64 branch contains 92 seconds of added steps.
These are partial failure measurements: all three prepare steps passed, then operator serve returned a bearer-token rejection. Packaging was skipped, so a successful baseline and full critical-path increment remain unavailable.
Published correction 0738258f starts Home gateway and checks its HTTP response. Before publication, a fresh isolated fixture passed HTTP200, enrollment and signed Home using the accepted #154 installed binary; all owned processes and copied fixture data are cleaned up.
The PR description carries the timing table. PR153 stays draft for exact installed replies/screenshots, successful timing, PR133 merge, PR155 integration and required review.

@andersalm

Copy link
Copy Markdown
Contributor Author

Completed CI 36865084148 at 7429962a measured added steps: x86 prepare 44s / failed journey 120s / total 166s; ARM 209s / 121s / 348s; Mac 19s / 120s / 144s. The added steps used 658s (10.97 runner-minutes). The observed critical ARM branch includes 348s of added steps; skipped packaging leaves the successful baseline and full critical-path increment open. These failed-run times are separate from installed acceptance. Candidate 7022fe93 checks gateway /healthz; its isolated installed route proof and four basic gates pass.

@andersalm

Copy link
Copy Markdown
Contributor Author

Independent agent source/security review: round 2 PASS for the Mac runner headroom and safe failure-receipt correction at fc5cdd97b1b552378483fd5d8b66c09991901395. The hosted-runner guard, physical application scope, refreshed Xcode selections, directory identities and final numeric space checks address all three first-round findings. Basic gates, release-policy tests and narrow execution checks pass. The original Mac HTTP409 cause remains unproven; exact installed CI is pending. PR153 stays draft for PR133 merge, reviewed PR155 integration and required human review.

@andersalm andersalm mentioned this pull request Oct 1, 2026
7 tasks
@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review, 2026-10-02), head aa9a2bd.
VERDICT: FAIL

Blocking findings:

  1. elastos/crates/elastos-server/src/setup.rs:347 calls verify_arm64_model_host(&platform)? inside the component loop, and the ? aborts the whole run. llama-server is part of the default home profile (components.json:1568), and scripts/install.sh:762 runs plain elastos setup. On a Linux ARM64 host that lacks dot product or FP16 (Cortex-A53/A72 boards such as Raspberry Pi 3/4, or Graviton1), the whole Home install now stops before any component installs. Before this PR, llama-server was a source-build skip and Home still installed. Only the model should be refused on that host. There is no test for "unsupported ARM host still installs Home", and the HWCAP tests only check the bit predicate.

  2. setup.rs:2734 makes every first-party Carrier fetch bind carrier_bind_addr, and carrier.rs:1270-1283 turns off the ephemeral fallback. The running Runtime already holds that UDP address (server_infra.rs:1525). So with the documented carrier_bind_addr 0.0.0.0:4433, three paths now fail with AddrInUse:

    • in-process installs through supervisor.rs:1487 (/api/supervisor/ensure-external)
    • gateway startup when an external is missing (gateway_cmd.rs:137,194)
    • elastos setup --with X while the Runtime runs

    All of these used to work. docs/COLLABORATION_NETWORK_PROFILE.md:16 still says the setting "changes the listener address only", which is now wrong. No test covers a running Runtime that owns the address.

Non-blocking notes:

  • Most LocalTiming::record stages print to stderr on every local run, whether ELASTOS_MODEL_TIMING_DIAGNOSTICS is set or not (local_llama.rs). This adds production log noise, but I found no secrets in it.
  • The ubuntu-22.04-arm job reports as check name ubuntu-24.04-arm, so the check name misstates the OS.
  • CI: prove installed journeys on GitHub machines #150 requires the PR to state the added CI time. It does not.

Not checked:

  • That run 36964468702 passed, and its receipts and screenshots.
  • Whether the Runtime forwards the diagnostics env var to model-provider.
  • iroh clear_ip_transports behaviour with relays.
  • The real ARM engine digest and Jetson hardware.

@andersalm

Copy link
Copy Markdown
Contributor Author

The reviewed head aa9a2bda predates the merged Carrier repair in #133. At develop 829a24ad, setup.rs:2766–2790 selects FirstPartyCarrierContext::Runtime and a temporary port on the configured IP for in-process downloads; the held-listener Runtime/Browser tests are recorded with #133's 32-test proof. Standalone setup retains its configured port and occupied-port refusal, as recorded in #97.

The unsupported ARM check still propagates through the whole Home setup at develop setup.rs:348; that finding remains open. The current candidate stays draft with this FAIL. After isolation hands off #202's accepted installed proof, the next candidate will integrate the Runtime repair and address the Home continuation/refused-engine case with narrow tests, then obtain a fresh Claude Opus verdict. The existing five-line #150 records and its three review-note bullets are updated.

@andersalm andersalm changed the title ci: prove installed Home and pinned model journeys on three platforms ci: prove signed SmolLM2 Get and Assistant reply in fresh Homes Oct 8, 2026
@andersalm

Copy link
Copy Markdown
Contributor Author

Independent development review — VERDICT: PASS; head 064c572, tree 64b030052a6f6aa15e4376e3da1845e4a96eb480.
The independent reviewer covered the full refresh and appended fixes; reviewed ancestry and unchanged paths are verified. Author checks include 2,803 Runtime + 51 model tests and the three-Home installed Mac reply proof.
Current proof, finding dispositions and remaining CI/human gates: #150.

@andersalm

Copy link
Copy Markdown
Contributor Author

Independent adversarial review (Claude Opus via elastos-pr-review) — VERDICT: PASS; head 064c572, tree 64b030052a6f6aa15e4376e3da1845e4a96eb480.
Full current source review found no blockers; diagnostics, signed disposable catalogue, failure privacy and owned-process checks pass review. CI/media/added-time and human approval remain separate acceptance gates.
Nonblocking dispositions and exact installed proof: #150.

@andersalm
andersalm marked this pull request as ready for review October 8, 2026 03:53
@andersalm
andersalm requested a review from irzhywau October 8, 2026 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant