Skip to content

talk to subduction server directly - #54

Open
chee wants to merge 23 commits into
mainfrom
subduction-direct
Open

chee wants to merge 23 commits into
mainfrom
subduction-direct

Conversation

@chee

@chee chee commented Oct 3, 2026

Copy link
Copy Markdown
Member

@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

bench: 9d0b36c vs main

scenario main median 9d0b36c median delta main min 9d0b36c min peak RSS
🟢 clone-local files=300 size=512B text=1 1060ms 643ms -39.3% 1049ms 614ms 231 → 179 MB
⚪️ offline files=300 size=512B text=0 1134ms 1026ms -9.5% 1100ms 1017ms 170 → 134 MB
⚪️ offline files=300 size=512B text=1 1318ms 1234ms -6.4% 1307ms 1188ms 170 → 145 MB

Median of 5 run(s) per scenario, base and head interleaved on one runner. Offline paths only. Shared CI hardware is noisy, so deltas under 20% are noise and a larger one wants a local profile before it counts as a regression.

@chee
chee marked this pull request as ready for review October 4, 2026 16:13
@chee
chee requested a review from expede as a code owner October 4, 2026 16:13
Copilot AI and others added 2 commits October 4, 2026 22:44
Co-authored-by: chee <178266+chee@users.noreply.github.com>
Co-authored-by: chee <178266+chee@users.noreply.github.com>
@chee
chee requested a balanced review from Copilot October 5, 2026 10:46

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Direct-sync persistence, migration, Keyhive locking, shape recreation, and URL validation contain unresolved correctness and reliability issues.

Review effort: Balanced
Findings: 5 High severity · 6 Medium severity

Open (11)
What changed in this PR

Replaces automerge-repo with direct Subduction synchronization, introduces versioned Keyhive support, and revises storage, shapes, configuration, and testing.

Changes:

  • Adds direct document storage and synchronization through Subduction.
  • Adds Keyhive 0.5/0.6 support, access management, and server configuration.
  • Introduces config v6, revised shape APIs, migrations, and local/network tests.
File Description
vitest.config.ts Separates local and network tests.
tsconfig.json Switches to NodeNext modules.
test/​unit/​url.test.ts Tests URL encoding and validation.
test/​unit/​sync-helpers.test.ts Removes legacy sync-helper tests.
test/​unit/​storage.test.ts Tests filesystem sedimentree storage.
test/​unit/​server.test.ts Tests the local Subduction server.
test/​unit/​migrations.test.ts Removes stepwise migration tests.
test/​unit/​keyhive.test.ts Tests Keyhive archives, access, and settings.
test/​unit/​key.test.ts Tests signing-key persistence.
test/​unit/​docs.test.ts Tests direct document persistence and sync.
test/​unit/​config.test.ts Tests config v6 and legacy detection.
test/​setup.ts Removes legacy module mocking.
test/​server.ts Adds local test servers.
test/​network/​public.test.ts Tests the public sync service.
test/​network/​keyhive.test.ts Tests public Keyhive workflows.
test/​integration/​roundtrip.test.ts Moves round-trip testing to the local server.
test/​integration/​resilience.test.ts Tests unavailable and stalled servers.
test/​integration/​local.test.ts Updates offline-command coverage.
test/​integration/​fixtures/​init-clone-roundtrip.ts Removes the old compiled fixture.
test/​global-setup.ts Builds and starts an isolated test server.
test/​cli.ts Adds shared CLI test utilities.
test/​bench/​sync-starvation.test.ts Removes the starvation regression test.
src/​version.ts Simplifies dependency version discovery.
src/​url.ts Implements Automerge URL handling.
src/​storage.ts Adds sedimentree-compatible filesystem storage.
src/​shapes/​vfs.ts Migrates VFS to the new shape API.
src/​shapes/​types.ts Redefines shapes around path-to-byte maps.
src/​shapes/​patchwork-folder.ts Migrates folder shapes and artifact links.
src/​shapes/​installed.ts Adds installed-shape management.
src/​shapes/​index.ts Resolves and exports installed shapes.
src/​shapes/​file.ts Updates file-document transformations.
src/​shapes/​file-docs.ts Adds shared file-document reads and writes.
src/​output.ts Removes noninteractive confirmation defaults.
src/​migrations.ts Removes stepwise config migrations.
src/​keyhive/​common.ts Adds shared Keyhive settings and locking.
src/​keyhive/​ark-0.6.ts Adds Keyhive 0.6 integration.
src/​keyhive/​ark-0.5.ts Adds Keyhive 0.5 compatibility.
src/​keyhive.ts Selects Keyhive implementations by version.
src/​key.ts Adds persistent signing keys.
src/​index.ts Expands the public API.
src/​fs-tree.ts Renames the file-writing helper.
src/​docs.ts Implements direct document storage and sync.
src/​config.ts Introduces config v6 and migration inputs.
pnpm-workspace.yaml Updates Automerge overrides.
package.json Reworks dependencies, scripts, and package metadata.
examples/​shapes/​slay.js Adds an inline-document custom shape example.
examples/​shapes/​package.json Marks shape examples as ESM.
design/​sync.md Documents direct Subduction sync.
design/​shapes.md Documents the revised shape API.
design/​README.md Updates the architecture overview.
design/​config.md Documents config v6 migration behavior.
design/​artifacts.md Updates pinned URL documentation.
CONTRIBUTING.md Updates development and release guidance.
bench/​sync-bench.ts Updates sync benchmarks for direct Subduction.
bench/​profile.ts Simplifies benchmark profiling.
bench/​incremental-bench.ts Updates incremental benchmarks.
bench/​compare.mjs Removes backend-specific comparison keys.
.github/​workflows/​spellcheck.yml Removes the spellcheck workflow.
.changeset/​subduction-direct.md Records the major direct-sync release.
.changeset/​settle-after-send.md Records sync-settling behavior.
.changeset/​keyhive-versions.md Records dual Keyhive-version support.
.changeset/​keyhive-access.md Records Keyhive access commands.
.changeset/​fancy-flies-admire.md Records fragment recovery behavior.
Files not reviewed (1)
  • pnpm-lock.yaml: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/config.ts
/** What `pushwork migrate` needs from a pre-v6 config, or undefined if it's current. */
export async function readOldConfig(root: string): Promise<Migratable | undefined> {
const old: OldConfig = JSON.parse(await fs.readFile(path.join(root, DIR, CONFIG), "utf8"));
if (old.version === CONFIG_VERSION) return undefined;
Comment thread src/config.ts
Comment on lines +55 to +59
return {
rootUrl,
shape: old.shape ?? (original ? "patchwork-folder" : "vfs"),
artifactDirectories: old.artifactDirectories ?? old.artifact_directories,
};
Comment thread src/keyhive/ark-0.5.ts
Comment on lines +172 to +178
async close() {
if (!unlock) return;
const tmp = `${file}.${process.pid}.tmp`;
await fs.writeFile(tmp, (await queue.run(() => kh.toArchive())).toBytes(), { mode: 0o600 });
await fs.rename(tmp, file);
await unlock();
},
Comment thread src/keyhive/ark-0.6.ts
Comment on lines +171 to +177
async close() {
if (!unlock) return;
const tmp = `${file}.${process.pid}.tmp`;
await fs.writeFile(tmp, (await queue.run(() => kh.toArchive())).toBytes(), { mode: 0o600 });
await fs.rename(tmp, file);
await unlock();
},
Comment thread src/storage.ts
Comment on lines +116 to +117
const file = path.join(dir, payloadDigest(meta));
if ((await size(`${file}.meta`)) === meta.length && (await size(`${file}.blob`)) === blob.length) return;
Comment thread src/docs.ts
Comment on lines +197 to +199
entry.dirty = false;
entry.touched = true;
await this.write(id, entry.doc);
Comment thread src/shapes/file-docs.ts
isArtifact: (posixPath: string) => boolean,
): Promise<VfsNode> {
const tree = newDir();
for (const [posixPath, bytes] of files) {
Comment thread src/shapes/file-docs.ts Outdated
Comment on lines +23 to +24
} else if (!byteEq(prev!.bytes, bytes)) {
await docs.change<UnixFileEntry>(url, d => applyFileEntry(d, fresh));
Comment on lines 49 to +51
if (previousRoot) {
dlog("encode reusing root=%s", previousRoot.url);
const handle = previousRoot as DocHandle<FolderDoc>;
// Root path is "" — never an artifact dir — so the returned flag is
// ignored; callers track the repo by this bare URL.
await syncFolder(repo, handle, tree, "", isArtifact);
return handle.url;
dlog("encode reusing root=%s", previousRoot);
await syncFolder(docs, previousRoot, tree, "", isArtifact);
Comment thread src/url.ts
Comment on lines +43 to +44
heads:
heads === "" ? [] : heads.split("|").map(h => toHex(bs58check.decode(h))),
chee added 3 commits October 9, 2026 20:25
Windows can't fsync a directory, os.homedir() reads USERPROFILE there,
and @napi-rs/xattr has no win32 binding.
chee added 3 commits October 9, 2026 21:13
Pass USERPROFILE as well as HOME to child processes, match storage
paths with path.sep, and swallow the ECONNRESET the silent server sees
when the client exits.
@expede
expede added this pull request to stack #57 October 29, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants