Repository navigation
Remote server update fails on Linux when node-pty must compile (no linux-x64 prebuild; opaque error) #6012
Description
Summary
Remote/boot-service self-update can fail on Linux while preparing a pinned t3@<version> runtime because node-pty has no linux-x64 prebuild and the native rebuild fails. The desktop/UI only shows:
Server update failed: Could not prepare t3@0.0.33.
The underlying npm/node-gyp error is not surfaced, so operators have to dig through server npm debug logs and effect traces.
Environment
- Host: Ubuntu 24.04 (Linux x64)
- T3 server mode: boot-service / headless (
t3 service,T3CODE_HOMEpinned runtimes underruntime/versions/) - From version:
0.0.32 - Target version:
0.0.33 - Node: v22.23.1
- System compiler:
g++→ g++-13 (nog++-11package installed)
What happens
- Client requests remote update to exact version
0.0.33. - Server runs
ensurePinnedRuntimeInstalled→npm install --prefix <staging> --no-fund --no-audit t3@0.0.33. - Install pulls
node-pty@1.1.0. node-ptyinstall script:node scripts/prebuild.js || node-gyp rebuild.- Prebuild check: no
prebuilds/linux-x64in the published package (only darwin/win32 prebuilds present). - Rebuild fails. In our case the make step looked for
g++-11and exited 127. - npm exits 1 →
PinnedRuntimeInstallError→ UI:Could not prepare t3@0.0.33.
Relevant packaging observation on node-pty@1.1.0 tarball contents: prebuilds exist for darwin-arm64, darwin-x64, win32-arm64, win32-x64 only.
Expected
- Linux installs should not require a local C++ toolchain for a normal server update (ship
linux-x64prebuilds fornode-pty, or vendor a prebuilt binary for the supported Node ABI). - If native compile still fails, the remote update error should include the actionable cause (e.g. missing compiler / node-gyp stderr tail), not only
Could not prepare t3@X.
Workaround that unblocked us
# stage with explicit compiler
CXX=g++ CC=gcc npm install --prefix ~/.t3/runtime/versions/.staging-... --no-fund --no-audit t3@0.0.33
# then t3 service update / normal activation
# prevent recurrence on the user service
# ~/.config/systemd/user/t3code.service.d/40-native-build.conf
# [Service]
# Environment=CXX=g++
# Environment=CC=gccWhy this matters
Headless / remote Linux boxes are a first-class T3 deployment path (t3 service, Tailscale, mobile/desktop remote control). Forcing an opaque native compile on every server version bump makes remote updates fragile and hard to support.
Nice-to-haves
- Document remote-update prerequisites (Node version, compiler, disk) if native build remains required on some platforms.
- Consider caching successful
node-ptybuilds across staged versions on the same Node ABI to avoid rebuild thrash.
Happy to provide redacted npm debug log excerpts if useful.
Activity
keithce commentedon Aug 31, 2026
Confirmed another instance of this failure through
t3 triage, with a different native-build trigger and an exact actionable cause.Environment
t3code.servicesystemd user background servicet3@0.0.37-nightly.20260829.1219t3@0.0.38-nightly.20260831.123624.20.00.0.38-nightly.20260831.1236Confirmed failure
Three remote-update attempts failed identically. The UI only reported:
Server update failed: Could not prepare t3@0.0.38-nightly.20260831.1236.The server trace recorded
PinnedRuntimeInstallErrorwith npm exit code 1. Matching npm debug logs showednode-pty@1.1.0falling back tonode-gyp rebuild, after which GCC failed with:fatal error: error writing to /tmp/cc*.s: Disk quota exceededThe actionable cause was a per-user quota on
/tmp, not globally exhausted disk space:/tmp: 15.7 GiB tmpfs mounted withusrquota/tmp: approximately 12.588 GiB, apparently at the per-user limitdfstill reported approximately 2.9 GiB globally available/homehad approximately 66 GiB freeThis is easy to misdiagnose if the updater checks only filesystem-wide free space.
The current error path retains the npm exit code and stdout/stderr lengths in
PinnedRuntimeInstallError, then exposes the genericCould not prepare t3@…wrapper. The useful npm stderr never reaches the remote client.Suggested improvements
df/statfsfree space.TMPDIRunder the T3 state directory or another user-writable location with sufficient capacity.No workaround was applied during triage.
cjkihl commentedon Sep 9, 2026
Additional data point: same failure on linux-arm64 with
t3@0.0.40(stable) — so this isn't x64-only.24.21.0npx t3@latest service installsucceeded, butt3code.servicecrash-looped withNodePtyModuleLoadError: Failed to load node-pty for linux-arm64. The bundlednode-pty@1.1.0ships onlydarwin-*/win32-*prebuilds — nolinux-*at all.npx node-gyp rebuildinside~/.t3/runtime/versions/0.0.40/node_modules/node-pty, copybuild/Release/pty.node→prebuilds/linux-arm64/pty.node, restart the service. (This box had make/g++/python3; without a toolchain this path is a dead end, which is the core of this issue.)service updateneeds it redone until node-pty ships Linux prebuilds (upstream Prebuilt binary for Linux ARM is actually x86-64 microsoft/node-pty#860 → PR UI Bugs when pinging files #857, 1.2.0 line).+1 for shipping Linux prebuilds (or vendoring the binary) and for surfacing the node-gyp stderr instead of
Could not prepare t3@X.juliusmarminge commentedon Sep 14, 2026
Fixed by the self-contained CLI / release-archive runtime stack that just landed on main (notably #11316 and #11510).
Pinned runtimes are unpacked from signed release archives with prebuilt natives — no
npm install/ node-gyp compile path on remote or boot-service update anymore.