fix(evals): read the LIFEOS results dir SuiteManager's runner writes

[pai-scaffolde]

Reproduced on a fresh LifeOS 7.40.4 install (macOS, Claude Code in the Claude desktop app); the fix was applied to that install and run there before filing.

Observed

On a case-sensitive volume, SuiteManager.ts check-saturation <suite> finds no run history for any suite, however many runs EvalRunner.ts has written. The trend is empty and ensureDirs() quietly creates a second, wrong results tree. On a default macOS install nothing looks wrong, which is why this survived.

Root cause

Two spellings of the same directory:

skills/Evals/Tools/SuiteManager.ts:16
  const RESULTS_DIR = join(EVALS_DIR, '..', '..', 'LifeOS', 'MEMORY', 'STATE', 'Evals-Results');

skills/Evals/Tools/EvalRunner.ts:31
  const RESULTS_DIR = join(CLAUDE_ROOT, 'LIFEOS', 'MEMORY', 'STATE', 'Evals-Results');

The runner writes LIFEOS; SuiteManager reads LifeOS. path.join does not normalize case, so on a case-insensitive volume the two resolve to one directory and on a case-sensitive one they are different paths, the second of which never exists. Every SuiteManager reader of RESULTS_DIR is affected — ensureDirs() (line 27) and checkSaturation()'s suiteResultsDir (line 134). ScenarioRunner.ts:111 already carries the corrected spelling with a comment naming this class:

// 'LIFEOS' exactly — 'LifeOS' only resolved on case-insensitive filesystems (#1461 class).

SuiteManager was the one file in skills/Evals left behind; it is the only remaining 'LifeOS' path segment there.

Fix

One string: 'LifeOS' becomes 'LIFEOS', plus a two-line comment recording why the case matters. No other change. The file's pre-existing unused imports (Task, basename) are left alone.

How tested

Transpiles: bun build --target=bun --no-bundle skills/Evals/Tools/SuiteManager.ts exits 0.

On a case-insensitive macOS volume both spellings resolve to the same directory, which is exactly why the defect is invisible there:

$ B=~/.claude/skills/Evals; for S in LifeOS LIFEOS; do printf '%-7s -> ' "$S"; realpath "$B/../../$S/MEMORY/STATE/Evals-Results"; done
LifeOS  -> ~/.config/LIFEOS/USER/MEMORY/STATE/Evals-Results
LIFEOS  -> ~/.config/LIFEOS/USER/MEMORY/STATE/Evals-Results

That equality holds only because the volume is case-insensitive. On a case-sensitive APFS image it does not. Probe: a disk image built with hdiutil create -size 20m -fs "Case-sensitive APFS" -volname CaseProbeY, holding the real layout (skills/Evals/Tools/, and one run at LIFEOS/MEMORY/STATE/Evals-Results/demo-suite/run_001/run.json), running checkSaturation()'s own lookup for both spellings:

LifeOS  existsSync=false  runs found=0
LIFEOS  existsSync=true   runs found=1

The old spelling loses the history; the new spelling finds it.

On the live install after the change, bun skills/Evals/Tools/SuiteManager.ts check-saturation core-dispositions reads the runner's real history: Saturated: No · Consecutive above threshold: 0/3 · Recommendation: keep.

Repro on a clean tree

Check out main on a case-sensitive filesystem, run any suite through EvalRunner.ts, then SuiteManager.ts check-saturation <suite>: it reports no history. With this patch applied it reports the run. On a case-insensitive volume the two spellings are indistinguishable and the repro requires the disk image above.

🤖 Generated with Claude Code