Give AI agents a cache for repetition: verified recipes on Cloudflare, escalation for everything new.
AI agents repeatedly spend powerful-model turns rediscovering work that is already understood: inspect status, summarize a known public test result, or find public documentation. cache-layer makes that distinction visible.
The repository includes an executable pi extension, a real local recipe proof, and an honest safety benchmark. If work requires edits, private data, external effects, or new judgment, it escalates instead of pretending.
request
→ approved read-only recipe match?
yes → verify and route cheaply
no → escalate to the frontier model
The proof site deploys on Cloudflare Workers. The local semantic-router benchmark uses Ollama with public/synthetic inputs; Workers AI is the next hosted verifier experiment, not a shipped dependency of the proof site.
git clone https://github.com/acoyfellow/cache-layer
cd cache-layer
bun install
bun run devOpen the local URL printed by Wrangler to inspect the implementation proof and benchmark evidence.
Run the verification loop:
bun run verify # typecheck, tests, Worker dry-run build
bun run smoke # headless HTTP checks against a local Worker
bun run bench # reproducible deterministic routing benchmark
bun run record # record a browser proof video to recordings/This repo includes a reproducible synthetic benchmark for a narrow but important question: can a router recover approved read-only recipe hits without incorrectly handling work that must escalate?
Measured locally on an Apple M4 Pro with Ollama. Local models ran three shuffled repetitions after warm-up (138 decisions each); the deterministic gate ran five repetitions (230 decisions).
| Router | Correct | Approved hits recovered | Unsafe false hits | Median latency | p95 latency |
|---|---|---|---|---|---|
| Deterministic policy/index | 175 / 230 (76.1%) | 35 / 90 (38.9%) | 0 / 140 | < 0.1 ms | < 0.1 ms |
Ollama gpt-oss:20b |
84 / 138 (60.9%) | 0 / 54 (0.0%) | 0 / 84 | 1,254.5 ms | 2,025.9 ms |
Ollama qwen3-coder:30b |
132 / 138 (95.7%) | 54 / 54 (100.0%) | 6 / 84 | 158.8 ms | 213.0 ms |
| Production Jev + Pantry | 30 / 46 (65.2%) | 2 / 18 (11.1%) | 0 / 28 | 246.1 ms | 703.8 ms |
The production Jev run used typesafe/jev through the configured AI Gateway. Pantry supplied one approved recipe, git-status-summary, so only six of the 18 intended hits were available to the router. Jev recovered two of those six catalog-eligible hits (33.3%). A 0.90 execution threshold removed all unsafe false hits but made this version less useful than the deterministic matcher. That is a measured negative result, not evidence to replace the current default.
The evidence is useful precisely because it is not flattering: qwen3-coder:30b recovered intended hits but made unsafe choices, while conservative Jev stayed safe by escalating most requests. A model therefore cannot be the safety boundary. The architecture must keep deterministic policy in front of semantic routing.
This is routing evidence, not proof of coding quality or premium-token savings. See docs/benchmarks.md for method, failure cases, raw results, and reproduction commands. A pre-publication simulated red-team review and resolved concerns are summarized in docs/red-team.md.
The repository also contains a minimal pi extension at extensions/cache-layer/ and one executable read-only recipe: git-status-summary.
Against this public repository, a real in-memory pi AgentSession was prompted with:
summarize my git status
The extension executed git status --short --branch, produced a local result, and the session contained zero assistant/frontier-model messages:
| Prompt | Local recipe result | Frontier assistant messages | Elapsed time |
|---|---|---|---|
summarize my git status |
hit | 0 | 305.3 ms |
Raw proof: benchmarks/results/pi-extension-public-repo.json.
That proves one real narrow avoidance path. It does not yet prove net token savings across realistic workloads; prompts expected to escalate have deliberately not been paid/run until a controlled upstream baseline is designed.
Reproduce it:
bun run bench:piThe production Jev and Pantry proof requires Cloudflare Access, a local proof token in macOS Keychain, and the deployed Worker:
bun run bench:jev
bun run proof:jevRaw production evidence: benchmarks/results/production-jev-pantry-v1.json.
Click Deploy to Cloudflare above, or deploy from a checkout:
bun install
bun run deployThe app uses:
| Primitive | Purpose |
|---|---|
| Workers | Public proof site, health endpoint, and authenticated Jev route |
| Workers AI | typesafe/jev through a configured AI Gateway |
| Pantry | Approved recipe metadata through Cloudflare Access |
| Observability | Deployed Worker request visibility |
The release avoids user repository access and a public prompt box. The authenticated route accepts only the fixed public or synthetic proof corpus. Recipe execution remains local.
0.0.2 proves two narrow claims:
A real pi extension can complete one bounded read-only recipe without a frontier-model turn, while the routing benchmark demonstrates why semantic matching cannot own the safety boundary.
Example safe recipe:
id: git-status-summary
risk: read_only
handles:
- summarize my git status
- what changed locally?
proof:
- runs read-only git inspection
- modifies no files
- makes no remote calls
fallback: escalateExample escalation:
refactor the authentication architecture
→ ESCALATE
novel, write-capable, or judgment-heavy work is outside the read-only cache boundary
This project is deliberately conservative.
| Request type | 0.0.2 behavior |
|---|---|
| Public, read-only, objectively bounded workflow | May route to a verified recipe |
| Public or dummy output summarization | May route locally |
| Source edits, commits, deploys, comments, external actions | Escalate |
| Private repositories, non-public code, secrets, customer or personal data | Escalate |
| Architecture or security judgment | Escalate |
| No high-confidence match | Escalate |
The production route requires a local proof token and a short-lived Cloudflare Access assertion for Pantry. It rejects every prompt outside the committed public or synthetic proof corpus. The executable recipe lives in the local Pi extension and runs only inside the user's current repository.
Local comparisons use Ollama with public or dummy input. The production comparison uses Jev behind the same deterministic policy and selects only from the approved Pantry catalog.
This repository does not claim that all model weights or third-party local runtimes are approved by any employer or organization. Confirm your own tool, model-license, and data-handling policies before using local inference for work.
Cloudflare Worker
├── static proof site
├── /health
└── authenticated /api/route
├── PublicReadOnlyPolicy
├── Access-authenticated Pantry catalog
└── Jev through the configured AI Gateway
Pi extension
├── hosted candidate selection
├── local policy recheck
└── known local executor or frontier fallback
The deterministic policy rejects obvious writes and sensitive or private contexts before Jev and again before local execution. Jev selects only from the catalog; it never grants authority.
Returns the deployed service/version posture.
Selects an approved recipe or escalates for an exact prompt in the committed proof corpus. It requires the cache-layer proof token and a short-lived Pantry Access assertion. It is not a general or public prompt endpoint.
- Additional executable pi recipes with controlled baseline comparisons.
- Opt-in recipe persistence and local metrics.
- Local reduction of large public tool outputs before premium inference.
- Real session benchmarks measuring upstream token/cost avoidance against a baseline.
- User-approved proposal flow for turning successful repeated tasks into recipes.
Not next: silent file-edit recipes or pretending a small router should solve novel engineering tasks.
0.0.2. Public-data-only experiment. MIT.