Repository navigation
feat(ai-harness): resume crashed turns and serve sessions to clients #1518
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
06b138d
feat(ai-harness): resume crashed turns and serve sessions to clients
AlemTuzlak 3dbd6bb
Merge branch 'feat/harness-p1-core' into feat/harness-p2-protocols
AlemTuzlak 4ace802
Merge branch 'feat/harness-p1-core' into feat/harness-p2-protocols
AlemTuzlak 0a8be10
fix(ai-harness): read AG-UI text parts in /run prompts; cover protoco…
AlemTuzlak 1c7a013
Merge branch 'feat/harness-p1-core' into feat/harness-p2-protocols
AlemTuzlak 60c3ac0
Merge branch 'feat/harness-p1-core' into feat/harness-p2-protocols
AlemTuzlak c2d3534
Merge branch 'feat/harness-p1-core' into feat/harness-p2-protocols
AlemTuzlak File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| '@tanstack/ai-harness': minor | ||
| '@tanstack/ai-harness-cli': minor | ||
| '@tanstack/ai-acp': minor | ||
| '@tanstack/ai': minor | ||
| --- | ||
|
|
||
| Harness sessions now survive a crash and talk to clients. | ||
|
|
||
| - **Resume.** A session holds a lease on each running turn, saves the transcript around each tool phase, and records tool calls that have no result yet. When a host opens a session whose turn lost its lease, it continues the turn in a new run and emits `harness.operation.resumed`. `toolDefinition({ replay: 'safe' | 'never' })` in `@tanstack/ai` decides whether an unfinished tool runs again. | ||
| - **Protocol.** `createHarnessHandler` serves capabilities, standard AG-UI runs, the session event stream with cursors, control inputs with receipts, and snapshots. `authorize` is required. `handleHarnessSocket` serves the same session tier over a WebSocket. `createHarnessClient` in `@tanstack/ai-harness/client` is a typed, reconnecting client. | ||
| - **`harnessText`** runs a harness as the text adapter of another `chat()` call. | ||
| - **ACP v2.** `@tanstack/ai-acp/agent` adds `createAcpAgent` and `serveAcp`, which serve a harness as an ACP v2 agent (experimental, like the draft protocol). `@tanstack/ai-acp` now uses `@agentclientprotocol/sdk` 1.5. The existing ACP client adapters keep working. | ||
| - **New package `@tanstack/ai-harness-cli`.** `runCli(harness)` gives a harness an Ink terminal UI, a print mode with exit codes, NDJSON output, `--acp`, and `--serve`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| --- | ||
| title: Run a harness in the terminal | ||
| id: harness-cli | ||
| order: 3 | ||
| description: "Give your harness a terminal UI, a print mode for scripts and CI, NDJSON output, an ACP mode for editors, and an HTTP server." | ||
| keywords: | ||
| - tanstack ai | ||
| - harness | ||
| - cli | ||
| - terminal | ||
| - ink | ||
| --- | ||
|
|
||
| You built a harness and want to use it like Claude Code: type in a terminal, watch it work, approve tools. You also want to run it in CI. `runCli` gives one harness all of these modes. | ||
|
|
||
| ## Install | ||
|
|
||
| <!-- ::start:tabs variant="package-manager" mode="install" --> | ||
|
|
||
| react: @tanstack/ai-harness-cli | ||
| vue: @tanstack/ai-harness-cli | ||
| solid: @tanstack/ai-harness-cli | ||
| svelte: @tanstack/ai-harness-cli | ||
| preact: @tanstack/ai-harness-cli | ||
| angular: @tanstack/ai-harness-cli | ||
| vanilla: @tanstack/ai-harness-cli | ||
| octane: @tanstack/ai-harness-cli | ||
|
|
||
| <!-- ::end:tabs --> | ||
|
|
||
| The interactive UI uses Ink, which needs Node 22 or later. | ||
|
|
||
| ## 1. Write the entry file | ||
|
|
||
| ```ts group=harness-cli | ||
| import { defineHarness } from '@tanstack/ai-harness' | ||
| import { runCli } from '@tanstack/ai-harness-cli' | ||
| import { openaiText } from '@tanstack/ai-openai' | ||
|
|
||
| const assistant = defineHarness({ | ||
| name: 'acme/assistant', | ||
| adapter: openaiText('gpt-5.6'), | ||
| }) | ||
|
|
||
| process.exitCode = await runCli(assistant) | ||
| ``` | ||
|
|
||
| ## 2. Pick a mode | ||
|
|
||
| - No flags: the interactive UI. Type a message and press Enter. While the agent works, Enter steers it and Esc cancels. | ||
| - `-p "prompt"`: run one prompt, print the answer, and exit. | ||
| - `-p "prompt" --output ndjson`: print every AG-UI event as one JSON line. | ||
| - `--acp`: serve the harness as an ACP v2 agent over stdio, for editors. Needs `@tanstack/ai-acp`. | ||
| - `--serve`: serve the session protocol over HTTP on `127.0.0.1:8787`. Every request needs the bearer token. Pass `--token`, set `HARNESS_TOKEN`, or copy the token the CLI prints. | ||
|
|
||
| When stdin is a pipe, the CLI reads one message or command per line and waits for each turn. | ||
|
|
||
| ## 3. Use it in CI | ||
|
|
||
| `-p` exits with a code your script can check: | ||
|
|
||
| | Code | Meaning | | ||
| |---|---| | ||
| | 0 | The turn finished. | | ||
| | 1 | The turn failed. | | ||
| | 2 | The turn waits for an approval. | | ||
| | 130 | The turn was cancelled. | | ||
|
|
||
| ## Commands in the interactive UI | ||
|
|
||
| - `/agents`: list the agents. | ||
| - `/agent <name> {"json":"input"}`: run an agent in the background. When it is done, a new turn starts with its result. | ||
| - `/cancel`: cancel the running turn. | ||
| - `/status`: show what runs and what waits. | ||
| - `/exit`: quit. | ||
|
|
||
| When a turn stops for an approval, type `y` to approve or `n` to reject. | ||
|
|
||
| ## What you have now | ||
|
|
||
| - One entry file that runs your harness as a terminal app, a script step, an editor agent, or a server. | ||
|
|
||
| Next: keep long turns alive through crashes with [durable sessions](./durable-sessions). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| --- | ||
| title: Connect clients to a harness | ||
| id: harness-connect | ||
| order: 2 | ||
| description: "Serve a harness session over HTTP, SSE, or WebSocket. Talk to it from a web app, an editor over ACP, or another chat() call." | ||
| keywords: | ||
| - tanstack ai | ||
| - harness | ||
| - AG-UI | ||
| - ACP | ||
| - websocket | ||
| - harnessText | ||
| --- | ||
|
|
||
| Your harness runs on a server, but the people who use it are in a browser, an editor, or another agent. `createHarnessHandler` serves a session over HTTP. A web app talks to it with `createHarnessClient`, an editor with ACP, and another `chat()` call with `harnessText`. | ||
|
|
||
| ## Serve the session over HTTP | ||
|
|
||
| Mount one fetch handler on a route. `authorize` is required, so no endpoint is open by accident. | ||
|
|
||
| ```ts group=harness-connect | ||
| import { defineHarness, createHarnessHandler, createHarnessHost } from '@tanstack/ai-harness' | ||
| import { memoryPersistence } from '@tanstack/ai-persistence' | ||
| import { openaiText } from '@tanstack/ai-openai' | ||
|
|
||
| export const assistant = defineHarness({ | ||
| name: 'acme/assistant', | ||
| adapter: openaiText('gpt-5.6'), | ||
| }) | ||
|
|
||
| const host = createHarnessHost({ persistence: memoryPersistence() }) | ||
|
|
||
| export const handler = createHarnessHandler({ | ||
| host, | ||
| harness: assistant, | ||
| authorize: (request) => | ||
| request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` | ||
| ? { id: 'user-1' } | ||
| : null, | ||
| canAccess: (principal, threadId) => threadId.startsWith(principal.id), | ||
| }) | ||
| ``` | ||
|
|
||
| The handler answers these paths under your route: | ||
|
|
||
| - `GET capabilities`: the AG-UI capabilities, with the agents in `expose.agents`. | ||
| - `POST run`: standard AG-UI. One request runs one prompt and streams it as SSE. Any AG-UI client works. | ||
| - `GET events?threadId=`: every event of the session as SSE. Each event id is a cursor, so a reconnect with `Last-Event-ID` continues where it stopped. | ||
| - `POST control`: send `{ threadId, input }`, for example `{ op: 'prompt', message }`. You get a receipt back. | ||
| - `GET snapshot?threadId=`: the status, running operations, and waiting approvals. | ||
|
|
||
| ## Talk to it from a web app | ||
|
|
||
| `createHarnessClient` wraps those endpoints. Import the harness with `import type`, so no server code reaches the browser. | ||
|
|
||
| ```ts group=harness-connect-client | ||
| import { createHarnessClient } from '@tanstack/ai-harness/client' | ||
| import type { assistant } from './harness' | ||
|
|
||
| const client = createHarnessClient<typeof assistant>({ | ||
| url: '/api/harness', | ||
| threadId: 'user-1-thread', | ||
| headers: { authorization: 'Bearer my-token' }, | ||
| }) | ||
|
|
||
| await client.prompt('Summarize my open tickets.') | ||
|
|
||
| for await (const entry of client.events()) { | ||
| if (entry.event.type === 'TEXT_MESSAGE_CONTENT') { | ||
| console.log(entry.event.delta) | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| `events()` reconnects after a network error and continues from the last cursor. A second tab, a phone, or a reload all see the same session. | ||
|
|
||
| ## Use a WebSocket | ||
|
|
||
| For one connection that carries events and inputs, authorize the upgrade, then hand the socket to `handleHarnessSocket`: | ||
|
|
||
| ```ts group=harness-connect | ||
| import { handleHarnessSocket } from '@tanstack/ai-harness' | ||
| import type { WebSocketLike } from '@tanstack/ai' | ||
|
|
||
| export function onUpgrade(socket: WebSocketLike) { | ||
| handleHarnessSocket({ host, harness: assistant, socket, principal: { id: 'user-1' } }) | ||
| } | ||
| ``` | ||
|
|
||
| The client sends `{ type: 'harness.subscribe', threadId }` first. After that, each `harness.input` frame gets a `harness.receipt`, and every event arrives as a `harness.event` frame with its cursor. | ||
|
|
||
| ## Use it from an editor (ACP) | ||
|
|
||
| Editors such as Zed start agents as a process and talk ACP over stdio. `serveAcp` makes the harness an ACP v2 agent: | ||
|
|
||
| ```ts group=harness-connect | ||
| import { serveAcp } from '@tanstack/ai-acp/agent' | ||
|
|
||
| serveAcp({ host, harness: assistant }) | ||
| ``` | ||
|
|
||
| Tool approvals become permission requests in the editor. ACP v2 is still a draft, so this API is experimental. | ||
|
|
||
| ## Use it as the model of another chat | ||
|
|
||
| `harnessText` turns a harness into a text adapter. The outer chat sends a message, and the harness runs a full turn with its own tools, plugins, and agents: | ||
|
|
||
| ```ts group=harness-connect | ||
| import { chat } from '@tanstack/ai' | ||
| import { harnessText } from '@tanstack/ai-harness' | ||
|
|
||
| const stream = chat({ | ||
| adapter: harnessText(assistant, { host }), | ||
| messages: [{ role: 'user', content: 'Fix the failing test.' }], | ||
| threadId: 'outer-thread', | ||
| }) | ||
| ``` | ||
|
|
||
| Each outer thread gets its own inner session, so the harness keeps its own history. | ||
|
|
||
| ## What you have now | ||
|
|
||
| - One handler that serves standard AG-UI and the session stream. | ||
| - A typed web client that reconnects and resumes from a cursor. | ||
| - A WebSocket, an ACP agent for editors, and a harness you can call from `chat()`. | ||
|
|
||
| Next: run the same harness in a terminal with the [CLI](./cli). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| --- | ||
| title: Durable sessions | ||
| id: harness-durable-sessions | ||
| order: 4 | ||
| description: "Keep a harness turn alive through a crash or a deploy. The next host continues it from the last checkpoint." | ||
| keywords: | ||
| - tanstack ai | ||
| - harness | ||
| - durability | ||
| - resume | ||
| - checkpoint | ||
| --- | ||
|
|
||
| A long agent turn can outlive the process that started it: a deploy restarts the server, or the machine crashes in the middle of a tool call. With durable stores, the next host that opens the session continues the turn from its last checkpoint. | ||
|
|
||
| ## Give the host durable stores | ||
|
|
||
| Use your own stores built with `defineAIPersistence`. Resume needs these: | ||
|
|
||
| - `messages`: the transcript. | ||
| - `runs`: the run records, with leases and checkpoints. Implement `listByThread`. | ||
| - `inbox`: messages that were accepted but did not run yet. | ||
|
|
||
| ```ts group=harness-durable | ||
| import { createHarnessHost } from '@tanstack/ai-harness' | ||
| import { memoryPersistence } from '@tanstack/ai-persistence' | ||
|
|
||
| // Swap in your own stores. Memory stores do not survive a restart. | ||
| const host = createHarnessHost({ persistence: memoryPersistence() }) | ||
| ``` | ||
|
|
||
| ## What the session saves while a turn runs | ||
|
|
||
| - A lease on the run record. The host renews it every 10 seconds. It expires after 30 seconds. | ||
| - The transcript before and after each tool phase. | ||
| - Each tool call that started and has no result yet. | ||
|
|
||
| ## What happens after a crash | ||
|
|
||
| When a host opens the session and finds a turn whose lease expired, it continues that turn in a new run: | ||
|
|
||
| 1. The old run is marked failed. | ||
| 2. Tool calls without a result get handled by their `replay` setting. | ||
| 3. A new chat run starts from the saved transcript. | ||
| 4. Clients see a `harness.operation.resumed` event, then the new events. | ||
|
|
||
| ## Mark tools that are safe to run again | ||
|
|
||
| A crash can stop a run after a tool started but before its result was saved. Tell the harness what to do with `replay`: | ||
|
|
||
| ```ts group=harness-durable | ||
| import { toolDefinition } from '@tanstack/ai' | ||
| import { z } from 'zod' | ||
|
|
||
| const lookup = toolDefinition({ | ||
| name: 'lookup_order', | ||
| description: 'Read an order', | ||
| inputSchema: z.object({ id: z.string() }), | ||
| replay: 'safe', | ||
| }).server(async ({ id }) => ({ id, status: 'shipped' })) | ||
| ``` | ||
|
|
||
| - `replay: 'safe'`: the tool runs again. Use it for reads and other idempotent calls. | ||
| - `replay: 'never'` (default): the model gets a note that the tool may or may not have run, and checks before it tries again. | ||
|
|
||
| A model response that was cut in the middle runs again after resume. The provider can bill that call twice. | ||
|
|
||
| ## What you have now | ||
|
|
||
| - Turns that continue after a crash or a deploy. | ||
| - Tools that either run again safely or tell the model to check first. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🛡️ Detected with Advanced Tier | 🟡 Minor | ⚡ Quick win
Authorization Bypass
Exploitability: Moderate
CWE: CWE-863 — Incorrect Authorization
Harden the
authorizeandcanAccessexample. Readers will copy it.The sample has two flaws:
HARNESS_TOKENis unset, the template literal becomesBearer undefined. A request that sendsAuthorization: Bearer undefinedis then authorized asuser-1.threadId.startsWith(principal.id)is a prefix check. Principaluser-1can openuser-10-threadanduser-12-.... This breaks the per-user thread isolation thatcanAccessshould provide (HarnessHandlerOptions.canAccessinpackages/ai-harness/src/http.ts).🔒 Proposed doc fix
Update the client example's
threadId: 'user-1-thread'to match, for example'user-1:thread'.📝 Committable suggestion
View in Security blast radius
🤖 Prompt for AI Agents