[v4.0.0] Phase 12: Robust Node/Python Subprocess Bridge #283

Overview

Overhaul the Node/Python subprocess bridge to support robust NDJSON streaming via readline, timeout watchdogs, and clean signal termination.

Part of Milestone v4.0.0 — Phase 12 of the Multi-Agent SRE Architecture.

⚠️ IMPORTANT CONTRIBUTOR & PR INSTRUCTIONS

TARGET BRANCH: All Pull Requests implementing this phase MUST target branch v4.0.0 (DO NOT target main).
PR TITLE: feat(core): Phase 12 - Robust Node/Python Subprocess Bridge
PR SCOPE: Child process lifecycle and stream parsing.

1. Description of What Has to Be Done

Node.js needs a production-grade child process runner that executes the Python agent council, streams events in real-time, traps termination signals, and protects against hung processes.

Contributors must:

1. Refactor src/agent/python-bridge.ts using node:child_process.spawn with python3 -u (unbuffered mode).
2. Connect readline.createInterface to proc.stdout so events are parsed immediately per line without waiting for process exit.
3. Setup a watchdog timer (timeoutMs, default: 45s): if exceeded, send SIGTERM, wait 1.5s, then force SIGKILL.
4. Clean up the child process immediately if the user presses Ctrl+C (process.on('SIGINT')).
5. Capture stderr lines for diagnostic reporting if the Python runner exits with a non-zero code.

2. Desired Outcome & Expected Behavior

Expected Outcome

• Real-time event streaming: The moment Python emits an NDJSON event, the TypeScript callback onEvent is invoked in < 5ms.
• Clean process lifecycle: When the user presses Ctrl+C or when the timeout expires, zero zombie Python processes remain.
• Clear error surfacing: If Python crashes due to a missing dependency (e.g. ModuleNotFoundError: No module named 'ollama'), the error is captured from stderr and reported cleanly to the user.

3. The Implementation Plan & Architectural Blueprint

Files to Create and Modify

• [MODIFY] src/agent/python-bridge.ts: Overhaul runner with readline, watchdogs, and signal handling.
• [NEW] src/__tests__/python-bridge-lifecycle.test.ts: Process lifecycle and timeout test suite.

4. Detailed Task Breakdown

• Task 12.1: Implement Streaming Readline in Bridge (python-bridge.ts)

  • Spawn python3 -u with script path and JSON input argument.
  • Pipe stdout through readline.createInterface({ input: proc.stdout }).
  • Dispatch each parsed event to options.onEvent(event).

• Task 12.2: Implement Timeout Watchdog & Signal Trapping

  • Set a timer for options.timeoutMs || 45000.
  • On expiry or SIGINT, invoke proc.kill('SIGTERM').
  • If process has not exited after 1500ms, invoke proc.kill('SIGKILL').

• Task 12.3: Lifecycle Tests (python-bridge-lifecycle.test.ts)

  • Test normal successful event stream.
  • Test timeout triggers process kill.
  • Test non-zero exit code captures stderr.

5. Technical Specifications & Concrete Code Signatures

// src/agent/python-bridge.ts
import { spawn } from 'node:child_process';
import readline from 'node:readline';
import { NDJSONEvent, ConsensusDiagnosis } from './types';

export interface CouncilBridgeOptions {
  failureText: string;
  context: { namespace?: string; kind?: string; name?: string };
  model?: string;
  timeoutMs?: number;
  onEvent?: (event: NDJSONEvent) => void;
  onStderr?: (chunk: string) => void;
}

export async function runAgentCouncilProcess(options: CouncilBridgeOptions): Promise<ConsensusDiagnosis> {
  // Spawns python3 -u, attaches readline, enforces watchdog, resolves on diagnosis_ready
  ...
}

6. Verification & Acceptance Checklist

• Run npm test -- src/__tests__/python-bridge-lifecycle.test.ts — all tests pass.
• Verified zero orphan python processes on timeout or cancellation.
• PR targets branch v4.0.0.