Skip to content

fix: preserve malformed tool arguments across provider wires - #501

Merged
jpr5 merged 4 commits into
mainfrom
fix/misbehavior-pr0-arguments
Oct 8, 2026
Merged

fix: preserve malformed tool arguments across provider wires#501
jpr5 merged 4 commits into
mainfrom
fix/misbehavior-pr0-arguments

Conversation

@jpr5

@jpr5 jpr5 commented Oct 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Malformed fixture tool arguments were silently replaced with {}, hiding the failure an application was supposed to handle. This PR preserves authored malformed text on string-based wires and returns an explicit error before content on parsed-value wires.

  • Covers Anthropic Messages, Bedrock InvokeModel/Converse, Cohere, Gemini, Gemini Interactions, Gemini Live, and Ollama, including tool-only, mixed, and ordered-block paths.
  • Preserves existing valid JSON behavior, including arrays/primitives/null and empty or omitted legacy defaults. Fixture validation is unchanged.
  • Returns provider-shaped HTTP500 errors or a Gemini Live WebSocket error, records the diagnostic in the journal, and preserves the Live connection for recovery. Bedrock uses InternalServerException and a top-level message so the official AWS SDK exposes the actionable diagnostic.
  • Updates fixture documentation and the Unreleased changelog. Scope is PR0 argument integrity; no fault engine, package version bump, or release.

Local RED → GREEN proof

Tests use real local aimock HTTP/WebSocket endpoints and installed provider SDKs. The original three-cell reproduction covers Anthropic streaming, Cohere string arguments, and Gemini parsed arguments with six valid/empty controls. Production baseline: f5e17a6f.

pnpm exec vitest run src/__tests__/pr0-tool-arguments-http.test.ts --silent=false

Actual output excerpts from the frozen reproduction and integrated rerun:

# RED: unchanged baseline production; malformed arguments became {}.
 Test Files  1 failed (1)
      Tests  3 failed | 6 passed (9)

# GREEN: same nine assertions; all three malformed cells now flip.
 ✓ src/__tests__/pr0-tool-arguments-http.test.ts (9 tests) 37ms

Review also found that the new Bedrock error's raw HTTP diagnostic was lost by the AWS SDK. The following six real SDK cases were run before and after correcting the envelope: InvokeModel and Converse, each with tool-only, mixed, and ordered-block fixtures.

pnpm exec vitest run src/__tests__/pr0-bedrock.test.ts src/__tests__/pr0-converse.test.ts --testNamePattern='(stream=false malformed|malformed.*stream=false)' --maxWorkers=1 --no-file-parallelism --silent=false
# RED: SDK name/message were Unknown / UnknownError.
 Test Files  2 failed (2)
      Tests  6 failed | 102 skipped (108)

# GREEN: same cases assert InternalServerException and the full diagnostic.
 Test Files  2 passed (2)
      Tests  6 passed | 102 skipped (108)

Independent confirmation at e39cf4f2 observed the actual SDK exception:

name: InternalServerException
message: aimock: fixture tool call "lookup" has invalid JSON arguments; this wire carries arguments as an object. Use a wire that carries tool arguments as a string to test malformed JSON. (Unexpected end of JSON input)
status: 500
journalStatus: 500

The final test-only commit updates one legacy Converse HTTP regression that still expected the removed {} fallback. Production source is unchanged from the reviewed e39cf4f2 tree. This is a stale assertion correction, not an additional runtime defect:

pnpm exec vitest run src/__tests__/bedrock-stream.test.ts
# RED before correcting the obsolete success expectation:
 Test Files  1 failed (1)
      Tests  1 failed | 84 passed (85)

# GREEN after asserting HTTP500 and the AWS-native diagnostic:
 Test Files  1 passed (1)
      Tests  85 passed (85)

Final local quality at 95d98297 passed: 306 files, 10,190 tests, one intentional external-spec opt-in skip, plus all 439 PR0 proofs. This total combines the unchanged full-suite results with focused reruns after building missing CLI artifacts; it is not a claim of one all-green full-suite invocation. Format, lint, source/test/scripts typechecks, build and frozen-lockfile installation passed. Seven independent confirmation reader roles and the executable consumer/mutation gate report zero remaining mandatory findings; the final test-only correction was independently confirmed.

Known limitation

Bedrock InvokeModel streaming has a pre-existing AWS PayloadPart envelope defect: the official SDK receives chunk: {} even for valid streams. This PR preserves malformed arguments using captured real HTTP eventstream frames and decoded payloads; it does not claim SDK streaming compatibility. That envelope correction remains a separate cohort backlog item. Non-streaming Bedrock and Converse still use official SDK proof.

Merge and release behavior

Versions remain @copilotkit/aimock@1.44.0 and aimock-pytest@0.6.0; both were confirmed already published. A main-branch merge triggers the existing Release workflow, whose already-published checks normally skip numbered package publication and release/tag creation. Docker PR builds do not push images, and ordinary main pushes do not trigger Docker publication. Existing automation may publish a PR preview artifact and update GitHub Pages documentation; no new numbered package version is introduced here.

Validation

  • Seven independent confirmation reader roles: zero mandatory findings at e39cf4f2; unchanged production source reviews remain valid at 95d98297.
  • Final test-only assertion correction independently confirmed; executable boundary has zero remaining mandatory findings.
  • Fixture documentation and Unreleased updated; versions unchanged.
  • All 439 provider/helper/boundary proof tests pass at the final code tree.
  • Real consumer checks and scratch mutation checks completed; unchanged proof reused.
  • Format, lint, typecheck, full test coverage and build pass; frozen lockfile installation passes. Public entrypoints are unchanged; new helper/error remain internal.
  • Four commits grouped by concern preserve the exact tested tree; local completion checklist PASS at a014ada2.
  • All 30 CI checks passed at a014ada2, including tests, lint, pytest, and Docker build.

Evidence retained locally under ~/.local/share/copilotkit/cr/aimock-misbehavior-pr0-20261007: red.log, integrated-partial-green.log, integrated-green.log, slots/http-boundary/aws-diagnostic-{red,green}.log, and round-2-slot-*-1.md. The output excerpts above carry the required proof in this PR body without depending on those local paths.

Merge status

CI is green. Auto-merge using a merge commit is enabled; the PR remains open and awaits the repository's required review approval. It has not merged.

@pkg-pr-new

pkg-pr-new Bot commented Oct 7, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@501

commit: a014ada

@jpr5
jpr5 enabled auto-merge October 8, 2026 00:01
@jpr5
jpr5 merged commit b02cb2d into main Oct 8, 2026
30 checks passed
@jpr5
jpr5 deleted the fix/misbehavior-pr0-arguments branch October 8, 2026 00:11
This was referenced Oct 9, 2026
Merged
Merged
jpr5 added a commit that referenced this pull request Oct 10, 2026
@jpr5
fix: Mastra-parity follow-ups — restore 1.44.0 types and harden MCP r…
f7f8470 
…ecording (#517)

## Why

#509 (MCP fakes recording) changed the published TypeScript surface of
`@copilotkit/aimock@1.44.0`. aimock ships no breaking changes, type-only
ones included, so this PR restores each break additively. The items use
the labels from the 1.44.0 compatibility audit:

- **S1.** The `declare module "./fixture-loader.js"` augmentation in
`src/mcp-fakes.ts` was emitted by tsdown as a literal `sideEffect();`
statement in `dist/mcp-fakes.d.ts` and `.d.cts`. Every consumer that
type-checks with `skipLibCheck: false` failed on every entry point
(TS1036, TS2304). The augmentation is gone. A malformed recorded key now
fails with the 1.44.0 rule `mcp-fakes/bad-block:h`, and the message
names its `record-replay/fa-*` rule.
- **B8.** `McpFakeAddOrigin.kind` had gained `"record"`, which breaks a
custom `Mountable.addMcpFakes` that switches exhaustively on it. The
public type is back to `"file" | "code" | "control-api"`. The recorder
uses an internal origin type and a private `MCPMock` method.
- **B9.** `AimockHandle` in `/vitest` and `/jest` had gained required
`fakesFor()` and `fakesReport()`, which breaks code that constructs or
stubs a handle. `AimockHandle` is restored exactly. `useAimock()` now
returns a new `AimockFakesHandle` that extends it, and both types are
exported from `/vitest` and `/jest`.
- **B11.** `Mountable` had gained `setReplaySpeed?(speed: number):
void`, so a 1.44.0 mount with its own `setReplaySpeed` member of another
type no longer assigned. `Mountable` is back to its exact 1.44.0 shape.
- **R12.** The runtime half of B11: a custom mount with a
`setReplaySpeed` member was called at start, and a non-function member
threw a `TypeError` at `start()`. The server and `LLMock` mount loops
now give the replay speed only to aimock's own `MCPMock`.
- **R13.** A bad recorded key reported `record-replay/fa-*` as the error
rule instead of the 1.44.0 `mcp-fakes/bad-block:h`, and a bad recorded
key together with another error reported a different first rule. The
recorded keys are now checked where 1.44.0 rejected them as unknown
keys, so the first rule matches 1.44.0.

A new gate, `src/__tests__/package-types-compat.test.ts`, runs `npm
pack` on the build, unpacks it into a temporary consumer, and
type-checks every `exports` entry point (ESM and CJS) under `strict` and
`skipLibCheck: false`. The consumer holds the 1.44.0 usage patterns for
B8, B9 and B11.

## The hardening fixes

These fix defects in the MCP recording feature that the review of the
Mastra-parity work found. All of it is new since 1.44.0.

- **Start-time validation and credential redaction.** `llm.record.mcp`
and the record destination are checked in a pre-pass at start, before
anything is mounted. The start fails with an error that names the mount
and the destination for: `null`, `false`, `0` or `""` as
`llm.record.mcp`; mount keys the server can never route (a trailing `/`,
`//`, a query, a fragment, whitespace, a dot segment, an escaped
character), on both `llm.record.mcp` and `--mcp-record` /
`--mcp-proxy-only`; an upstream that is not an http(s) URL; an empty or
non-string `fixturePath`; a broken symbolic link, `EACCES` or a
non-writable directory as the destination; and `maxRecordBufferBytes`
that is not greater than 0 and at most 256 MiB. Credentials in mount
keys are redacted in these errors with the record sanitizer's URL rule.
- **Upstream death mid-body fails loudly on every relay path.** Before,
a truncated upstream answer was relayed as a clean-looking 200, a
`SyntaxError` in the client, or a hang until the client timed out, with
nothing logged. Now the buffered, SSE, pass-through and post-overflow
paths all fail: a 502 before headers, a terminated stream after them,
and a JSON-RPC error event first for an SSE call that never got its
response. Each failure logs one `MCP-RECORD:` line and journals
`recordSkipped: "upstream-aborted"`. A client that disconnects is not
reported as an upstream failure.
- **Config warnings.** `aimock --config` defaulted to the silent log
level, which hid the `MCP-RECORD:` warnings, so a run that recorded
nothing printed nothing. A config whose `llm.record.mcp` has at least
one mount now defaults `logLevel` to `"warn"`. An explicit
`llm.logLevel` still wins, and an empty `llm.record.mcp` stays silent.

## Red/green evidence

Each fix was reproduced on the real surface (the built CLIs, a packed
tarball, real MCP clients `@modelcontextprotocol/sdk` 1.31.0 and
`@modelcontextprotocol/client` 2.2.0) before the change and re-run after
it.

- **S1, B8, B9.** RED: the compat gate's consumer against the base build
failed with `mcp-fakes.d.ts(13,1): error TS1036: Statements are not
allowed in ambient contexts` and `TS2304: Cannot find name 'sideEffect'`
(both `.d.ts` and `.d.cts`), `TS2345: Argument of type '"record"' is not
assignable to parameter of type 'never'`, and `TS2739: ... missing the
following properties from type 'AimockHandle': fakesFor, fakesReport`.
GREEN: the gate passes 2/2. In the packed `dist/`, `sideEffect` and
`declare module` match 0 of 86 `.d.ts` and 86 `.d.cts` files.
- **B11.** RED: a strict consumer with a `Mountable` whose
`setReplaySpeed` is a string fails with `TS2322 ... Types of property
'setReplaySpeed' are incompatible`; npm 1.44.0 compiles it. GREEN: it
compiles, and the `interface Mountable` block in `dist/types.d.ts` and
`.d.cts` is byte-identical to npm 1.44.0.
- **R12.** RED: `LLMock({ replaySpeed: 2 })` with such a mount threw
`TypeError: handler.setReplaySpeed is not a function` at `start()`,
before and after a late mount. GREEN: it starts and the member is never
called, the same as 1.44.0.
- **R13.** RED: a malformed recorded key gave
`"rule":"record-replay/fa-list"`; a bad recorded key with `scope: 5`
gave `bad-block:b` where 1.44.0 gives `bad-block:h`. GREEN:
`"rule":"mcp-fakes/bad-block:h"` with `[record-replay/fa-list]` in the
message. A probe of 722 inputs that 1.44.0 rejects, each loaded both
through `addMcpFakes` and from a file, went from 238 rule mismatches to
0. A separate class of 66 rows, with a recorded key that is valid now,
is listed under the follow-ups below.
- **Start-time validation.** RED: `llm.record.mcp` of `null`, `false`,
`0` and `""`, the mount keys `/mcp/`, `/mcp?x=1`, `/a b` and `//`, an
empty `fixturePath`, a dangling symlink destination and
`maxRecordBufferBytes` of 0, -5 and 314572800 all started (exit 0). A
credentialed URL as a mount key printed
`http://user:tok3nSECRET@host/mcp` in the error. GREEN: each exits 1
with a message that names the mount; the credentialed key prints
`http://host/mcp (credentials removed)`, and the full stderr has 0
secret matches. Valid configs still start.
- **Upstream death mid-body.** RED (killable upstream): SSE record path,
both clients `TIMEOUT 4000ms`; pass-through chunked JSON, a raw 200 with
a truncated body and `SyntaxError: Unterminated string in JSON`;
pass-through content-length, a hang; buffered JSON, `Unexpected content
type: null`; nothing logged. GREEN: SSE gives `McpError -32603 ... the
upstream response failed` at once; the pass-through and post-overflow
paths give `TypeError terminated` at once; buffered JSON gives a 502
with `{"error":"aimock MCP recorder: the upstream response failed"}`.
Each logs `MCP-RECORD: the upstream response failed mid-body
(upstream-aborted) ...` and journals `upstream-aborted`. A client abort
produced no aimock output.
- **Config warnings.** RED: `aimock --config` with an MCP record mount
and a request with no testId printed only the listening line. GREEN: it
prints `MCP-RECORD: forwarded, not recorded: a request on mount /mcp has
no testId or context ...`. `mcp: {}` stays silent, like a config with no
`mcp`.
- The happy path still works: `examples/mcp-record-proof/proof.sh` exits
0, recording server-everything (13 tools, 0 secrets) and replaying it
offline at speed 1 and 100, identical to the recording and to the live
run.

## Not breaking

Every runtime change touches only features added since 1.44.0: MCP
recording (`llm.record.mcp`, `--mcp-record`, `--mcp-proxy-only`), the
recorded `mcpFakes` keys, and the replay-speed hook. The type changes
return the published surface to 1.44.0 and add only new names
(`AimockFakesHandle`).

## Coordination

Another session is fixing the #501, #508, #511 and #513 breaks
separately and will reuse this compat gate. The gate's evidence consumer
still shows their errors (`c-others.ts`, `c508.ts`) and none from #509.
Release PR #516 is held until both land.

## Known follow-ups

- 66 probe rows remain where a recorded key that is now valid, together
with an unrelated error, gives a different first rule than 1.44.0 (for
example `timing` plus `scope: 5` gives `bad-block:b`, not
`bad-block:h`). Both versions reject the input; only the rule code
differs. This waits on decision D4.
- Programmatic `wireMcpRecording` can leave mounts half-wired if a later
step fails.
- `@modelcontextprotocol/client` 2.2.0 maps the 502 to
`CLIENT_HTTP_NOT_IMPLEMENTED` (client side).
- The pass-through SSE and SSE-after-overflow paths terminate the stream
but do not inject a JSON-RPC error event.
- Some test-strength mutants beyond what the new tests cover, and a few
pre-existing log-wording nits.
- `AimockRecordConfig` requires `providers`, but the runtime and docs
accept an `llm.record` with only `mcp`.
- The aimock-cli docs say the `"warn"` default applies when
`llm.record.mcp` is set; the code applies it only when it has at least
one mount.

## Test plan

At the head of this branch, each exit code captured on its own:
`prettier --check .`, `eslint .`, `pnpm typecheck` (three tsconfigs),
`pnpm build`, `package-types-compat.test.ts`, and the recorder,
record-CLI, aimock-cli and MCP fakes test files. The full suite's only
failures are the pre-existing vitest worker `Timeout calling
"onTaskUpdate"` error (it reproduces on the base) and a `validate-cli`
timeout under load that passes alone (96/96); neither file is touched
here.
@jpr5 jpr5 mentioned this pull request Oct 10, 2026
Merged
jpr5 added a commit that referenced this pull request Oct 10, 2026
@jpr5
fix: restore 1.44.0 default behavior; make new Responses/misbehavior/…
c9c19d1 
…strict-args behavior opt-in (#518)

## Summary

With default options, this branch has zero breaking changes compared
with the published `@copilotkit/aimock` 1.44.0. The rule is:

1. An input that 1.44.0 accepted behaves identically.
2. An input that 1.44.0 rejected may now be accepted.
3. New behavior is opt-in.

The changes from #501, #508, #511 and #513 that altered 1.44.0 behavior
now sit behind explicit opt-ins. The 1.44.0 types (`ToolCall`,
`FixtureToolCall`, block unions, `CollapseResult`,
`resolveFixtureBlocks`, WebSocket handler arity,
`FixtureLoadRuleRegistry`) are restored. The new carriers
(`customToolCalls`, `responsesBlocks`, `MisbehaviorConfigError`,
`StartFromConfigOverrides`) are additive.

## Opt-ins

| Feature | Programmatic | CLI flag (`llmock` / `aimock`) | `aimock
--config` key |
|---|---|---|---|
| Extended Responses tool handling (#511/#513): custom tool calls,
namespaces. The fixture keys `match.toolNamespace`, `customToolCalls`
and `responsesBlocks` are honored **only** under `"extended"`; in
`"legacy"` they are ignored, as in 1.44.0 | `responsesTools: "legacy" \|
"extended"` (default `"legacy"`; an invalid value warns and runs
`"legacy"`); aimock-pytest: `AIMockServer(responses_tools="extended")` |
`--responses-tools legacy\|extended`; `aimock validate --responses-tools
legacy\|extended` | `llm.responsesTools` (an invalid value warns and
runs `"legacy"`) |
| Strict invalid tool arguments (#501): 500
`aimock_invalid_tool_arguments` on object wires, authored text on string
wires | `strictToolArguments: true` | `--strict-tool-arguments` |
`llm.strictToolArguments: true` |
| Model misbehavior (#508) | `enableMisbehavior: true` (also passed
through the plugins' `useAimock`) | `--misbehavior` |
`llm.enableMisbehavior: true` (`llm.misbehavior` is read only when
misbehavior is enabled) |
| MCP recording from config | n/a (`MCPMock.enableRecording()` is
unchanged) | n/a (`llmock --mcp-record` / `--mcp-proxy-only` are
unchanged) | `llm.enableMcpRecording: true` (`llm.record.mcp` is read
only with this opt-in; without it the key is ignored with one warning,
as in 1.44.0) |

On the `aimock` bin, a flag overrides the matching config key. If a
boolean flag is absent, the config key applies. Without its opt-in, each
setting keeps the 1.44.0 behavior. For example, invalid fixture tool
arguments are served as `{}` with a warning, and a `misbehavior` key in
a fixture or config is ignored (the config case logs one warning).

## Proof

All comparisons use the packed build of this branch against the npm
1.44.0 tarball.

| Proof | Result |
|---|---|
| Strict consumer matrix: TS 5.0.4 / 5.4.5 / 5.9.3 × bundler /
node16-esm / node16-cjs × 7 entry points (root, mcp, a2a, vector, agui,
vitest, jest), `skipLibCheck: false` | **126/126 cells pass** (exit 0, 0
errors) for both 1.44.0 and this branch |
| Type proofs b01–b12, s01, b07, b10-ws-handler-arity: TS 5.0.4 / 5.4.5
/ 5.9.3 × nodenext / node16-esm / node16-cjs / bundler / node10, strict
| 0 src errors in every cell, **except b12** (see Known open item).
node10 has the same 4 TS1192 lib errors as 1.44.0 |
| `src/__tests__/package-types-compat.test.ts` (strict gate) | 2/2 pass
|
| Runtime A/B, default options: http-ab (all wires × fixtures, stream
and non-stream) | 704 same / 0 diff |
| ws-ab / record-ab / record-ns-ab2 | 28/0, 60/0, 6/0 |
| #501 ab501, 1.44.0 vs default (wire bytes, journal, warn count) |
260/0 |
| #501 ab501, pre-fix main vs `strictToolArguments: true` | 260/0 on
bytes and journal (with strict, Ollama, Interactions and Live also log
the 1.44.0 warning) |
| #508 enabled-ab, main vs `enableMisbehavior: true` | 39/0 |
| Codex CLI 0.161.0 against the packed `dist/cli.js` with
`--responses-tools extended` | **4/4**: custom `apply_patch` over HTTP
and over WS, namespaced MCP over HTTP and over WS. Contrast without the
flag: 404 "No fixture matched", as in 1.44.0 |

There are 3 documented deltas from 1.44.0 with default options. All 3
are allowed by the rule:

1. **`GET /__aimock/misbehavior`**: 1.44.0 returns 404, this branch
returns 200. This is a new additive route (rule 2).
2. **`aimock --config` with a valid or invalid `llm.misbehavior` and no
opt-in**: the exit code (0) and the served responses are identical. The
only difference is one extra stderr warning: `Ignoring llm.misbehavior
because misbehavior is not enabled. ...`
3. **Fixture with invalid JSON tool arguments loaded through `validate`
/ `addFixturesFromJSON`**: 1.44.0 already rejected this input, and this
branch rejects it the same way. The message now ends with a pointer to
`misbehavior: tool-args-invalid-json`.

### Red / green

Every RED was observed on the real failure surface (packed builds driven
over HTTP/WS, the real CLI, and `tsc` consumer projects) before the fix.
Every GREEN was observed afterwards on the same probe.

| Track | RED (before the fix) | GREEN (after the fix) |
|---|---|---|
| #501 invalid tool arguments (ab501: 10 fixtures, 22 HTTP routes in
stream and non-stream modes, 3 WS wires) | 1.44.0 vs pre-fix main: **119
diffs**, 7 on each of the 17 affected wire/mode cells. Controls
identical | 1.44.0 vs fix (default): **260/260 identical** on bytes,
journal and warn count. Audit #501 cells: 30 → 0 (HTTP), 1 → 0 (WS) |
| #508 misbehavior types and WS arity (B7, B10-WS consumer matrix, 15
cells) | Consumer projects fail to compile against main (registry and
4-parameter WS handler breaks) | Same as 1.44.0 in every cell: 0 errors,
except the 4 node10 TS1192 lib errors that 1.44.0 also has |
| #508 `aimock --config` with `llm.misbehavior: "empty-response"`
(fixture `weather` → `sunny`) | 1.44.0 `content="sunny"`, branch before
the fix `content=""` | `content="sunny"` plus the warning. With
`llm.enableMisbehavior: true` or `--misbehavior`: `content=""`. Invalid
value with the opt-in: exit 1 with a rule-prefixed message |
| #511/#513 types (b01–b06, b10) | **14 src errors per cell, 210 in
total** across 15 cells (TS2339 `arguments`, `"function" \| "custom"`,
`CollapseResult.toolCalls`, `resolveFixtureBlocks` arity, ...) | **0 src
errors** in all 15 cells |
| #511/#513 runtime (namespace / custom fixtures F08–F13, F27–F31,
Q01–Q23, loader-ab, record-ns) | Up to 500 on 21/21 routes per fixture,
200 → 400 on Q01–Q06, loader rejects inputs that 1.44.0 accepted, 3/6
recordings differ | Identical to 1.44.0 by default. With
`responsesTools: "extended"`, records `namespace` and `customToolCalls`
|

## Known open item: B12

A consumer object that already has its own **non-fresh** `misbehavior`
key of a different type (for example `misbehavior: string` on a value
assigned to `Fixture`, or `misbehavior: { enabled: true }` on a value
assigned to `MockServerOptions`) no longer assigns to those types: 2 ×
TS2322 per type-proof cell. Adding any optional key has this effect, so
it cannot be avoided without renaming the key. It is the only type proof
that does not compile, and `main` fails it too. This needs a maintainer
decision: accept it, or rename the `misbehavior` key.

## Quality gates (head of this branch)

- `prettier --check .`, `eslint .` and `tsc --noEmit` (`tsconfig.json`,
`tsconfig.test.json`, `scripts/tsconfig.json`) all pass. The `tsdown`
build passes.
- The 65 touched test files, including the strict gate: 3695 passed, 2
skipped (env-gated).
- commitlint over the branch: 0 problems, 0 warnings. All commits are
signed.

## Review fixes (F1, F2, F3, C1, C2, validation)

- **F1**: `match.toolNamespace` is honored only with `responsesTools:
"extended"`. In the default legacy mode it is ignored, as in 1.44.0.
- **F2**: `customToolCalls` and `responsesBlocks` are served only with
`"extended"`. In legacy mode they are dropped on every wire (HTTP, SSE,
Responses WS, Realtime, Gemini Live), as in 1.44.0.
- **F3**: an invalid `responsesTools` (option, constructor or
`llm.responsesTools`) logs one warning and runs `"legacy"` instead of
failing startup. An invalid `--responses-tools` value still exits 1,
because the flag is new.
- **C1**: tests for the `llmock --misbehavior` and `--responses-tools`
pass-through.
- **C2**: a non-boolean `llm.strictToolArguments` or
`llm.enableMisbehavior` logs one warning and the option stays off.
- **Validation**: the validating doors (`validateFixtures`,
`addFixturesFromJSON()`, `POST /__aimock/fixtures`,
`--validate-on-load`, `aimock validate`) check the three keys only in
`"extended"`. New opt-in: **`aimock validate --responses-tools
extended`** (default `legacy` = 1.44.0 parity; an invalid value is a
usage error).
- **aimock-pytest**: new opt-in `AIMockServer(responses_tools=None)`,
which passes `--responses-tools <value>` when set. Unset keeps the
1.44.0 default. The plugin's `toolNamespace` test opts in to
`"extended"`.

| Repro | RED (PR head before the fixes) | GREEN (608a484) |
|---|---|---|
| F1/F2 runtime, default vs 1.44.0 (32 cases + 4 supplementary) | 3/32
identical, 29 differ | **32/32** and **4/4** identical |
| F1/F2 runtime, extended vs PR head extended | n/a | **32/32**
identical (new behavior kept) |
| Validating doors, default vs 1.44.0 (12 inputs x 6 doors) | 20/72
identical, 52 differ | **72/72** identical |
| Validating doors, extended vs PR head | n/a | **48/48** identical |
| F3/C2: `aimock -c` with invalid `responsesTools` / non-boolean flags,
constructor with `responsesTools: "bogus"` | exit 1 / `TypeError`; no
warning for non-boolean flags | starts, one `[aimock] Ignoring ...`
warning each, legacy output identical to 1.44.0 |
| Python plugin suite (CI commands, `AIMOCK_CLI_PATH=dist/cli.js`) | 1
failed (`test_add_fixture_routes_tool_namespace_into_match`: "No fixture
matched"), 247 passed | **248 passed**. Removing the opt-in makes the
test fail again |

Gates at 608a484: full vitest (12 shards) 14,012 passed / 3 skipped;
`package-types-compat.test.ts` 2/2; typecheck, eslint, prettier and
build pass; commitlint clean; all commits signed.

Refs #505 #501 #508 #509 #511 #513


## CI fix: Python Tests (fe5db7c)

All 16 `test (3.10..3.13, 20..26)` jobs failed with `188 failed, 60
passed`. All 188 were in the new
`test_misbehavior_{openai,anthropic,controls}.py` proofs. The pytest
plugin started the CLI without
`--misbehavior`, so faults were ignored, for example
`'{"city":"Paris","units":"metric"}' != '{"city":"Paris",'`,
and `/__aimock/misbehavior` answered `409 Conflict`. No test outside the
misbehavior modules failed, so this is not a
default-behavior regression.

Fix: `AIMockServer(enable_misbehavior=False)` appends `--misbehavior`
when true. The default stays off, as in
1.44.0. The three misbehavior modules override `aimock` with a server
that opts in.

Local proof (Python 3.12, Node 26, `AIMOCK_CLI_PATH=dist/cli.js`, `cd
packages/aimock-pytest && pytest tests/ -v`):

```
RED   (cbcca44): 188 failed, 60 passed, 103 warnings in 71.36s (0:01:11)
GREEN (fe5db7c): 248 passed, 103 warnings in 65.26s (0:01:05)
```

- Bedrock stream default-mode test parses event-stream frames instead of
regex (was flaky on random CRC bytes)

## Review round 2 fixes

- **R2-F1/F2/F3**: in legacy mode, the fixture listing (`GET
/__aimock/fixtures`) and the journal match 1.44.0 byte for byte.
- **Public export parity**: `isContentWithToolCallsResponse`,
`collapseOpenAISSE` and `collapseStreamingResponse` return the 1.44.0
output again (93/93 export probe calls identical to 1.44.0).
- **O7**: when `llm.responsesTools` is invalid, the warning names the
mode that actually runs (`Using "extended" from --responses-tools.`);
with no flag the text is unchanged.
- **record.mcp opt-in**: `llm.record.mcp` is read only with the new
`llm.enableMcpRecording: true`; without it the key is ignored with one
warning, as 1.44.0 ignored it.


## Review round 3 fixes

- **R3-F1**: the stream builders reproduce the 1.44.0 output when
`toolCalls` is not an array.
- **R3-F2**: the `responsesTools` mode is per server, so servers that
share fixtures no longer affect each other.



## Review round 4 fixes

- Default mode keeps caller fixture identity
(addFixture/on/addFixtures/prependFixture) as 1.44.0 did; the per-add
copy happens only with enableMisbehavior
- Extended servers count and journal the caller's fixtures (identity
matches 608a484)
- POST /__aimock/fixtures on an extended server no longer leaks extended
normalization into a shared caller array
- Identity APIs behave the same with enableMisbehavior on
(journal/counts/findByFixture name the caller's fixture; same object
added twice still gives two fault sources)
- Docs: responsesTools mode is fixed at server start
jpr5 added a commit that referenced this pull request Oct 10, 2026
@jpr5
chore: release v1.45.0
c19e178 
Includes #501, #504, #506, #508, #509, #510, #511, #512, #513, #514, #515, #517, #518.

## [1.45.0] - 2026-10-10

Every fixture, config and request that 1.44.0 accepted behaves the same by default. New behavior
that would change those inputs is opt-in.

### Added

- Record live MCP servers into `mcpFakes` files and replay them offline: `--mcp-record
  <mount>=<url>` and `--mcp-proxy-only <mount>=<url>` (`llmock` bin and Docker image),
  `llm.record.mcp` with the opt-in `llm.enableMcpRecording: true` for `aimock --config` (without the
  opt-in, `llm.record.mcp` is ignored with a warning, as 1.44.0 ignored it), and
  `MCPMock.enableRecording()` / `disableRecording()` (#509).
- Recordings never contain headers and redact known secrets (`AIMOCK_RECORD_SECRET_VALUES`,
  `AIMOCK_MCP_UPSTREAM_AUTH`); a file is not written while a secret remains (#509).
- Recorded MCP fakes replay list order, progress and log notifications, and call timing (scaled by
  `--replay-speed`). The recorded `mcpFakes` keys (`list`, `recorded`, `timing`, `notifications`,
  `durationMs`) need 1.45.0 or later (#509).
- Per-test MCP fake report at `GET /__aimock/mcp/fakes/report` (#509).
- Vitest and Jest: `fakesFor()` and `fakesReport()` on a new `AimockFakesHandle` returned by
  `useAimock()`, and opt-in `useAimock({ fakesReport: "warn" | "fail" })` (default `"off"`); Jest
  option `testIdRoot`. New exports `assertFakesReport`, `AimockFakesReportError` and
  `AimockTestIdCollisionError` (#509).
- aimock-pytest 0.7.0: `fakes_for`, `fakes_report` and `assert_fakes_report` (#509), and
  `AIMockServer` opt-ins `enable_misbehavior` and `responses_tools` (#508, #511).
- Workflow-target examples: a Mastra Workflow and a LangGraph JS graph call MCP fakes with no LLM
  (`examples/mcp-workflow-targets`), plus record and report proofs (#509).
- Model misbehavior faults (invalid tool arguments, schema violations, unknown tools, duplicate IDs,
  length stops, empty responses, refusals, content filtering, reasoning-only) across provider wires,
  with seeded rates, `/__aimock/misbehavior` runtime controls, journal evidence and the
  `aimock_misbehavior_total` metric. Off unless enabled with `enableMisbehavior: true`
  (programmatic, including `useAimock`), `llmock --misbehavior` / `aimock validate --misbehavior`,
  or `aimock --misbehavior`, or the config opt-in `llm.enableMisbehavior: true` (an
  `llm.misbehavior` key alone never enables it, as in 1.44.0). Invalid misbehavior settings throw
  the new `MisbehaviorConfigError` (#508).
- Namespaced function calls and custom tool calls on the OpenAI Responses API, over HTTP, SSE and
  WebSocket, plus `match.toolNamespace` (#511). Like the fixture fields below, `match.toolNamespace`
  applies only with `responsesTools: "extended"`; in the default mode it is ignored, as in 1.44.0.
- Responses namespaced and custom tool calls: with `responsesTools: "extended"` (CLI
  `--responses-tools extended`, also on `aimock validate`; config `llm.responsesTools`), tools
  inside `namespace` tools, `custom` tools and `additional_tools` / `tool_search_output` tools are
  visible to `toolName`, `toolNamespace` and predicates, custom tool rounds count for
  `hasToolResult`, `toolCallId` and `turnIndex`, and fixtures can answer with namespaced calls and
  `customToolCalls` / `responsesBlocks`. The default `"legacy"` keeps 1.44.0 behavior. New types:
  `CustomToolCallMessage`, `ResponsesToolsMode` (#511).
- Record mode keeps Responses namespaces and custom tool calls (`customToolCalls`,
  `responsesBlocks`, `CollapseResult.customToolCalls`) in streamed recordings when `responsesTools:
  "extended"` is set (#513).
- Strict tool arguments: with `strictToolArguments` (CLI `--strict-tool-arguments`, config
  `llm.strictToolArguments`), malformed tool-call argument strings are sent unchanged on string
  wires and return an explicit error on object wires. The default keeps 1.44.0 behavior (#501).
- Vertex and Bedrock modeled-contract comparisons for the misbehavior faults in the daily drift
  checks (#508).

### Fixed

- MCP recording checks its config at start, redacts credentials in errors, fails loudly when the
  upstream dies mid-body (502 or a terminated stream, logged and journaled), and defaults `aimock
  --config` with record mounts to `logLevel: "warn"` (#517).
- Kept 1.44.0 default behavior for strict tool arguments, model misbehavior, Responses namespaced
  and custom tools, `llm.record.mcp`, fixture identity and the exported stream helpers, and made
  each new behavior opt-in (#518).
- Kept the published types compatible with 1.44.0: `AimockHandle`, `Mountable` and
  `McpFakeAddOrigin` keep their 1.44.0 shape, and every entry point type-checks with `skipLibCheck:
  false` (#517).
- Align Cohere streaming length-stop faults with captured provider framing (#508).
- Retain drift evidence across retries and for oversized records (#510, #512).
- Classify `claude-haiku-5-5` in the drift model registry (#515).
- Docs: dark-site font smoothing, consistent homepage comparison headings, the test-suite highlight,
  and the copy-button fallback (#504, #506, #514).
jpr5 added a commit that referenced this pull request Oct 10, 2026
@jpr5
chore: release v1.45.0
51c1447 
Includes #501, #504, #506, #508, #509, #510, #511, #512, #513, #514, #515, #517, #518.

## [1.45.0] - 2026-10-10

Every fixture, config and request that 1.44.0 accepted behaves the same by default. New behavior
that would change those inputs is opt-in.

### Added

- Record live MCP servers into `mcpFakes` files and replay them offline: `--mcp-record
  <mount>=<url>` and `--mcp-proxy-only <mount>=<url>` (`llmock` bin and Docker image),
  `llm.record.mcp` with the opt-in `llm.enableMcpRecording: true` for `aimock --config` (without the
  opt-in, `llm.record.mcp` is ignored with a warning, as 1.44.0 ignored it), and
  `MCPMock.enableRecording()` / `disableRecording()` (#509).
- Recordings never contain headers and redact known secrets (`AIMOCK_RECORD_SECRET_VALUES`,
  `AIMOCK_MCP_UPSTREAM_AUTH`); a file is not written while a secret remains (#509).
- Recorded MCP fakes replay list order, progress and log notifications, and call timing (scaled by
  `--replay-speed`). The recorded `mcpFakes` keys (`list`, `recorded`, `timing`, `notifications`,
  `durationMs`) need 1.45.0 or later (#509).
- Per-test MCP fake report at `GET /__aimock/mcp/fakes/report` (#509).
- Vitest and Jest: `fakesFor()` and `fakesReport()` on a new `AimockFakesHandle` returned by
  `useAimock()`, and opt-in `useAimock({ fakesReport: "warn" | "fail" })` (default `"off"`); Jest
  option `testIdRoot`. New exports `assertFakesReport`, `AimockFakesReportError` and
  `AimockTestIdCollisionError` (#509).
- aimock-pytest 0.7.0: `fakes_for`, `fakes_report` and `assert_fakes_report` (#509), and
  `AIMockServer` opt-ins `enable_misbehavior` and `responses_tools` (#508, #511).
- Workflow-target examples: a Mastra Workflow and a LangGraph JS graph call MCP fakes with no LLM
  (`examples/mcp-workflow-targets`), plus record and report proofs (#509).
- Model misbehavior faults (invalid tool arguments, schema violations, unknown tools, duplicate IDs,
  length stops, empty responses, refusals, content filtering, reasoning-only) across provider wires,
  with seeded rates, `/__aimock/misbehavior` runtime controls, journal evidence and the
  `aimock_misbehavior_total` metric. Off unless enabled with `enableMisbehavior: true`
  (programmatic, including `useAimock`), `llmock --misbehavior` / `aimock validate --misbehavior`,
  or `aimock --misbehavior`, or the config opt-in `llm.enableMisbehavior: true` (an
  `llm.misbehavior` key alone never enables it, as in 1.44.0). Invalid misbehavior settings throw
  the new `MisbehaviorConfigError` (#508).
- Namespaced function calls and custom tool calls on the OpenAI Responses API, over HTTP, SSE and
  WebSocket, plus `match.toolNamespace` (#511). Like the fixture fields below, `match.toolNamespace`
  applies only with `responsesTools: "extended"`; in the default mode it is ignored, as in 1.44.0.
- Responses namespaced and custom tool calls: with `responsesTools: "extended"` (CLI
  `--responses-tools extended`, also on `aimock validate`; config `llm.responsesTools`), tools
  inside `namespace` tools, `custom` tools and `additional_tools` / `tool_search_output` tools are
  visible to `toolName`, `toolNamespace` and predicates, custom tool rounds count for
  `hasToolResult`, `toolCallId` and `turnIndex`, and fixtures can answer with namespaced calls and
  `customToolCalls` / `responsesBlocks`. The default `"legacy"` keeps 1.44.0 behavior. New types:
  `CustomToolCallMessage`, `ResponsesToolsMode` (#511).
- Record mode keeps Responses namespaces and custom tool calls (`customToolCalls`,
  `responsesBlocks`, `CollapseResult.customToolCalls`) in streamed recordings when `responsesTools:
  "extended"` is set (#513).
- Strict tool arguments: with `strictToolArguments` (CLI `--strict-tool-arguments`, config
  `llm.strictToolArguments`), malformed tool-call argument strings are sent unchanged on string
  wires and return an explicit error on object wires. The default keeps 1.44.0 behavior (#501).
- Vertex and Bedrock modeled-contract comparisons for the misbehavior faults in the daily drift
  checks (#508).

### Fixed

- MCP recording checks its config at start, redacts credentials in errors, fails loudly when the
  upstream dies mid-body (502 or a terminated stream, logged and journaled), and defaults `aimock
  --config` with record mounts to `logLevel: "warn"` (#517).
- Kept 1.44.0 default behavior for strict tool arguments, model misbehavior, Responses namespaced
  and custom tools, `llm.record.mcp`, fixture identity and the exported stream helpers, and made
  each new behavior opt-in (#518).
- Kept the published types compatible with 1.44.0: `AimockHandle`, `Mountable` and
  `McpFakeAddOrigin` keep their 1.44.0 shape, and every entry point type-checks with `skipLibCheck:
  false` (#517).
- Align Cohere streaming length-stop faults with captured provider framing (#508).
- Retain drift evidence across retries and for oversized records (#510, #512).
- Classify `claude-haiku-5-5` in the drift model registry (#515).
- Docs: dark-site font smoothing, consistent homepage comparison headings, the test-suite highlight,
  and the copy-button fallback (#504, #506, #514).
- Invalid opt-in values never stop startup: an invalid `responsesTools` warns once and runs
  `"legacy"`, and a non-boolean `llm.strictToolArguments`, `llm.enableMisbehavior` or
  `llm.enableMcpRecording` warns once and stays off (#518).
jpr5 added a commit that referenced this pull request Oct 10, 2026
@jpr5
chore: release v1.45.0 (#516)
bc43634 
Release npm/plugin **1.45.0** and **aimock-pytest 0.7.0** from `main` at
c9c19d1 (#518 merged). Every fixture, config and request that 1.44.0
accepted behaves the same by default; new behavior is opt-in (#518).
This branch was rebuilt as one commit on top of the #518 merge,
replacing the earlier release commit bdfaeef.

> **Prepared, not merged.** `publish-release.yml` runs on every push to
`main`. Merging this PR publishes: npm `@copilotkit/aimock@1.45.0`, the
`v1.45.0` tag, a force-move of the `v1` major tag, the GitHub Release,
the Docker images (`ghcr.io/copilotkit/aimock` and
`ghcr.io/copilotkit/llmock`, `:1.45.0` and `:latest`), `aimock-pytest`
0.7.0 on PyPI and the Slack announcement. The docs site deploys from
`main:/docs` on merge. Merge only when the release is authorized. Do not
dispatch `publish-release.yml` from this branch.

## Versions

| Surface | Before | After |
|---|---|---|
| `package.json` `version` | 1.44.0 | 1.45.0 |
| `.claude-plugin/plugin.json` `version` | 1.44.0 | 1.45.0 |
| `.claude-plugin/marketplace.json` `source.version` | ^1.44.0 | ^1.45.0
|
| `charts/aimock/Chart.yaml` `appVersion` (chart `version` stays 0.1.0)
| 1.44.0 | 1.45.0 |
| `docs/index.html` CLI banner | aimock v1.44.0 | aimock v1.45.0 |
| `packages/aimock-pytest/pyproject.toml` `version` | 0.6.0 | 0.7.0 |
| `packages/aimock-pytest/src/aimock_pytest/_version.py`
`AIMOCK_VERSION` | 1.44.0 | 1.45.0 |
| `packages/aimock-pytest/README.md` `--aimock-version` default | 1.44.0
| 1.45.0 |
| `skills/write-fixtures/SKILL.md` `toolNamespace` pin note | "newer
than its default 1.44.0 pin" | "aimock 1.45.0+, its default pin from
0.7.0" |

"Unreleased" wording now names 1.45.0 in `README.md` (misbehavior note),
`docs/cohere/index.html` (length-stop fidelity) and
`docs/model-misbehavior/index.html` (two places).

The pytest pin bump matters: the new
`AIMockServer(enable_misbehavior=..., responses_tools=...)` options pass
`--misbehavior` / `--responses-tools`, which the 1.44.0 CLI does not
accept. With the pin at 1.45.0, default-download users get a server that
supports them.

**1.45.0 is a minor.** With #518, default behavior and the published
types match 1.44.0. The remaining type additions (the optional
`misbehavior` key, `ChatCompletionRequest.customTools`, `useAimock`
returning `AimockFakesHandle`) were accepted by the maintainer as
additive optional typing (minor).

## CHANGELOG

## [1.45.0] - 2026-10-10

Every fixture, config and request that 1.44.0 accepted behaves the same
by default. New behavior that would change those inputs is opt-in.

### Added

- Record live MCP servers into `mcpFakes` files and replay them offline:
`--mcp-record <mount>=<url>` and `--mcp-proxy-only <mount>=<url>`
(`llmock` bin and Docker image), `llm.record.mcp` with the opt-in
`llm.enableMcpRecording: true` for `aimock --config` (without the
opt-in, `llm.record.mcp` is ignored with a warning, as 1.44.0 ignored
it), and `MCPMock.enableRecording()` / `disableRecording()` (#509).
- Recordings never contain headers and redact known secrets
(`AIMOCK_RECORD_SECRET_VALUES`, `AIMOCK_MCP_UPSTREAM_AUTH`); a file is
not written while a secret remains (#509).
- Recorded MCP fakes replay list order, progress and log notifications,
and call timing (scaled by `--replay-speed`). The recorded `mcpFakes`
keys (`list`, `recorded`, `timing`, `notifications`, `durationMs`) need
1.45.0 or later (#509).
- Per-test MCP fake report at `GET /__aimock/mcp/fakes/report` (#509).
- Vitest and Jest: `fakesFor()` and `fakesReport()` on a new
`AimockFakesHandle` returned by `useAimock()`, and opt-in `useAimock({
fakesReport: "warn" | "fail" })` (default `"off"`); Jest option
`testIdRoot`. New exports `assertFakesReport`, `AimockFakesReportError`
and `AimockTestIdCollisionError` (#509).
- aimock-pytest 0.7.0: `fakes_for`, `fakes_report` and
`assert_fakes_report` (#509), and `AIMockServer` opt-ins
`enable_misbehavior` and `responses_tools` (#508, #511).
- Workflow-target examples: a Mastra Workflow and a LangGraph JS graph
call MCP fakes with no LLM (`examples/mcp-workflow-targets`), plus
record and report proofs (#509).
- Model misbehavior faults (invalid tool arguments, schema violations,
unknown tools, duplicate IDs, length stops, empty responses, refusals,
content filtering, reasoning-only) across provider wires, with seeded
rates, `/__aimock/misbehavior` runtime controls, journal evidence and
the `aimock_misbehavior_total` metric. Off unless enabled with
`enableMisbehavior: true` (programmatic, including `useAimock`), `llmock
--misbehavior` / `aimock validate --misbehavior`, or `aimock
--misbehavior`, or the config opt-in `llm.enableMisbehavior: true` (an
`llm.misbehavior` key alone never enables it, as in 1.44.0). Invalid
misbehavior settings throw the new `MisbehaviorConfigError` (#508).
- Namespaced function calls and custom tool calls on the OpenAI
Responses API, over HTTP, SSE and WebSocket, plus `match.toolNamespace`
(#511). Like the fixture fields below, `match.toolNamespace` applies
only with `responsesTools: "extended"`; in the default mode it is
ignored, as in 1.44.0.
- Responses namespaced and custom tool calls: with `responsesTools:
"extended"` (CLI `--responses-tools extended`, also on `aimock
validate`; config `llm.responsesTools`), tools inside `namespace` tools,
`custom` tools and `additional_tools` / `tool_search_output` tools are
visible to `toolName`, `toolNamespace` and predicates, custom tool
rounds count for `hasToolResult`, `toolCallId` and `turnIndex`, and
fixtures can answer with namespaced calls and `customToolCalls` /
`responsesBlocks`. The default `"legacy"` keeps 1.44.0 behavior. New
types: `CustomToolCallMessage`, `ResponsesToolsMode` (#511).
- Record mode keeps Responses namespaces and custom tool calls
(`customToolCalls`, `responsesBlocks`, `CollapseResult.customToolCalls`)
in streamed recordings when `responsesTools: "extended"` is set (#513).
- Strict tool arguments: with `strictToolArguments` (CLI
`--strict-tool-arguments`, config `llm.strictToolArguments`), malformed
tool-call argument strings are sent unchanged on string wires and return
an explicit error on object wires. The default keeps 1.44.0 behavior
(#501).
- Vertex and Bedrock modeled-contract comparisons for the misbehavior
faults in the daily drift checks (#508).

### Fixed

- MCP recording checks its config at start, redacts credentials in
errors, fails loudly when the upstream dies mid-body (502 or a
terminated stream, logged and journaled), and defaults `aimock --config`
with record mounts to `logLevel: "warn"` (#517).
- Kept 1.44.0 default behavior for strict tool arguments, model
misbehavior, Responses namespaced and custom tools, `llm.record.mcp`,
fixture identity and the exported stream helpers, and made each new
behavior opt-in (#518).
- Kept the published types compatible with 1.44.0: `AimockHandle`,
`Mountable` and `McpFakeAddOrigin` keep their 1.44.0 shape, and every
entry point type-checks with `skipLibCheck: false` (#517).
- Align Cohere streaming length-stop faults with captured provider
framing (#508).
- Retain drift evidence across retries and for oversized records (#510,
#512).
- Classify `claude-haiku-5-5` in the drift model registry (#515).
- Docs: dark-site font smoothing, consistent homepage comparison
headings, the test-suite highlight, and the copy-button fallback (#504,
#506, #514).
- Invalid opt-in values never stop startup: an invalid `responsesTools`
warns once and runs `"legacy"`, and a non-boolean
`llm.strictToolArguments`, `llm.enableMisbehavior` or
`llm.enableMcpRecording` warns once and stays off (#518).

## Release checklist

- [x] Opt-in names for #501, #508, #511, #513 match the merged compat PR
#518 (`strictToolArguments` / `--strict-tool-arguments` /
`llm.strictToolArguments`; `enableMisbehavior` / `--misbehavior` /
`aimock validate --misbehavior` / `llm.enableMisbehavior`;
`responsesTools` / `--responses-tools` / `aimock validate
--responses-tools` / `llm.responsesTools`; `llm.enableMcpRecording`;
pytest `enable_misbehavior` / `responses_tools`), checked against the
source on this branch.
- [x] Pytest pin `AIMOCK_VERSION` = `"1.45.0"` and the README
`--aimock-version` default = 1.45.0.
- [x] aimock-pytest `version` 0.6.0 -> 0.7.0.
- [x] Homepage "Requires 1.45.0+" (MCP scenario fakes card) and the
other "1.45.0 or later" docs notes match this release.
- [x] Version surfaces bumped (table above). `git grep -F 1.44.0`: every
remaining hit is a history comparison ("as in 1.44.0", "up to 1.44.0",
the 1.44.0 CHANGELOG section) or recorded `aimockVersion` provenance in
fixtures/examples.
- [x] "Unreleased" wording removed from README and docs.
- [x] `src/__tests__/package-types-compat.test.ts` on the release head:
2/2 pass.
- [ ] 1.44.0 A/B probe re-run on the release head: not re-run here; this
commit changes only version strings, docs text and the CHANGELOG, and
#518 carries the A/B proof on the same code.

## Local gates (release head)

- `pnpm install --frozen-lockfile`, `format:check`, `lint`, `typecheck`,
`build`, `test:exports` (publint + attw): pass
- Full vitest, 12 shards: 14,069 passed, 3 skipped (env-gated), 0 failed
- aimock-pytest, installed as CI does (hash-pinned hatchling,
`--no-build-isolation`), Python 3.12, `AIMOCK_CLI_PATH=dist/cli.js`: 248
passed
- `npm pack --dry-run`: `copilotkit-aimock-1.45.0.tgz`, 766 files
(1.44.0: 722), dist/, fixtures/, skills/, .claude-plugin/, README,
CHANGELOG, LICENSE; no src or tests
- commitlint: 0 problems, 0 warnings. Commit is SSH-signed.
q-qp-p pushed a commit to q-qp-p/copilotkit-aimock that referenced this pull request Oct 19, 2026
@jpr5
fix: serve invalid tool arguments as {} by default, strict behind an …
9297a0e 
…opt-in

1.44.0 logged a warning and served fixture tool-call arguments that are not
valid JSON as {} (or "{}" on Cohere and the streamed Anthropic, Bedrock,
Converse and Gemini Interactions deltas).

CopilotKit#501 changed that to a 500 aimock_invalid_tool_arguments on object wires and
the raw text on string wires, which breaks consumers that relied on 1.44.0.

Restore the 1.44.0 output on every affected wire and stream mode, including
the warning on Ollama, Gemini Interactions and Gemini Live, and move the CopilotKit#501
behavior behind the new strictToolArguments option (CLI
--strict-tool-arguments, config llm.strictToolArguments). The setting reaches
the response builders through a per-request scope, so builder signatures stay
as they are. The misbehavior planner skips a fixture whose invalid arguments
the default replaces, and an applied fault keeps its own malformed arguments
as authored.
q-qp-p pushed a commit to q-qp-p/copilotkit-aimock that referenced this pull request Oct 19, 2026
@jpr5
test: turn on misbehavior in the invalid tool arguments interplay tests
e47fa3a 
Misbehavior is opt-in after the CopilotKit#508 compat fix, so the two CopilotKit#501 interplay tests must enable it.
Without it the fixture misbehavior key is ignored and the tool-args-invalid-json case serves {}.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jpr5