Skip to content

fix(claude): don't execute a tool call cut off by max_tokens - #720

Open
Mike-7777777 wants to merge 1 commit into
hud-evals:mainfrom
Mike-7777777:fix/claude-truncated-tool-call
Open

Mike-7777777 wants to merge 1 commit into
hud-evals:mainfrom
Mike-7777777:fix/claude-truncated-tool-call

Conversation

@Mike-7777777

@Mike-7777777 Mike-7777777 commented Oct 14, 2026 •

Copy link
Copy Markdown

Problem

When Claude hits max_tokens in the middle of a tool_use block, ClaudeAgent still runs that call with whatever part of the arguments arrived.

The anthropic SDK's stream accumulator parses input_json_delta with jiter.from_json(..., partial_mode=True), which drops the unfinished trailing value and returns a well-formed dict. Feeding the real SDK stream (over httpx.MockTransport, no network) the input

{"command": "rm -rf build/tmp", "args": ["--keep", "src", "te

followed by stop_reason: "max_tokens" gives, on main:

block.input: {'command': 'rm -rf build/tmp', 'args': ['--keep', 'src']}

message_to_agent_step turns that into a dict-argument MCPToolCall and _dispatch_call executes it. The default stop_on is empty, so nothing else stops it.

Fix

Reuse the contract from #480: a str in MCPToolCall.arguments means the arguments arrived incomplete, and _dispatch_call answers it with an error result instead of running the tool. In message_to_agent_step, when stop_reason == "max_tokens" and the tool_use is the last content block, its arguments become json.dumps(block.input). Earlier tool_use blocks in the same message stay dicts and run. The comment on MCPToolCall.arguments now mentions this case.

What follows from the existing contract:

  • The model gets the "arrived incomplete ... Re-issue the call in full" tool result and can retry.
  • stop_on={"malformed_tool_call"} now also ends a Claude rollout at that turn, as it already does for the OpenAI paths.
  • The assistant turn replayed to the API still carries response.content, so the tool_use input stays an object.

Limitations

  • A trailing tool_use that was actually complete when max_tokens hit is also refused; the model is asked to re-issue it. Telling the two apart would need the raw JSON buffer, which the SDK only keeps on an undocumented attribute of the stream snapshot.
  • The non-streaming Bedrock path goes through the same message_to_agent_step, so the rule applies there too. Only the streaming path is tested; I have not checked what a real non-streamed max_tokens response puts in a cut-off tool_use.

Test

test_tool_use_cut_off_by_max_tokens_is_not_executed runs ClaudeAgent through __call__ against a FastMCP tool, with the real AsyncAnthropic stream accumulator over httpx.MockTransport. The first turn streams a complete tool_use and then a cut-off one, ending in max_tokens. The test asserts that only the complete call runs and that the next request carries an is_error tool result saying "arrived incomplete" for the cut-off one. It fails on main (both calls run) and passes with the fix.

ruff format --check, ruff check, ty check --error-on-warning (dev, train, modal, daytona extras) and pytest -q with all extras pass locally.

Overlap with #700

#700 deletes hud/agents/tests/test_claude_agent.py and moves agent tests to tests/agents/, so this test will conflict with it. Whichever lands second, the test belongs next to the existing cut-off-arguments case in tests/agents/test_faults.py. The test adds no findings under #700's scripts/check_tests.py. #700's changes to hud/agents/claude/agent.py don't touch message_to_agent_step.


Note

Medium Risk
Changes which Claude tool calls execute at the token limit; misclassifying a complete trailing call would skip execution, but the change reduces risk from running partially parsed shell arguments.

Overview
Stops Claude from running tool calls whose arguments were truncated when the model hits max_tokens.

In message_to_agent_step, when stop_reason == "max_tokens" and the tool_use block is the last content block, arguments are passed as json.dumps(block.input) instead of the SDK’s leniently parsed dict. That reuses the existing rule that string MCPToolCall.arguments are never executed—_dispatch_call returns an error tool result (“arrived incomplete”) so the model can retry. Earlier tool_use blocks in the same message still use dict arguments and run normally.

MCPToolCall.arguments is documented for this token-cap case. A new end-to-end test drives the real AsyncAnthropic stream accumulator over mocked SSE with one complete and one cut-off bash call, asserting only the complete command runs and the truncated call gets is_error.

Reviewed by Cursor Bugbot for commit 2efabd8. Bugbot is set up for automated code reviews on this repo. Configure here.

The SDK stream accumulator parses a truncated tool_use input in partial
mode and returns a plausible dict, so a call cut off by max_tokens was
dispatched with incomplete arguments. Pass the trailing tool_use's input
as a str, the form _dispatch_call already refuses to execute (hud-evals#480), and
document that use on MCPToolCall.arguments.
@Mike-7777777
Mike-7777777 force-pushed the fix/claude-truncated-tool-call branch from ba4d766 to 2efabd8 Compare October 14, 2026 12:41
@Mike-7777777
Mike-7777777 marked this pull request as ready for review October 14, 2026 12:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant